docs: the published benchmarks are re-measured on specsolve 0.6.0

[FBumann]

Prompt: The benchmark ran. Can you update our published benchmark with a PR?

Note

The following content was generated by AI.

The benchmark pages now show run 37298997947, measured on aa902393 (0.6.0). It covers the same five cases as the page it replaces. The page's sentences about the run are recomputed from the new files, and bench/reproduce.py is pinned to that run.

What changed

• Results. Five cases: highs dispatch and fleet, and gurobi dispatch, transport and fleet. The other three cases were stopped by the memory watchdog, as on the previous run (the OOM record dies with the run artifact instead of landing beside the numbers it explains #1498).
• Where the files come from. The network policy of the session blocks the artifact host, so the files come from the eight per-case artifacts, uploaded by hand. Every results file is byte-identical in each artifact that holds it. results-gurobi-fleet holds all ten files.
• casualties.json is not committed. It was never committed, and bench/results.py skips it. Its three entries are in the page text.
• Generated content. The tables come from pixi run -e bench report and the chart data from pixi run -e bench plot. Their output, fingerprint included, is the same as the run's own report and plot steps.
• Hand-written sentences, recomputed. Before I used each method on the new files, I checked that it reproduces the old sentence exactly from main's results.
  • Mean against median: 14 cells are above 1.10x, against 4 before. The worst is 3.41x, against 1.21x before.
  • The cell the median flips: now transport/w1 on gurobi against gurobipy-loop, in specsolve's favour. Before, it was dispatch/s against linopy, against specsolve. With it goes the the gurobi sink alternates between a fast and a slow build, round after round #1288 sentence: specsolve's rounds on that cell are now 88 to 94 ms.
  • The three casualties: 24.6, 26.3 and 23.9 GB.
  • Removed: the sentence on why storage survived on gurobi earlier. The previous run already contradicted it.
• bench/reproduce.py. test_the_lock_installs_what_the_published_numbers_were_taken_on failed on the new files, because the lock installed lpspec at 8d27e88b. Changes:
  • The script now installs specsolve[gurobi] at aa90239393, and linopy from master, because the linopy extra is gone (refactor(api): specsolve no longer builds a linopy model, and its linopy extra and LaneError are gone #1755).
  • The lock moves pandas to 3.0.6, as measured.
  • The docstring line about the lpspec 0.0.1a61 numbers is removed.
• main is merged in, with fix(deps): specsolve installs a polars older than 2.0, on which a row's dual can come back empty and a model can take nine times the memory #1852's polars cap. The cap was ported here first, and the merge makes it identical to main. The only conflict was in CHANGELOG.md, which now keeps both lines.

Found, and not fixed here

• A cold first round. In 13 cells, the first of the nine rounds of an xs cell on specsolve, linopy or pyomo is 1.9 to 22.6 times the median of the other eight. Pyomo dispatch/xs on highs takes 1524 ms, then about 67 ms. main's results from 14 September have no such round. Published medians do not move, because that round is always the largest of nine. The q3 of a band may move a little. _rounds had warmup_rounds=0 at the previous run too, so the cause is elsewhere in what changed after 8d27e88b. I did not look further.
• The 84-rounds sentence. benchmarks-scaling.html says "a quick cell here took 84 rounds and a slow one 9". That sentence is hand-written and was already false, because rounds are pinned at 9.
• linopy's commit. The run does not record which linopy commit it measured, so the lock pins today's master (f665a260). The guard checks only the versions the run records.

Gates

• pixi run check, before the main merge, on polars 1.44.2: 4996 passed, 564 skipped, 1 xfailed. After the merge: tools.changelog check, tests/test_changelog.py and tests/test_tooling_pins.py pass.
• pytest bench/test_harness.py: 262 passed, 16 skipped. This is test-bench, including test_the_report_renders_from_the_committed_results and the lock guard.
• pixi run docs-build, the command Read the Docs runs: clean.
• uv lock --script bench/reproduce.py --check: clean. pixi run lint and format-check: clean.
• Not run: uv run --locked bench/reproduce.py itself.

🤖 Generated with Claude Code

https://claude.ai/code/session_018nKGXp8uKnMeJACjFYwbdY

[read-the-docs-community]

Documentation build overview

📚 specsolve | 🛠️ Build #34972480 | 📁 Comparing 3040c5f against latest (aa90239)

  🔍 Preview build  

2 files changed

± about/benchmarks/index.html
± about/changelog/index.html

[FBumann]

Note

The following content was generated by AI.

Two signals on this PR come from polars 2.0.0, not from this PR: the red suite job and CodSpeed's memory regression. The fix is #1852, which caps polars at <2. It is ported here as dce68af8, which changes pyproject.toml and uv.lock. The port adds nothing once #1852 is on main.

polars 2.0.0 was released on PyPI on 2026-10-06 at 11:51 UTC. pyproject.toml asked only for polars>=1.30, and there is no lock, so CI and CodSpeed both installed 2.0.0.

• suite. Two tests fail:

  • tests/test_resolution_parity.py::test_the_empty_coordinate_builds_on_both_lanes
  • tests/test_resolution_parity.py::test_a_masked_scalar_variable_takes_its_row_with_it[masked-in]

  In both, dual('budget_row') returns 0 rows where the tests expect 1.

• CodSpeed. The storage/s specsolve cells go from about 17 MB to about 776 MB. On the same tree, test_emit[storage-s-specsolve-highs] with --benchmark-memory measures:

polars	peak_bytes	RSS
1.44.2	119–123 MB	234 MB
2.0.0	1091–1146 MB	1072 MB

How it was checked

• Tree 2de8fbc3, with linopy master at f665a260, which is what CI installs.
• The test run and the memory run were each taken twice, with only polars swapped between them.
• The memory figures are three rounds of one local run. They are not comparable in absolute terms with CodSpeed's instrument, only in direction and size.
• After the port, pixi run check passes on polars 1.44.2: 4996 passed, 564 skipped, 1 xfailed.

---

Generated by Claude Code

[codspeed]

Merging this PR will not alter performance

✅ 67 untouched benchmarks
⏩ 307 skipped benchmarks¹

---

_{Comparing claude/gifted-darwin-b7918s (3040c5f) with main (331a839)²}

Footnotes

1. 307 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on main (85d864a) during the generation of this report, so 331a839 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩