docs(benchmarks): the benchmark charts read a published JSON table of measurements and label our engine specsolve

[FBumann]

Prompt: Could we radically simplify and improve it by using tanstack charts 1.0.0 ?

Prompt: And leverage the descriptive nature fully! We should improve our data model around it probably! And use json etc — I would like to have a clean, simple data model which is used by tanstack charts, without too much data manipulation. — we shoud name it specsolve!

Note

The following content was generated by AI.

bench.plot writes benchmarks-scaling.json: tidy rows that TanStack Charts marks read by field name. A JSON spec on the page declares the charts. This replaces about 250 lines of hand-written SVG, and the page labels our engine specsolve. The rows hold the numbers re-measured on 0.6.0 in #1850.

The data model and the spec

One row per model, sink, ladder, rung and library. A measurement carries numbers. A refusal carries the budget label instead:

{"model": "dispatch", "sink": "gurobi", "ladder": "length", "rung": "xs", "variables": 10000, "library": "gurobipy-loop", "wall_s": 0.0235, "wall_q1_s": 0.0225, "wall_q3_s": 0.0239, "peak_gb": 0.1998}
{"model": "fleet", "sink": "gurobi", "ladder": "length", "rung": "m", "variables": 1200000, "library": "pyomo", "refused": ">30 s"}

variables is the rung's declared size (results.nominal). It equals the column count in every committed cell, so every library shares one x per rung. The page passes these rows straight to areaY (y1: wall_q1_s, y2: wall_q3_s) and lineY (y: wall_s, z/color: library). The only manipulation is filtering the rows to one facet. library is the harness arm name as it is.

The page spec is <script type="application/json" id="spec">. It holds facet: ["model", "sink"], the x field, one entry per metric with its y, band and format, the initial filter, and the series colour domain and range. The control buttons name a metric or a filter value from it. About 60 lines of JS turn the spec into TanStack definitions. The rest of the JS is the table, the legend and the controls. Formats are named in the spec because a function is not JSON.

bench.plot no longer edits the HTML. The page is purely hand-written, and the JSON is the generated file.

Loading TanStack without a bundler

dist/ is unbundled ESM with bare d3-* imports. An import map points @tanstack/charts/ at the raw dist/ files on jsdelivr, pinned to 1.0.0, so all the entry points share one module graph. d3-shape and d3-scale map to their pinned +esm builds, and d3-scale provides scaleLog. The six entry points the page imports pull 47 files, about 330 KB before compression, and 48 CDN requests in total. MathJax already loads from unpkg, so a runtime CDN has precedent here. Nothing joins the pixi or Python dependency sets.

What was verified, and what was not

• Parity: I compared every cell of the rows bench.plot writes from the results committed in docs: the published benchmarks are re-measured on specsolve 0.6.0 #1850 with the const DATA that docs: the published benchmarks are re-measured on specsolve 0.6.0 #1850 wrote into the old page on main (79258b1). All 112 cells are identical, measurements and refusals both, with polars read as specsolve.
• Browser: headless Chromium against docs/about/ and against the strict-built site/about/, in light and dark. 5 panels, the tooltip, the metric switch, the width ladder and hiding a library all work, with no console errors.
• Not checked: I could not load jsdelivr itself, because the session proxy blocks it. The test served the same files from the npm tarball: the raw dist/ unchanged, and an esbuild bundle in place of each d3 +esm URL. The first look at the deployed page is the check for that.
• Gates: on the branch rebased onto main, pixi run lint, format-check and typecheck are clean, the bench harness tests pass (266), and docs-build --strict passes and ships the JSON beside the page. CI on the previous head was green.

Mutation table for plot.rows

tools.mutate took the line deletions, with --tests bench/test_harness.py and CI=1 to pass the load guard: the run times nothing, and the load was my own earlier test run. Hand mutations on the refusal condition used the same three precautions. Line numbers are from before the name map was removed.

mutation	result
the emit-phase filter (plot.py:72)	caught
the no-peak filter (plot.py:73)	caught
the ladder-rung filter (plot.py:74)	caught by the new test_a_rung_neither_ladder_plots_is_left_out. It was green before that test.
the size-on-axis check dropped (line 84, by hand)	caught
the already-measured check dropped (line 84, by hand)	caught by the new test_a_rung_measured_past_a_ceiling_is_a_measurement_not_a_refusal. It was green before that test.
the 'error' not in r filter	It survived deletion. results.py says nothing produces an error record, so it is deleted.

Coverage moved, defaults departed from, and what was not done

• Tests: the five tests on plot.series and plot.panels now assert the same claims on plot.rows, through a _refused(records, library) helper. _plotted is gone.
• Type: the diff touches bench/, docs/, .github/ and pyproject.toml. By the table, the topmost row is docs/. The changelog line is added.
• Stacked on this: docs(benchmarks): the benchmark page highlights specsolve, projects the sizes a budget refused, and compares every library with it in one table #1855 redesigns the page's charts on top of this data model.
• Not done: no new measurements here. I also left the footer's stale references to bench/run.py and docs/benchmarks.md as they were. They predate this PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01K42KwZJtqPPhf5hDzUD91o

[read-the-docs-community]

Documentation build overview

📚 specsolve | 🛠️ Build #34973521 | 📁 Comparing 49cf2be against latest (aa90239)

  🔍 Preview build  

3 files changed

± about/benchmarks-scaling.html
± about/benchmarks/index.html
± about/changelog/index.html

[FBumann]

Note

The following content was generated by AI.

suite fails on two tests in tests/test_resolution_parity.py. The cause is polars 2.0.0, which was published to PyPI today at 11:44 UTC. This PR does not cause it.

• The failure: test_the_empty_coordinate_builds_on_both_lanes and test_a_masked_scalar_variable_takes_its_row_with_it[masked-in] get an empty frame from dual('budget_row') for a constraint row with no dimensions.
• Why every fresh solve gets 2.0: the requirement is polars>=1.30 and pixi.lock is gitignored.
• Why it is not this PR's: the diff touches nothing under src/ or tests/. The same 2 tests fail on an untouched main (aa90239) with polars 2.0.0. With polars 1.44.2 in the same environment, all 30 tests in that file pass. linopy returns the dual for that row correctly (1.0), so the empty frame comes from specsolve's own read.
• Fix: none exists yet. I did not port one into this PR, because a dependency cap is outside its scope. A stopgap is polars>=1.30,<2 in pyproject.toml (both the dependency list and the pixi table). The lasting fix is to read a dimensionless row's duals in the way polars 2.0 requires.

Reproduction

pixi run pytest tests/test_resolution_parity.py -q   # polars 2.0.0: 2 failed, 28 passed
uv pip install --target /tmp/pl1 'polars==1.44.2'
PYTHONPATH=/tmp/pl1 pixi run pytest tests/test_resolution_parity.py -q   # 30 passed

---

Generated by Claude Code

[codspeed]

Merging this PR will not alter performance

✅ 67 untouched benchmarks
⏩ 307 skipped benchmarks¹

---

_{Comparing claude/exciting-ritchie-ap020w (49cf2be) with main (85d864a)²}

Footnotes

1. 307 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on main (79258b1) during the generation of this report, so 85d864a was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩