Skip to content

docs(benchmarks): the benchmark charts and their method are one page of the docs, in the site's theme - #1860

Merged
FBumann merged 4 commits into
mainfrom
claude/exciting-ritchie-ap020w
Oct 6, 2026
Merged

FBumann merged 4 commits into
mainfrom
claude/exciting-ritchie-ap020w

Conversation

@FBumann

@FBumann FBumann commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Prompt: could we include the charts etc into the regular docs? Or does it need to be a standalone html page like it it right now? — Yes merge them

Note

The following content was generated by AI.

The benchmark charts move from the standalone benchmarks-scaling.html into about/benchmarks.md. That page now also holds the method, so there is one "Benchmarks" entry in the nav. The charts follow the site's light/dark toggle and work with instant navigation.

How the charts live inside the docs
  • Import map. overrides/main.html (theme.custom_dir) adds the TanStack/d3 import map to the head of every page. It has to be every page because instant navigation never re-renders the head: a reader who reaches the benchmark page from another page keeps the first page's head. A map costs nothing until a module asks for it.
  • Script. docs/javascripts/benchmarks.js is loaded through extra_javascript as type: module. On each document$ emission it destroys the previous page's charts. It then imports TanStack on demand, only when the page has a #benchmark-charts block, so no other page fetches the library. It stops if the reader has navigated away while the library loads.
  • Page. The controls, panels, table and JSON spec are one HTML block in the Markdown. The prose around them is Markdown: a purpose line, the model list, "Reading the charts", then the method sections that were benchmarks.md.
  • Theme. docs/stylesheets/benchmarks.css takes surfaces and text from the theme's --md-* variables and switches the series and table colours under [data-md-color-scheme="slate"], so the block follows the site's toggle. The standalone page only followed the operating-system setting. The panels sit two to a row in the narrower content column.
  • Data. benchmarks-scaling.json is renamed benchmarks.json, after its page. bench/plot.py, the published-benchmark workflow artifact list, docs/README.md and bench/README.md follow.
  • Links. The README card, the landing-page card icon selector in extra.css, the About index row and the two in-page links now point at about/benchmarks/. bench/report.py drops its link to the old chart page.
Prose
  • The old chart page said "a quick cell here took 84 rounds and a slow one 9", which contradicts the method page's current "every measurement gets the same nine rounds, pinned". That sentence is dropped, and the median section now opens with the nine-round rule.
  • The explanation of the length/width switch was an HTML comment on the old page. It is now a bullet in "Reading the charts".
  • The About index row said the page measures "build and solve cost against linopy". It measures build cost only, against four other ways of writing the model, and now says so.
  • Measured with the docs-writing script: 86 sentences, median 17 words, 18 over 25. The old method page: 54, median 17, 11 over 25.
What was verified, and what was not
  • Gates: pixi run check passes on 53ad43d (4999 passed). pixi run docs-build (strict), pixi run docs-test (112) and tests/test_docs_site.py + tests/test_doc_examples.py (140) pass. The bench harness tests pass (266).
  • Browser, on the built site: headless Chromium, with the jsdelivr files served from the npm tarball. Each check was run twice, once on a direct load and once after arriving by instant navigation from the About index.
    • The page draws 5 panels and 20 table rows.
    • Each toggle is asserted by its effect. Width gives 1 panel and 4 rows. Peak RSS gives axes in GB. Linear–linear gives a zero tick. Hiding pyomo in the legend removes its table column. The pressed state follows each click.
    • The dots read their computed colours. Every measured dot has a ring in the surface colour, and every projection ring is filled with the surface colour.
    • Leaving for another page leaves no charts or tooltips behind, and Back redraws them.
    • No other page fetched from jsdelivr before the benchmark page was opened.
    • Both themes follow the site scheme.
    • Console errors came only from MathJax, Mermaid, fonts and the GitHub API, whose hosts the test blocks. None came from the charts.
  • Two defects shipped in earlier commits of this PR, and both came from names carried over from the standalone page.
    • Dead toggles. The click handler still matched the old .seg class. My first browser test counted panels and rows, which stay the same across the scale and metric toggles, so it missed this. e9a71bb fixes it.
    • Dots without rings, and black projection rings. The dots used --surface-1, a token that the docs stylesheet does not define. 53ad43d switches them to --surface.
    • The toggle and dot-colour checks above are the ones that would have caught these defects.
  • Not checked: the real jsdelivr load, as in docs(benchmarks): the benchmark charts read a published JSON table of measurements and label our engine specsolve #1851 and docs(benchmarks): the benchmark page highlights specsolve, projects the sizes a budget refused, and compares every library with it in one table #1855.
What was not done
  • No redirect from the old URL. about/benchmarks-scaling.html now returns 404, under the no-compatibility rule.
  • A stale anchor left alone. bench/README.md links a #the-density-sweep-and-a-claim-it-refuses anchor that predates this PR and no longer exists. It is unchanged here.
  • The label, tooltip and polish changes discussed in the session are left for a follow-up.

🤖 Generated with Claude Code

https://claude.ai/code/session_01K42KwZJtqPPhf5hDzUD91o

claude added 2 commits October 6, 2026 21:05
…of the docs, in the site's theme

The standalone benchmarks-scaling.html is gone. about/benchmarks.md carries the
charts, a section on reading them, and the method that was its own page. The
chart code moves to docs/javascripts/benchmarks.js, loaded as a module on every
page and drawing only where the page has a #benchmark-charts block, on each
document$ emission so instant navigation redraws and clears it. The import map
moves into an overrides/main.html head block. Colours take the theme's surface
and text variables, so the block follows the site's light and dark toggle. The
rows file is renamed benchmarks.json after its page.

Prose, measured with the docs-writing script: 86 sentences, median 17 words,
18 over 25 (the old method page: 54, median 17, 11 over 25).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K42KwZJtqPPhf5hDzUD91o
@read-the-docs-community

read-the-docs-community Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Documentation build overview

📚 specsolve | 🛠️ Build #34979813 | 📁 Comparing 53ad43d against latest (99462fe)

  🔍 Preview build  

4 files changed
± about/index.html
± about/benchmarks/index.html
± about/changelog/index.html
- about/benchmarks-scaling.html

The click handler still matched the standalone page's `.seg` class, while the
docs markup names its groups `bench-seg`, so no toggle reached it. The legend
worked because it matches by id.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K42KwZJtqPPhf5hDzUD91o
@codspeed

codspeed Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 67 untouched benchmarks
⏩ 307 skipped benchmarks1


Comparing claude/exciting-ritchie-ap020w (53ad43d) with main (99462fe)

Open in CodSpeed

Footnotes

  1. 307 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

…ollow again

The dots took their ring and fill from `--surface-1`, a token of the
standalone page that the docs stylesheet does not define. Chromium then
drew the measured dots without a ring and filled the projection rings
black. They now use `--surface`, which follows the site theme.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K42KwZJtqPPhf5hDzUD91o
@FBumann
FBumann enabled auto-merge (squash) October 6, 2026 21:21
@FBumann
FBumann merged commit 7af455a into main Oct 6, 2026
15 checks passed
@FBumann
FBumann deleted the claude/exciting-ritchie-ap020w branch October 6, 2026 21:22
FBumann added a commit that referenced this pull request Oct 6, 2026
…pecsolve, and the specsolve label sits clear of the other lines (#1862)

> **Prompt:** I woud like to quickly explore some minor improvements to
the visal, hover and specsolve text positionon.

> [!NOTE]
> The following content was generated by AI.

The specsolve label moves to the right of its last point, clear of the
other lines. The tooltip lists the libraries slowest first, as the lines
stack, with each value and its ratio to specsolve aligned in columns.
pyomo takes the gold of its logo and linopy the blue.

<details><summary>What changed</summary>

- **Label.** The label sat above and left of specsolve's last point,
where the lines bunch up, and overlapped them. It now sits right of that
point, vertically centred, in a 76 px right margin. It is a chart
control, not a text mark: the marks share one clip group, and the clip
cut the label off in the margin. At a size where specsolve is projected,
the label follows the end of the projection.
- **Tooltip rows.**
- The rows are sorted by value, slowest at the top and fastest at the
bottom, the order of the lines at that size. They were in legend order.
- Every row reads `library value · ratio`, specsolve's own `1.0×`
included. specsolve's row is `active`, so TanStack draws it bold on a
tinted background.
- A projected row says so in its label, `pyomo (projected)`. The value
no longer carries a `≈` prefix, which would break the column.
- **Tooltip columns.** TanStack lays out each row as its own grid, with
the value right-aligned in tabular digits. A column therefore lines up
only if every entry has the same shape. Three rules give that:
- one unit for the whole tooltip, the smallest value's (`14781 ms` over
`761 ms`, not `14.78 s` over `761 ms`);
- fixed decimals in each column: the ratio always has one, and the value
has as many as its smallest entry needs;
  - left padding with figure spaces (U+2007), which are one digit wide.
- **Tooltip size.**
- TanStack's default font is 0.75rem, which is 15 px under the theme's
20 px root. The tooltip now uses .64rem, the table's size.
  - The width cap is 24rem rather than TanStack's 80% of the panel.
- Labels do not wrap, so a long `(projected)` label near the panel edge
stays on one line.
- **Colours.**
- **pyomo** takes the gold. Its logo is a bird, mostly gold and orange.
- **linopy** takes the blue. Its logo is a grey-mauve wordmark with an
orange dot, so it has no blue, but it has no other distinctive colour
either.
- **Orange for pyomo was tried and refused.** Beside gurobipy-loop's
red, orange scores ΔE 11 for normal vision, below the validator's floor
of 15.
- **HiGHS** (red and black) appears only inside the neutral matrix
floor, so it keeps no colour of its own.
- **Dark-mode blue.** linopy is the line nearest specsolve, so in dark
mode its blue moves from `#3987e5` to `#35b9f2`. That is drawn as
`#329ccb` after the 85% mix with the surface. Against specsolve's
violet, the blue drawn before (`#3372c0`) scored ΔE 14.3 for normal
vision and 6.7 for protanopia. The new blue scores 16.2, and 6.4 for
deuteranopia. The validator allows 6.4 only with a second cue; specsolve
has three: a thicker line, larger dots and its label. In light mode the
drawn pair scores 27, and 22 for deuteranopia.

</details>

<details><summary>What was verified, and what was not</summary>

- **Gates:** `pixi run check` passed (4999 tests) on ee0c424. The later
commits change only the changelog, the JS, the CSS and the page's JSON
spec, which `check` does not lint. `pixi run docs-build` (strict) passes
on the head commit.
- **Browser, on the built site:** headless Chromium, with the jsdelivr
files served from the npm tarball.
- **Alignment, measured:** the sweep hovers every panel at 18 points
each, for wall time and peak RSS on both ladders. That gives 204
tooltips. In every one, the non-bold value cells are the same width, a
spread of 0.00 px. The bold specsolve cell is 0.8 px wider, because its
digits are bold. No label wraps.
  - **Tooltip:** `dispatch — gurobi` at 1M reads:

    ```
    pyomo             14781 ms · 18.2×
    gurobipy-loop      2420 ms ·  3.0×
    linopy              941 ms ·  1.2×
    specsolve           814 ms ·  1.0×
    gurobipy-matrix     761 ms ·  0.9×
    ```

    Checked in light and dark.
- **Colours:** read back from the page as drawn and checked with the
dataviz skill's `validate_palette.js`.
- **Label:** one `specsolve` label per panel, drawn in full right of the
last point. Checked in log–log on the length ladder and in linear–linear
on the width ladder.
- **Toggles:** the toggle test from #1860 still passes, on a direct load
and after instant navigation.
- **Not checked:** phone widths. At phone width the 76 px margin leaves
a plot of about 185 px.

</details>

<details><summary>What was not done</summary>

- **Gridlines, the vertical hover line and dot sizes are unchanged.** I
had proposed fainter gridlines, a lighter hover line, larger specsolve
dots and a stronger dark-mode band. In the screenshots the gridlines
already read as faint, and the specsolve dots are already the largest.
The hover line stays as it is.
- **The gold–red pair is left as it is.** As drawn, gold and red score
ΔE 3.7 for deuteranopia in both themes, and 12.6 (light) and 14.7 (dark)
for normal vision. This pair was there before this PR, when linopy had
the gold; now pyomo has it. Both meet only in the Gurobi panels, where
pyomo's line runs well above gurobipy-loop's. Fixing it means a new
colour for one of them, so it is a separate change.

</details>

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01K42KwZJtqPPhf5hDzUD91o

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants