Visual tests flaky on math pages: MathJax can size math before the web font loads #463

Symptom

Visual tests on math-heavy pages have started failing intermittently because the page renders at a different height. A size mismatch fails toHaveScreenshot whatever the tolerance, and retries don't help, because the height also varies between screenshots in the same attempt. All 21 ci.yml runs in September and the 1 October run passed with no flaky retries. Of the 17 runs on 2026-10-09, 3 failed and 2 more were flaky:

Other runs of the same code passed cleanly, including main at 11d4b12 and the 5 other dependabot runs from the same few minutes. Within one failing attempt the height moved between consecutive screenshots, by 3 px on desktop and mobile math (2939 ↔ 2942, 2971 ↔ 2974) and 1 px on mobile long-page (5901 ↔ 5902), so the page never settled.

What the short render shows

I compared the desktop math page's math-actual.png (2942 px) from the visual-test-diff artifact of run 37874459436 with a full-height render made on the same day by /update-snapshots on #457 (3009 px; it has the same header and toolbar). The two are identical down to the end of the page header (y ≈ 525). From there, every equation in the short render is drawn at a smaller scale: inline math, display math, the aligned and cases environments, and the matrix. The surrounding text and headings are the same size. The smaller inline math also changes where the first paragraph wraps. The 67 px of lost height comes entirely from math being smaller.

Likely cause

The theme loads its text fonts (Source Sans Pro, PT Serif) from Google Fonts with display=swap, via the @import in assets/styles/index.scss. The page's MathJax is the CHTML build (mathjax@3/es5/tex-mml-chtml.js, which Sphinx loads by default). CHTML's matchFontHeight option is on by default, so MathJax sets the scale of all math from the ex-height of the surrounding font at the moment it typesets, and it doesn't redo this when a web font swaps in later. If Google Fonts arrives after MathJax has typeset, the math is sized against the fallback font's metrics. Which happens first depends on network timing on the runner. That would explain failures starting with no change to fonts, MathJax or the math fixtures.

waitForReady in tests/visual/theme.spec.ts (from #402) waits for MathJax's startup promise, then document.fonts.ready, then a stable height. By the time fonts are ready the math has already been sized, so waiting doesn't undo a bad measurement.

This is a hypothesis based on the screenshots and on how MathJax CHTML works. I haven't confirmed it by controlling font-load timing.

Not only a test problem

A reader on a slow connection, where the Google Font arrives after MathJax typesets, gets the same wrongly sized math for that page load. The lecture sites are math-heavy, so a fix in the theme would help readers as well as the tests.

Options

1. Theme: make MathJax's initial typeset wait for fonts, by overriding startup.pageReady to await document.fonts.ready before calling MathJax.startup.defaultPageReady(). Math would then be sized against the real font. It needs a timeout, raced against fonts.ready, so that math still renders promptly where Google Fonts is slow or blocked. That matters for readers in China, where Google Fonts is often unreachable. The visual baselines are unaffected if the "big" render is the correct one.
2. Theme: turn off matchFontHeight (set the chtml scale explicitly) so math size no longer depends on the font. This is deterministic, but it changes math size on every page, so it needs a full baseline refresh and a design check.
3. Tests only: after document.fonts.ready, re-typeset (MathJax.startup.document.rerender()) and wait for the height to settle again before taking the screenshot. This makes CI deterministic but leaves the race in place for readers.

Also noticed

The theme's inline MathJax config in layout.html sets svg: {fontCache, scale: 0.92, displayAlign}, but the page loads the CHTML build, so that block has no effect. Any fix to math sizing will touch the same config, so it is worth deciding whether the 0.92 scale was meant to apply.

References

#402 added the current waitForReady. #457 refreshed the baselines on the same day; its description has the run details.