Reduce float32 fields through muGrid, and floor the inner-CG tolerance once

[pastewka]

Two float32 defects that only show up at scale, and one that had drifted. Found while chasing non-convergence of 512³ single-precision runs.

Requires muGrid 1.4.0 (muSpectre/muGrid#208) — the pin in pyproject.toml is not cosmetic, see the last section.

1. The reductions bypassed muGrid

solve_rhs and the consistent-objective correction reduced raw field buffers with xp.dot / xp.sum. On a float32 field that is BLAS sdot: a float32 accumulator, whose error grows linearly in the number of entries. Measured against a double reference:

grid	N	xp.dot	linalg.norm_sq
32³	98,304	4.2e-09	1.6e-15
64³	786,432	4.1e-07	1.8e-14
128³	6,291,456	5.5e-06	6.9e-14
512³	~4e08	~4.6e-04	—

Nothing else in the solve degrades with resolution like that. For contrast, the single-precision FFT round-trip grows only 37% from 32³ to 512³ (1.39e-07 → 1.91e-07), so the FFT is not what changes.

They also copied. Field.p is a strided view whenever the collection carries ghosts, and both engines always build with ghosts=1, so every .p.ravel() silently materialised a full contiguous copy — dim*N*4 bytes per call, on the device for GPU runs.

And b_norm is now bit-identical to the ||b|| the CG itself converges against, instead of a float32 denominator under a double-accumulated residual.

The consistent-objective correction -λᵀr gets the same treatment. It is both the reported objective and the input to the trust region's accuracy control, so it carries the tightest error budget here.

2. The tolerance floor had leaked

float32 eps is 1.19e-7, so the true residual b - Kx stagnates around 1e-6 while only the recursive CG residual keeps shrinking. Asking for less burns cg_maxiter and is rescued by the stagnation guard with a worse iterate than the reachable tolerance would have given.

That floor lived in simulate.py alone — which is exactly why it had drifted. Audit of every path that set cg_tol_min:

location	respected the floor?
simulate.py defaults (TR and L-BFGS)	yes
simulate.py explicit --cg-tol-min	no — warned, then used the value unclamped
simulate.py --cg-tol-start <= 0 + TR	no — reassigned cg_tol_min after the clamp
simulate.py Homogenization(cg_tol=args.cg_tol)	no — raw CLI value
simulate_conduction.py	no floor anywhere in the file
optimize.py _make_inner_tolerance	no — inherited the unreachable cg_tol=1e-8

It is now applied wherever a tolerance is consumed, so no branch can route around it. AdaptiveInnerTolerance is deliberately left alone — it stays a pure dtype-agnostic controller, and its three existing tests pin that.

cg_tol defaults to None and resolves per precision. A tolerance the caller explicitly asked for and cannot have warns; a default they never chose is adjusted quietly.

Behaviour change worth a look: an explicit unreachable cg_tol is now raised to the floor, not just warned about. I think that is right — the alternative is burning cg_maxiter and salvaging — but it is a behaviour change, not only a diagnostic.

3. simulate_conduction.py caught up

It never had the precision-aware block simulate.py has: no rtol_floor, and a hard-coded bfgs_gtol = 2.5 where simulate.py uses 25.0 in single precision. args.precision was used only for the dtype, the banner and a NetCDF attribute.

Why the muGrid pin is hard

Without muGrid 1.4.0's reduction fix, linalg.norm_sq returns inf on exactly the large float32 solves this PR routes through it — worse than what it replaces. And muGrid.Solvers.FLOAT32_RTOL_FLOOR, the shared constant muTopOpt.precision reuses so the two packages cannot disagree about the floor, does not exist before it.

Testing

172/172 pass, serial and under mpirun -np 2. New: test_precision_floor.py (10 cases covering the floor at every consumption point, and its absence in float64), plus a reduction-accuracy regression in test_single_precision.py that pins the relationship — whatever solve_rhs uses must track the double reference far more closely than a naive float32 dot — rather than an absolute error.

End-to-end simulate.py --precision single runs clean at 32³, and the previously-bypassed path now reports fixed cg-rtol 1e-06 with a warning where it used to silently pin 1e-9.

🤖 Generated with Claude Code