fix(coupling): an IQN step returns when its iterate leaves float range; the gradient bound leaves out a constant the pass does not resolve

[NicholasEhsanRoy]

Two findings of the nonlinear coupling search (PR #255), fixed. Branch starts from that PR's branch and carries it (tests only); origin/release/0.4.0 at 7acbe0d6 (which has #255) is merged in.

1. step() did not return under IQN once the iterate left float range (MADD-ANO-231)

Cause (measured, every probe under a wall-clock limit). iqn_ils_update solves the secant least-squares with jnp.linalg.pinv, an SVD with vectors. On the pass where part of the iterate overflows, the new secant column holds inf in those entries and finite numbers in the rest, and LAPACK's SVD does not return on such a matrix. On the pinned hub the eleventh call was handed an 8 x 119 matrix with four inf entries, no NaN, and finite entries up to 5.8e285 (the ten before it were finite); the pass after would have added NaNs, and a matrix with a NaN beside the inf returns. jnp.linalg.svd / pinv alone, float32 and float64, jaxlib 0.11.0, CPU: no return in 6 s (0.2 s where it returns) on 4 x 4, 12 x 8 and 16 x 16 matrices with one inf, one -inf or an inf column, and on some with a row and a column at finfo.max; an all-inf matrix, a NaN, and an inf with a NaN return. It is not a loop condition: the while_loop counter advances and none / aitken run to the cap.

Fix. acceleration._lapack_input(A) returns (A or zeros, ok), ok being that the sum of the squares is finite (so NaN, inf and entries whose squares overflow are all withheld); the matrix is replaced before the factorisation (under jit both sides of a where run). In iqn_ils_update a withheld matrix means the quasi-Newton step is not taken (is_valid carries ok), so the Aitken fallback's iterate is returned, which is what a non-finite c already meant. The group then runs to its cap and reports converged=False, residual=inf, exactly as none and aitken do (the search's unaccelerated control reads RETURNED 120 False; so does IQN now). I did not add an early exit on a non-finite iterate: the cap is what CPL-053 / CPL-084 and the other accelerations do, and an exit in the loop condition would change the program of every looped graph.

Sibling sweep: every LAPACK-backed call in core/coupling/. 288 non-finite cases (8 patterns x 3 shapes x 8 primitives) and 576 extreme-finite cases (finfo.max down to subnormals) per dtype, each with its own time limit:

Site	Call	Can a non-finite matrix reach it?	Measured	Now
acceleration.iqn_ils_update	pinv (SVD with vectors)	yes: a secant column of an overflowing pass	does not return on an inf	guarded (_lapack_input); QN step not taken
acceleration._spectral_analysis	svd(I - Hk, compute_uv=False)	yes: a Jacobian at a non-finite or huge state, with diagnostics=True	returned NaN on every non-finite case; did not return on one of 96 with entries at finfo.max	guarded; zeros have no singular value above zero, so the amplification reads inf as before
_bounds._full_resolvent_norm_and_rows	norm(inv(I - M), ord=2), norm(T, ord=2) (SVD)	yes: I - M singular or the basis non-finite	as the row above	_spectral_norm: NaN for a withheld matrix (what the SVD returned where it returned)
_bounds._gradient_error_bound_body	norm(dG, ord=2) batched, norm(resolve(change), ord=2)	yes: a second derivative that overflows	as above	_spectral_norm
acceleration._spectral_radius	eigvals	yes	returned on all	already replaced a non-finite matrix by zeros; unchanged
acceleration._rounding_keeps_the_radius	eigvals, inv	yes	returned on all	already guarded (finite); unchanged
acceleration.jacobian_range_basis, _bounds (two sites)	qr	yes	returned on all 96 + 96	not guarded
acceleration.resolvent_apply, _ift._dense, _bounds	solve (LU)	yes	returned on all	not guarded
mapping.py (two sites)	np.linalg.solve at build time, NumPy	host-side, raises LinAlgError	n/a	unchanged

A NaN any_dir (a non-finite derivative) still compares False in any_dir > 0 and its term is dropped, as on the base; I have no reproducer of a wrong number from it and did not change it.

Released? Yes. v0.3.0 and v0.3.1 (git archive of each tag, empty HOME, 25 s limit): solver="ift" with acceleration="iqn-imvj" (jnp.linalg.lstsq there) does not return on the test's exp pair in float32 or float64; the default solver="fori" solved the normal equations by LU and returns a NaN state; iqn-ils under ift returns. v0.2.1 and earlier have no solver="ift". affected_versions: ">=0.3.0, <0.4.0".

Values of finite runs. Bit-identical against the base (75748b62), jaxlib 0.11.0: 200 random finite calls of the compiled iqn_ils_update (five sizes, both dtypes, magnitudes 1e-100 to 1e100 in float64, rank-deficient, graded and empty column sets, with and without a previous pass; all eight outputs), and 108 drawn three-member nonlinear rings (both IQN kinds, both dtypes, ift Gauss-Seidel and Jacobi, fori), two steps each: states, iterations, residual and every numeric key of coupling_diagnostics() except gradient_relative_error_bound on two (finding 2).

Step-program digests. 8 of the 24 gate graphs moved, on all three jax versions, 5 programs each: named/{chain-into-ring, nested-loop-additive, star-and-ungrouped-ring, two-groups-one-scc}/choice-1 and the same four under sparse-ragged/. Each has a group under acceleration="iqn-ils" (topology_knobs(topo, 1)); no other gate graph has a quasi-Newton group, and none has diagnostics=True. The other 16 hold the same digests as the earlier capture (396eb59a) under 0.10.2, 0.11.0 and 0.11.2. Recaptured the way test_step_program_digests_are_frozen.py requires: the fix commit 457f4b4f removes the file, the capture is taken on it in a fresh process per version (--write), and c3f10735 adds it. scripts/compile_counts.py --check on jax 0.10.2: counts match.

2. gradient_relative_error_bound below the error for a constant the fixed point does not respond to (MADD-ANO-232)

Which quantity the bound covered on the two pins (float64 reference at the returned iterate):

	stalled float32 ring	converged float64 hub
reported bound	1.385e-6	1.0902
the failing constant	r1.s0: g_k 6.0e-26, g* 2.2e-25, error 1.6e-25	l1.s1: g_k 1.5e-34, g* 2.0e-35, error 1.7e-34
relative to its own gradient	2.72 (not covered)	1.134 (not covered)
relative to its own right-hand side	the float32 right-hand side is exactly 0	1.13 (right-hand side 2.4e-33 scaled, tangent the same size): not covered
error over bound x the largest gradient of any constant	1.1e-19	4.3e-38 (covered, by 37 orders)
worst over every constant of that ratio	2.4e-3	6.0e-14
the resolved constants (gains, biases) alone, relative to their own	3.9e-9 against 1.4e-6	5.31e-12 against 5.34e-12 once the others are left out

So the absolute error was always tiny beside the gradients that have a size; what failed is a relative statement about a gradient that has none. On the ring the float32 tangents of the centres and curves are exactly zero and were never in the bound (the search scored the float64 tangent at the float32 iterate). On the hub the right-hand sides of those probes were 2e-33 to 5e-18 beside ones of order one: for the saturating c + tanh(s (u - c)) / s, d/ds is two terms of size d / s that cancel to -(2/3) s d^3, 24 orders below either at d = 1e-12, so the tangent is what the cancellation left.

Choice: (b), per constant instead of per group. A statement relative to the whole gradient vector (a) would mix the units of different constants and silently weaken the per-constant claim the earlier rounds verified. Withdrawing the flag where the bound exceeds about 1 would withdraw it wholesale on these cells: wherever such a constant is resolved (an iterate stopped short) its relative error really is of order one, the bound reads 1.03 to 16 and holds, and that is information. So: a probe is in the worst only where the pass resolves its constant: moving the constant by the probe's whole size (its own magnitude) moves one pass from x_k by more than the residual's float floor (the floor spectral_error_bound adds, in the bound's norm). This is the existing zero-tangent rule ("the fixed point does not respond to the probe") made at the resolution it can be made at. gradient_bound_usable is unchanged unless no probe is left (NaN, as before for "responds to no constant"). The docstring (coupling_diagnostics), the algorithm guide, CPL-093's conditions and the search's score (PassReference.pass_responses above RESOLVED_MARGIN = 2 floors) say the same thing.

What it costs, said plainly. The rule is a magnitude, so a gain or forcing so weak that its whole value moves the pass by less than the floor is left out too, although its tangent may be exact. I looked for a measured quantity that tells the two apart and found none: the tangent's sensitivity to a floor-sized change of the state (the bound's own undirected term) read 1.6% for the hub's curve, whose relative error is 113%, because the cancellation is inside the node's derivative, not in the state. On the 108 drawn rings above the bound moved on two (2.27e-5 to 6.8e-6; 1.8e-2 to 2.5e-4: a weak gain was the worst probe) and no flag changed. Registered as the entry's residual risk; a constant left out is not named in the report. For the orchestrator: if weak-but-exact constants must stay covered, the alternative is one more JVP per probe to test whether the right-hand side depends on the iterate at all (additive constants could then be kept at any size); I did not add cost to the diagnostics without being asked.

Usable fraction, same draws before and after (the search's strategy, derandomised; 142 distinct draws on each of two blocks of seven cells; base 75748b62 against this branch, jaxlib 0.11.0):

	block 0	block 4
gradient_bound_usable, before	0.845	0.873
after	0.845	0.873
draws whose flag changed	0	0
usable on both: bound lower / higher	75 / 0 of 120	69 / 0 of 124
usable bound reading 1 or more, before -> after	106 -> 67	99 -> 61
usable draws with a nonlinearity constant resolved and scored, after	66	60
over the limit, resolved constants (gains; nonlinearity constants)	0; 0 on both trees	0; 0 on both trees

No flag is withdrawn. The bound stops reading "of order one" on 39 and 38 draws where every constant with a size has a small error, and still reads one or more wherever a nonlinearity's constant is resolved. In the slow hunt the worst scores (limit 1) are 0.036 to 0.989 over gains, biases and weights and 0.656 to 0.99999631 over the nonlinearity constants, by block; the fraction of examples with such a constant scored is 0.17 to 0.57; the usable fraction 0.86 to 1.00. The linear search's gradient hunt (all seven blocks, not one seed): nothing over the limit, worst 0.942 to 0.999, usable 0.92 to 1.00.

Why the earlier every-constant sweep did not see this. test_differential_gradient_bound.py and the far-from-the-fixed-point tests measure the bound over every gain and bias of linear groups and over a, g of maps such as x <- a + g u**2. Every one of those constants enters the pass at first order with a sensitivity of order one at the fixed point: a gain multiplies an input that is not zero, a bias is added. So g* is never zero, g_k is never small beside it, and a relative error is well defined at every iterate. The search's cells centre each nonlinearity on its input at the fixed point (phi(c) = c, phi'(c) = 1), which makes the pass's sensitivity to c and s vanish exactly there: the first constants in any harness whose true gradient is zero while the computed one is not.

Tests

• tests/core/test_coupling_iqn_returns_on_a_non_finite_iterate.py (new). Per push: the step returns within 30 s (it takes under 2) for both IQN kinds in float32 and float64, converged=False, residual=inf, 40 of 40 passes, on a pair whose first entry is exp of itself beside three that contract (fails on the base: no return); iqn_ils_update on eleven kinds of poisoned input at three sizes in both dtypes, with the Aitken fallback's iterate returned for a withheld matrix; _lapack_input / _spectral_norm on inf, -inf, NaN and finfo.max. Slow: the step under fori, with diagnostics, under l2 and interface, Jacobi, with aitken and none beside. A stuck call runs in a daemon thread and fails the test; the process then leaves by os._exit(1) at exit (a thread stuck in LAPACK aborts the interpreter's finalisation otherwise).
• tests/property/test_coupling_nonlinear_search.py: the two IQN pins un-xfailed and held to RETURNED 120 False; the skip of examples that leave float range on an IQN cell removed (leaves_float_range is gone, every example is stepped); gradient_vanishing is now one of the searches (per push and in the hunt, where the fraction of examples with such a constant scored must be above zero: measured 0.17 to 0.57 by block); a per-push seed with six of twelve centres and curves resolved (bound 1.27, error 0.79 of it); the two gradient pins are regression tests (test_a_constant_the_pass_does_not_resolve_is_not_in_the_gradient_bound: none of the nonlinearity's constants is resolved, the bound is below 1e-5 and not below the gains' error).
• tests/property/coupling_reference.py: PassReference.pass_responses.

Mutants (scratch copy of src/, plans/tools/mutants.py): 9 of 9 caught.

Mutant	Caught by
M1 the IQN guard off	the per-push step test (no return in 30 s)
M2 _lapack_input returns the matrix unreplaced	test_no_svd_...
M3 ok is "every entry finite" (squares may overflow)	test_no_svd_... (finfo.max)
M4 the quasi-Newton step taken on a withheld matrix	test_the_secant_update_returns_...
M6 a withheld matrix's norm reads 0 instead of NaN	test_no_svd_...
M7 an unresolved probe kept (rhs >= 0)	the hub regression (slow)
M8 resolved probes dropped (1e5 floors)	the per-push seed
M9 a non-finite right-hand side dropped	test_coupling_gradient_bound_poisoned_by_a_non_finite_tangent.py
M10 the floor a 1e-20th	the hub regression (slow)

No mutant for the amplification's guard: zeros and a NaN singular value both read inf there, so removing the select is equivalent (I removed it).

Code paths newly enabled or changed

• A group under IQN with a non-finite secant matrix now completes its loop: every later pass runs the Aitken fallback on a non-finite iterate, and with diagnostics=True the analysis runs at a non-finite state (tested per push without and slow with diagnostics, fori and ift, three norms, both schedules).
• The warm start of iqn-imvj can now carry non-finite columns into the next step of a graph that returned a non-finite state; that step's matrix is withheld the same way.
• gradient_relative_error_bound can be smaller than before on a group with a weak or a non-responding constant, and is NaN (unusable) where no probe is resolved.

Records

MADD-ANO-231 (shipped, >=0.3.0, <0.4.0, major, resolved) and MADD-ANO-232 (never shipped, major, resolved); two CHANGELOG lines under Fixed; one line each in the release notes' Resolved paragraph; SOUP table regenerated; CPL-053, CPL-084 (tests added) and CPL-093 (conditions and tests); testing_standards.md (the search's skip is gone, the second gradient score).

Run locally (three cores, jaxlib 0.11.0 unless said)

Every slow test of every file that cites gradient_relative_error_bound / gradient_bound_usable, iqn_ils_update, the resolvent and spectral helpers, the two search modules and the digest gate, once each on the final code:

• 27 files under tests/core (the 24 that cite the gradient bound or its helpers, test_coupling_acceleration.py, test_coupling_accelerators_are_units_invariant.py, and the new file), -m slow: 105 passed, 6 failed, all six in the new file's own slow test and all its premises, not the library (fori reports only with diagnostics=True; under Jacobi the quasi-Newton iterate of the fixture stalls finite, on the base too). Corrected and the new file rerun whole: 28 passed (8 per push, 20 slow).
• tests/property: test_coupling_nonlinear_search.py, test_coupling_targeted_search.py, test_differential_coupling_interactions.py, test_differential_fixed_point.py, test_differential_gradient_bound.py, test_differential_precision.py, test_differential_transforms.py, test_differential_coupling_solvers.py, test_interface_reading_twins.py, -m slow: 340 passed (one hour).
• tests/compliance/test_step_program_digests_are_frozen.py and tests/core/test_step_program_digests.py, -m slow: 22 passed.
• Per push: the two changed test files (21 passed), the gradient-bound and resolvent files of tests/core (80 passed), the registry / claims / release-notes / guard-mutation / digest-frozen / slow-rule / sharding compliance tests, the three source scans, every scripts/check_*.py, pyright 1.1.414 on the two changed modules (0 errors), compile_counts.py --check on jax 0.10.2.
• Not run: the slow tests of the other 60 files that only parametrise over "iqn-ils" / "iqn-imvj" (the bit-identity runs above stand for them), and nothing on jax 0.10.2 or 0.11.2 except the digest captures and the compile counts.

One thing to watch in the slow lane on other jax versions: the nonlinearity-constant score is tight by construction (the true relative error is 1 and the bound is it times distance / ||delta|| >= 1): the worst here is 0.99999631 of the limit after the score's own allowances.

🤖 Generated with Claude Code

https://claude.ai/code/session_013UkCde7g23gTziUjYAvnKD