Skip to content

test(coupling): instruments for geometry in coupling diagnostics, stage 0 - #255

Merged
NicholasEhsanRoy merged 5 commits into
release/0.4.0from
feat/p4-geom-diag-instruments
Oct 7, 2026
Merged

NicholasEhsanRoy merged 5 commits into
release/0.4.0from
feat/p4-geom-diag-instruments

Conversation

@NicholasEhsanRoy

@NicholasEhsanRoy NicholasEhsanRoy commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Tests only: no library change. Instruments that later work on coupling diagnostics which read a moving geometry will be held to, and that give the coupling search its first nonlinear coverage today.

1. Pre-check: the harness with the phase-1 narrowing removed

Scratch copy of the tree at 1f66801c (never the worktree): DIAGNOSTICS_READ_GEOMETRY = True in tests/property/geometry_graphs.py, and in coupling_diagnostics() the one narrowing site (geometry_keys = self._committed_geometry_edges.get(key, ()) made ()), so a group with a geometry edge reports the step's own numbers. The interface-norm refusal at compile was left in place. jaxlib 0.11.0, CPU, three cores.

tests/property/test_differential_geometry_edges.py and tests/property/test_geometry_time_levels.py, per-push and slow: 207 passed, 7 failed, 2 skipped.

Cell Result Interface norm or an interface reading's float floor involved?
test_an_edge_mapped_graph_steps_as_its_node_inlined_twin, the 11 per-push cases under l2 / mixed (6 of them groups) pass no
test_every_drawn_edge_mapped_graph_steps_as_its_node_inlined_twin, the slow cases under l2 / mixed (26 of them groups) pass no
the gradient, batched, restart, frozen-geometry, named-topology and domain cells of both files pass no
test_an_edge_mapped_graph_steps_as_its_node_inlined_twin[interface norm, three passes] (per push) fail yes: refused at compile
test_every_drawn_...[group IQN-IMVJ, interface norm], [group interface norm, Jacobi, three passes], [sub-cycled target, interface norm, three passes] (slow) fail yes: refused at compile
test_a_report_s_float_floor_counts_a_source_anchored_geometry (slow) fail yes: refused at compile (the interface reading's floor)
test_a_moving_geometry_is_read_at_its_documented_time_level[group, mixed anchors, interface norm], float32 (per push) and float64 (slow) fail yes: refused at compile

No unexpected failure. Every failure is the RuntimeError of the refusal that was left in place.

The harness compares the estimates only to a factor of two, so the gap was also measured: the 32 group cases under l2 / mixed, each with its own knobs and again with diagnostics=True (64 rows, three steps each), every key of the report, edge-mapped graph against node-inlined twin: no flag differs, no key is finite in one and not the other, and no number differs by more than 1e-6 of itself; 16 of the rows have spectral_usable set.

With the refusal removed as well (not asked for; one more scratch edit), the four interface-norm cases and the floor case do not report a wrong number: the step cannot be traced (TypeError: unsupported operand type(s) for @: 'NoneType' and 'DynamicJaxprTracer' for the matrix kind, ValueError: a multilinear_grid mapping reads a moving geometry and was called without one for the multilinear kind). The interface reading calls the mapping without its geometry; phase 1 never built that reading.

2. A numerical float64 reference (tests/property/coupling_reference.py)

PassReference computes, for a small group of any nodes and edges: the fixed point (Picard passes, then Newton with the dense Jacobian, to a few float64 eps; the residual history is returned), the dense jacfwd Jacobian of one pass at any iterate, the spectral radius, the distance of a returned iterate in the norm the report states its bound in, the exact residual, the implicit-gradient error for every scalar constant (or a scalar loss), and a measure h of how far the Jacobian moves between an iterate and the fixed point.

How the pass map is built. An x64 twin of the graph (same nodes, edges, schedule, sub-cycling; float64), its group at max_iterations=1, no acceleration, predictor="linear". One step is one pass, differentiated straight through. The predictor starts the pass from 2 pred_0 - pred_1, kept in three _meta slots; writing the iterate into both makes the step compute P(x; pre), one pass from the iterate x with each member integrating from the pre-step state. It is the graph's own compiled step (gm._raw_step_fn) called on a state: nothing of the estimator is called. The slots' layout is found by stepping the twin once on a drawn example and matching the stored pred_0.

Catches: the estimator disagreeing with the map the solve iterates (a term missing from its Jacobian, a reading at another time level, a Jacobian at another state, a bound in another norm). Does not catch: a defect in the pass itself (inherited; the closed-form and time-level references hold the pass); a difference between the single-pass branch and the iterated pass (checked: three passes of an iterating twin are three compositions, to 64 eps); a node that branches on its dtype; more than one group in a graph.

Validated against the closed form on every per-push draw of the linear search (600 draws, 556 distinct, five cells: both dtypes, both schedules, l2 and interface norms, a mapped ring, twelve scalars), every one converged:

Answer Worst difference from the closed form
spectral radius 1.0e-15
Jacobian 1.6 float64 eps of its largest entry
fixed point 5.7 eps of the field times the resolvent's norm
distance to the fixed point 5.5e-11 norm units on the interface-norm cell (2.4e-12 of the distance); at most 1.3e-12 elsewhere
exact residual 9.8e-12 norm units on the same cell; at most 5e-15 elsewhere
gradient error 1.9e-12

3. Nonlinear cells in the search (tests/property/test_coupling_nonlinear_search.py)

Structures of the linear search (rings of 2, 3, 5, a pair of 3, the hub, the mapped ring, and tri) with each port's input through a nonlinearity centred on the input at the linear fixed point: a saturating gain, a quadratic term, a product of two fields. phi(c) = c, phi'(c) = 1, so the fixed point and the Jacobian there are the linear ones (a second check of the reference on every example) and the numbers are drawn exactly as the linear search draws them, plus the curve (0.01 to 100 over a field's size). 43 cells; one per push.

Scores: the linear search's four, against the reference. The error bound is claimed for a linear F and "asymptotic" otherwise (CPL-088), so the distance is held to the bound times 1/(1-h), which is exact (x - x* = (I - J_mean)^{-1}(x - F(x))); the radius is scored at the returned iterate (CPL-087); the gradient bound separately over gains, biases and weights and over the nonlinearity's own constants.

The hunt (six blocks of seven cells, 115 random examples a score a block, seeds 1000 to 1005, jaxlib 0.11.0):

Score Flag set Worst score (limit 1) Over the limit
error bound 0.90 to 1.00 0.81 to 1.00 by block none; and the unadjusted distance / bound was never over 1 in any example
radius 0.91 to 1.00 0.042 none
radius, the user-checkable statement 0.64 to 1.00 0.61 none
gradient bound, gains / biases / weights 0.83 to 1.00 0.48 none
floor 0.94 to 1.00 0.13 none
gradient bound, a nonlinearity's c and s 0.85 to 1.00 3.0e5 finding 2

The reference found a fixed point for 0.93 to 1.00 of the examples that returned a finite state (a capped solve from a start a whole field away can return where Newton reaches none; such an example is not scored, and the slow test fails under 0.75). On the block where a hunt climbs furthest towards diverging starts, 0.68 of the examples returned a finite state.

Findings (pinned strict-xfail, not fixed):

  1. step() never returns for a group under acceleration="iqn-ils" or "iqn-imvj" whose iterate becomes non-finite before max_iterations (EDGE; CPL-053, CPL-084, which both say a diverged group is reported). Hub of four members, float64, products of two fields, Jacobi, cap 120, a start a whole field from the fixed point: plain passes overflow on the eighth; no return in 240 s, where a converging step of the same compiled graph takes 0.05 s. With diagnostics=False and under l2 as well. Returns with acceleration="none" or "aitken" (120 passes, residual=inf, not converged) and at a cap of 12. Pinned in a subprocess with a 30 s limit, with the unaccelerated control beside it. Not in the registry (anom.py list --similar: nearest is MADD-ANO-173, run_adaptive). The search does not step an example that plain passes carry out of float range on an IQN cell.
  2. gradient_relative_error_bound below the error for a constant the fixed point does not respond to (EDGE; CPL-093). The derivative of the fixed point with respect to a nonlinearity's centre or curve is zero at the fixed point and not at any other iterate, so the relative error is of order one however close the iterate is. Where the solve stopped with a nonzero residual the bound reads above it (1.01x to 1.3x in most examples); at a start stalled on its float floor (one pass, residual=0, precision_limited) it reads 1.4e-6 for a relative error of 2.7, usable; on a float64 hub converged in four passes, 1.09 for 1.134. Two pins.

4. Twins with the edge-mapped graph's interface reading (geometry_graphs.py, test_interface_reading_twins.py)

A free-standing relay node cannot do it: inside the group its input edge S.x -> relay is a second internal edge the interface norm reads, and outside it sits in the group's cycle. So:

  • relay_twin: the relay is fused with the source (RelayedSourceNode): a state field of the source is the mapped value, a plain edge delivers it, the mapping (and a source-anchored geometry) is inside. Same reading entry for entry.
  • transform_twin (static mappings only): the mapping written as the edge's transform. Same reading and same state.

On a static mapped internal edge under convergence_norm="interface" (two-body graph, 5 and 4 scalars, three passes; also Jacobi, Aitken, IQN-ILS, float32 and float64):

transform twin relay twin (Gauss-Seidel)
states, residual, pass count, verdict, flags identical (bit for bit) identical
rho_spectral identical 1.6e-15 (float64), 6.6e-7 (float32)
spectral_error_bound identical factor 1.1% to 3.4% apart
float floor identical 13% apart
gradient_relative_error_bound differs differs

Tolerances and why they are not zero: the transform twin is held to 2**10 eps (two programs; zero was measured). The relay twin's state has the relay's field in it: the bound's factor is the norm of a resolvent compressed onto a Krylov basis of the state (held to 10%); the floor counts an inner product for a mapping the norm evaluates and none for a field read as stored (not compared); the gradient bound's norm is the raw source fields, and it probes parameters, which the weights are not in a transform (not compared). Under Jacobi the relay doubles the scalars a pass carries (18 for 9), past what eight Krylov steps resolve, so spectral_usable differs: the relay comparison is made under Gauss-Seidel.

Seeded fault (scratch src/: _interface_readings reads the edge without its mapping): both equalities fail, the two residuals 8.8% apart (1664.8 against 1518.4; 1.2e-4 allowed).

Geometry: on the solve path, which 0.4.0 supports, the relay twin of a moving source-anchored geometry steps as the edge-mapped graph bit for bit (plain step, group under both schedules, geometry following the iterate, both mapping kinds). The interface-norm geometry cells are RELAY_INTERFACE_CASES in the harness's phase-1 block: refused today (asserted), their relay twins build and report; the comparison runs when DIAGNOSTICS_READ_GEOMETRY is set. A target-anchored geometry has no relay twin (refused with the reason).

5. The seeded-fault list for the later stages

# Fault Instrument Analogue on today's tree, measured
1 geometry term dropped from the reading's Jacobian numerical reference (radius, bound, gradient scores; its Jacobian has the geometry field in the iterate); relay twin's rho_spectral none (a static mapping has no geometry term). The reference's sensitivity: R1 to R3 below
2 geometry read at the wrong time level inside the reading relay twin; the time-level reference for the solve solve path, geometry read from the pre-step state: relay twin's equality fails after two passes, a field 2.6e-4 apart, 5.4e-7 allowed, zero unmutated
3 geometry from the iterate instead of the pre-step state (or the reverse) relay twin; numerical reference the same seeded fault (it is the reverse)
4 the geometry's own rounding left out of the precision floor numerical reference's floor score on cells with a float32 geometry not seeded. The relay twin does not judge a floor (13% apart by construction)
5 mapping dropped from the interface reading both twins seeded: residuals 8.8% apart
6 the reading's Jacobian-vector product disagreeing with a finite difference PassReference.finite_difference_gap (the pass, or any reading written in JAX) a reading with a term its tangent does not see: gap over 1e-3, under 1e-7 for the honest one

Mutants (scratch copy of src/, plans/tools/mutants.py; 5 of 5 caught)

Mutant Caught by
F5: _interface_readings reads the edge without its mapping transform twin (per push) and relay twin (slow)
F2-F3: a group's source-anchored geometry read from the pre-step state relay twin of a moving geometry, two passes (per push)
R1: rho_spectral reported 10% low per-push nonlinear radius search
R2: spectral_error_bound halved per-push seed (distance 0.88 of the bound)
R3: gradient_relative_error_bound a twentieth per-push seed (error a fourteenth of the bound)

Not caught, and said so: a geometry read from the previous snapshot in the sub-cycling interpolation branch survives the single-rate relay cases (that line is not on their path); a gradient bound halved survives the per-push lane (the per-push cell's bound is 14x above the error; the slow hunt's worst is 0.48 of the bound, so a bound 2.1x too small is seen there).

Cost per push (three cores, quiet, jaxlib 0.11.0)

  • nonlinear search module: 22 s. 9 s is the one nonlinear cell's compile with its diagnostics (the reference adds under 2 s of it), 8 s the validation (4 s of it a cell of the linear search, which that search then does not compile), the searches and seeds under 5 s.
  • twin module: 20 s. Two compiles of 8 s (the edge-mapped graph, the transform twin), 3 s for the moving-geometry relay cases.
  • Four tests carry a compile and are over 5 s: allowlisted with the reason (tests/duration_allowlist.txt).

The brief's 10 s for the nonlinear additions is the cell plus its searches (about 14 s here); the per-push lane has one nonlinear cell, with all three nonlinearities in it, for that reason.

Records

docs/developer_guide/testing_standards.md: one new subsection (the reference, the nonlinear cells, the twins). No registry entry, no CHANGELOG line, no claims cell (decisions for the orchestrator below).

Not done / to decide

  • Finding 1 (the step that never returns under IQN) is machinery and loud. It is pinned, not registered: a registry entry, its affected_versions and a fix are the orchestrator's call.
  • Finding 2 is at the edge of CPL-093's wording ("for the gradient with respect to one scalar constant"): either the bound should not be usable at a stalled iterate for a constant with no response, or the claim's conditions should say so.
  • Fault 4 has no measured analogue.
  • The relay twin holds the bound to 10% and does not judge the floor or the gradient bound; a later stage that needs those tighter under the interface norm needs the numerical reference on geometry cells (the reference takes a fields= reading written in JAX; no geometry cell is built in this PR).
  • A target-anchored geometry has no twin with the same interface reading.

Slow tests run once locally

Three cores, jaxlib 0.11.0, once each:

  • tests/property/test_coupling_nonlinear_search.py -m slow: the validation on 713 draws (worst miss 0.025 of what is allowed), the compositions on the mapped and multi-rate cells, the 30 hunts, the two pins (xfail), the unaccelerated control and the two IQN pins (xfail). 34 passed, 4 xfailed. (First run: 31 passed and three hunts, [0-radius_strict], [5-gradient] and [5-radius_strict], failed an assertion of this PR's own, the referenced fraction taken over the stepped examples instead of those that returned a finite state; no score was over its limit. Corrected, and those hunts rerun with the other radius_strict blocks: 7 passed.)
  • tests/property/test_interface_reading_twins.py -m slow: 14 passed.
  • tests/property/test_coupling_targeted_search.py -m slow (its observe was refactored: the radius scores moved into a helper the nonlinear search shares): 28 passed.
  • Not run at this head: the slow lanes of test_differential_geometry_edges.py and test_geometry_time_levels.py (the harness module they import only gained code). Their per-push lanes were run with the other touched files and the source scans: 334 passed.

🤖 Generated with Claude Code

https://claude.ai/code/session_013UkCde7g23gTziUjYAvnKD

NicholasEhsanRoy and others added 4 commits October 7, 2026 04:38
…erface-reading twins

Work in progress: the float64 pass-map reference, the nonlinear cells of the
coupling search and the twins whose interface reading is the edge-mapped
graph's.  Tests only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013UkCde7g23gTziUjYAvnKD

[skip ci]
…able

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013UkCde7g23gTziUjYAvnKD

[skip ci]
…erface-reading twins

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013UkCde7g23gTziUjYAvnKD
…urned a finite state

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013UkCde7g23gTziUjYAvnKD
…ep guard) into feat/p4-geom-diag-instruments
@NicholasEhsanRoy
NicholasEhsanRoy merged commit 7acbe0d into release/0.4.0 Oct 7, 2026
15 checks passed
NicholasEhsanRoy added a commit that referenced this pull request Oct 7, 2026
) into fix/p4-40-coupling-r8

# Conflicts:
#	CHANGELOG.md
#	docs/validation/soup_package.md
NicholasEhsanRoy added a commit that referenced this pull request Oct 7, 2026
…nto fix/p4-41-iqn-nonfinite-and-gradient-bound

No content change: the branch already carried #255's head.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013UkCde7g23gTziUjYAvnKD
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant