test: score the convergence predicates against an external oracle (#861)

[InauguralPhysicist]

Reopens #861. I wrote #889; this is a correction to its scope, not a new finding.

The defect is a region, not a point

The saturation gate lands at |v| >= 1e308. The false converged starts ~305 decades earlier:

x *= 1.5   first certifies `converged` at x = 287,626      (~1e5)
x *= 2.0   first certifies `converged` at x = 8,388,608

h_low = 0.1 against H(x) = H_b(1/(1+|x|)) puts every |x| > ~76 under the entropy floor:

held value	report	converged
76	equilibrium	0
77	converged	1
1e15	converged	1
1e307	converged	1
1e308	diverging	0

A linear runaway y is y + 1.0 exits loop while not (converged of y) at y = 88. #889 closed one point at the top of [77, 1e307].

The oracle

tests/test_convergence_oracle.eigs — 27 sequences whose behaviour is known analytically, no implementation consulted. The observer is the one component in this project that never had an external reference to be wrong against; every other subsystem has one (Blargg ROMs, the Go liferaft, drat-trim, POSIX, the C compiler), and every other subsystem is correct.

ENTROPY CHANNEL (what converged/report use): 19/27  FP=3  FN=5
VALUE CHANNEL   (report_value)             : 25/27  FP=2  FN=0

Cases 25–27 are the cleanest single result — the same computation (halve the distance each step) targeting 5000, 5 and 0.005:

target	report
5000	converged
5	equilibrium
0.005	converged

Identical mathematics, identical convergence rate, verdict decided by the units you measured in.

Two corrections made while landing it

1 — Ground truth was wrong on three labels. Cases 6, 7 and 8 (1 + 1/n, (-1)ⁿ/n, Basel partial sums) were labelled "settles", but their tails decay like 1/n, so at step 60 they sit 1.6e-2, 1.6e-2 and 9.9e-3 from their limits — outside the file's own 1e-9 criterion. They converge; they have not arrived. Verified against the closed forms.

This moves entropy 16→19 and value 26→25: two of the value channel's apparent wins were scored against mislabelled truth and are really false positives. The direction of the argument is unchanged, but the "26/27, zero FP" headline was flattered.

2 — Aggregate scores alone are too coarse a gate. Planting a real semantics change (h_low 0.1→0.5, which makes a held 10.0 certify converged) left both scores unmoved while flipping cases 7 and 13 from stable to moving — offsetting changes cancel in a count. Per-case bands are pinned too; that version catches it and names the cases:

BAND DRIFT case 7 ((-1)^n/n (slow)): report was stable, now moving
BAND DRIFT case 13 (x += 1 -> INF): report was stable, now moving

Green, not red

The baseline is pinned, not failing. A permanently red suite erodes every other gate, and [99c] is the existing precedent for inverting an expected failure into a pass. Movement in either direction fails, so a fix must land as a deliberate, reviewed edit here rather than silently changing what a green suite means. The file header states plainly that it pins a defect baseline, not a target.

test_observer_saturation.eigs

Case 4 asserted "1e307 constant should still converge". A constant sequence does converge, so the assertion was not false — but the entropy channel cannot distinguish it from a runaway that flattened there, so pinning the verdict pinned the region. It now asserts only that the saturation gate is tight to the ceiling, which is the property that is actually defensible.

Gates

gate	result
release suite	3787/3787, 0 failed
ASan+UBSan detect_leaks=1	3785/3785, 0 failed — LeakSanitizer tally at its 0 floor

Not in scope

The fix this corpus argues for — routing the predicates to the value channel, dropping h_low from converged, and making tolerance visible and scale-relative — touches the entropy-constant fence and the language's headline vocabulary. That is a design decision, not a runtime-engineering one, and it is deliberately not in this PR. This PR only makes the defect measurable and prevents it from drifting.

Related: #862 (the H(x) ≡ H(1/x) level sets), #865 (saturation indistinguishability).

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Adds a pinned, external-oracle regression test for the observer convergence predicates (issue #861), and adjusts the existing saturation test so it no longer implicitly endorses the currently-defective “large constant” convergence region below the saturation ceiling. This fits the codebase’s testing philosophy by anchoring observer behavior to an implementation-independent reference and preventing silent semantics drift.

Changes:

• Add tests/test_convergence_oracle.eigs, scoring report/converged (entropy channel) and report_value (value channel) against 27 analytically-understood sequences, and pin both aggregate scores and per-case bands.
• Update tests/test_observer_saturation.eigs case 4 to only assert that the saturation gate does not fire below 1e308 (rather than asserting converged is “correct” at 1e307).
• Register the new oracle suite in tests/run_all_tests.sh as a pinned baseline gate.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
tests/test_observer_saturation.eigs	Narrows the below-ceiling assertion to “not saturated” (no longer pins the defective converged region).
tests/test_convergence_oracle.eigs	Introduces an external-oracle baseline that pins current predicate performance and per-case verdict bands.
tests/run_all_tests.sh	Adds the oracle suite as a single-slot gate in the observer section.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.