Skip to content

fix: empty-window encoding and exact Fréchet; run-2 pre-registration Amendment 2 - #14

Merged
Mattral merged 3 commits into
mainfrom
fix/frechet-symmetric-and-empty-windows
Oct 9, 2026
Merged

Mattral merged 3 commits into
mainfrom
fix/frechet-symmetric-and-empty-windows

Conversation

@Mattral

@Mattral Mattral commented Oct 9, 2026 •

Copy link
Copy Markdown
Owner

Summary

Two fixes found by the pre-registered blind test on run 1's footage, plus Amendment 2 to the run-2 pre-registration. Everything here precedes any run-2 recording.

1. Encoder crash on empty windows (399164e, carried over: PR #13 was merged before this fix was pushed)
A window with nothing detected in any frame is fully masked. PyTorch's inference fast path raised to_padded_tensor: at least one constituent tensor should have non-zero numel, because the analyzer encodes windows one at a time. Such windows are now unmasked; presence-weighted pooling already discards their frame outputs, so every empty window maps to one "nothing detected" summary. A regression test reproduces the exact error on the old code.

2. Fréchet safeguard misfire (d326435)

  • Problem: in a condition where every window was empty (noise_32 in the blind test), all latents are identical and the shrunk covariance is zero. The old ratio test, imag/real > 1e-3 on sqrtm(Σ_A Σ_B), then divided by about 1e-12 and refused, although the distance is exactly ‖μ_A − μ_B‖² + tr(Σ_A).
  • Fix: the symmetric formulation tr((Σ_B^½ Σ_A Σ_B^½)^½) = Σᵢ √λᵢ, with eigenvalues clipped at zero. It's exact, has no free parameter (unlike a ridge ε), and is stable for singular and near-singular covariances.
  • Verified:
    • matches sqrtm on 20 well-conditioned pairs to 1e-9;
    • V2's published Fréchet values reproduce exactly (0.000263 overall; each per-level value; ρ = +0.943);
    • the exact closed form at Σ_B = 0;
    • continuous as a condition approaches all-empty;
    • against the old code, the near-degenerate test fails.
  • Spec 7.1 deviation documented. sqrtm_had_complex_component is kept for API compatibility and is now always False.

3. Amendment 2 (037670a), dated 2026-10-09, before any run-2 data:

  • §7.1: the symmetric Fréchet computation. Every condition gets a gap score; none is ranked or dropped by convention on the gap side.
  • §8.4, sensitivity, equal prominence: Rules A and B also computed without conditions whose landmark error is undefined. Those are ranked worst by convention, not measurement, so they can contribute partly definitional agreement. Both are reported side by side. If they disagree, the headline says so and no single verdict is claimed.
  • §8.3: a per-take breakdown for every condition. In the blind test, under noise at std 20, MediaPipe tracked 100% of frames in two takes and never found the body in the third, so pooled means can hide all-or-nothing behaviour. The present-frames-only gap counts as "not computable" below 3 such windows.
  • §12: the blind-test outcome.

The analysis script implements both analyses (rules_all_24, rules_measured_only, rules_disagree) and prints the per-take table.

Blind test, after both fixes (run 1's 3 clean takes, 300 frames each, scores discarded unread): every stage passes, with 24 finite gap scores, finite CIs for both rules, the sensitivity analysis, all secondary analyses, 72 per-take rows and the report.

Verification

  • pytest -q: 192 passed (105 s)
  • ruff check .: clean
  • V2 re-run: values identical to the runbook
  • Privacy greps clean
  • CI green on 3.10 / 3.11 / 3.12

🤖 Generated with Claude Code

Mattral and others added 3 commits October 9, 2026 12:05
A window with no detection in any frame is fully masked; PyTorch's
inference fast path raised "to_padded_tensor: at least one constituent
tensor should have non-zero numel" (windows are encoded one at a time),
and training-mode attention is undefined there. Unmask fully missing
windows; masked_mean_pool already discards their frame outputs, so all
empty windows map to one "nothing detected" summary. Found by the run-2
blind test; regression test reproduces the exact error on the old code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ovariance

The trace term took sqrtm of the non-symmetric product and raised when
imag/real exceeded 1e-3. With a zero (shrunk) covariance the real part
is ~1e-12, so the test divided by ~zero although the distance is exactly
||mu_a - mu_b||^2 + tr(Sigma_a). Use tr((Sb^1/2 Sa Sb^1/2)^1/2) =
sum sqrt(eig), clipped at zero: exact, no free parameter, stable for
singular and near-singular covariances. Matches sqrtm on well-conditioned
data to 1e-9; V2's published values reproduce exactly. Spec 7.1
deviation documented.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… output

Amendment 2 (2026-10-09, before any run-2 data), prompted by the
pre-registered blind test: symmetric Frechet (section 7.1); Rules A and B
also computed without conditions whose landmark error is undefined,
reported with equal prominence, no single verdict if they disagree;
per-take breakdown for every condition; present-frames-only gap not
computable below 3 windows; blind-test outcome recorded. The analysis
script implements both analyses and prints the per-take table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Mattral
Mattral merged commit b612e4e into main Oct 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant