[META] Platform drift is systemic — plan: twin ledger, Linux-enabled differential testing, coverage ratchet

[bhelm]

This is a meta-issue: the platform drift behind #5–#16 is systemic, and fixing findings one at a time will not converge. This lays out a drift-control plan for this fork, built as small single-concern pieces so the useful parts can be offered upstream.

All references verified against f2271cd (current main).

The problem is structural, not a pile of one-off bugs

The parity model (docs/CROSS_PLATFORM.md: "parity, not shared code" — a hand-ported Kotlin mirror kept honest by discipline) has no machine behind the discipline. Twin tests pin each platform's own interpretation, so both suites stay green while the implementations drift apart. Evidence that this is systemic:

• One function, three independent divergences. Rolling RMSSD ([P1] N-05: Rolling RMSSD splices across rejected beats; Swift/Kotlin windows drift #9): window boundary — Swift keeps [t−w,t] (HRVAnalyzer.swift:362), Kotlin uses (t−w,t] (HrvAnalyzer.kt:715-719); cleaning scope — Swift cleans per window (HRVAnalyzer.swift:365-369), Kotlin cleans the whole series once (HrvAnalyzer.kt:695-708); plus Kotlin's value-based timestamp rematching (HrvAnalyzer.kt:700-708). The Kotlin comment says "matching the Swift twin" — over code that demonstrably doesn't.
• A one-sided function whose comment claims a twin. HRVAnalyzer.swift:304 documents "Kotlin twin: HrvAnalyzer.sdnnIndex" — that function does not exist; sdnnIndex/avgSdnn have zero matches under android/.
• Perfect parity on the wrong function. Both platforms wire production to the ungated resting-HR path and leave the hardened one test-only ([P1] N-08: Production resting-HR path has no artifact gate — a single-sample bin can set daily RHR #10) — in lockstep, and both wrong. No amount of cross-platform comparison can see this class; only a callsite inventory can.
• Drift where Kotlin is right and Swift silently loses data. backfillWorkout rebuilds WorkoutRow and drops steps ([P1] backfillWorkout rebuilds WorkoutRow and drops stored steps — silent data loss + Kotlin parity drift #16, WorkoutDetector.swift:378-387); the Kotlin twin uses real.copy(...) and is correct (IntelligenceEngine.kt:1739-1744).
• Fix-once-miss-the-copies has already happened. The day-zone fix landed in one copy while its duplicate stayed broken ([P2] N-06: Daily windows use one UTC offset and fixed 86,400 s days (DST/travel drift) #11). The tree currently holds ≥8 dayString implementations and ≥4 independent Pearson correlations in Android UI files alone (CompareScreen.kt:345, LabBookScreen.kt:750, MindSection.kt:309, InsightsScreen.kt:1712).
• The surface is large. Counting same-name function declarations in mirrored file pairs across Packages/{StrandAnalytics,StrandImport,WhoopStore} ↔ com.noop.{analytics,ingest,data}: ~95 file pairs, ~650 twin function pairs. Nothing machine-checks any of them.

Each fixed finding is O(1); the drift source (every twin-touching commit) is continuous. Without a system, this issue tracker refills itself.

The plan (this fork; small single-concern PRs)

Design principle: every tool is a local command first (matching Tools/doc_comment_lint.py + baseline idiom); CI wiring is fork-side convenience, since Actions is deliberately off upstream (docs/SAFEGUARDS.md).

1. Twin ledger lint — a stdlib-Python check over both trees: every declared twin reference in source comments must resolve on the other side (the tree already carries hundreds of such annotations); constants pair up value-for-value; "has test callsites but no production callsite" is an error (catches the [P1] N-08: Production resting-HR path has no artifact gate — a single-sample bin can set daily RHR #10 class); duplicate/callsite counters ratchet against a baseline (catches the [P2] N-06: Daily windows use one UTC offset and fixed 86,400 s days (DST/travel drift) #11 class and the Pearson copies).
2. Targeted fixes, one PR each — [P1] backfillWorkout rebuilds WorkoutRow and drops stored steps — silent data loss + Kotlin parity drift #16, the missing avgSdnn decision, [P1] N-08: Production resting-HR path has no artifact gate — a single-sample bin can set daily RHR #10 wiring, each with a test that fails before and passes after.
3. Linux enablement (~10 lines) — two #if canImport gates (RawOutbox.swift / BackupSettings.isBoolean) plus one XCTExpectFailure test gate. Verified locally: with a snapshot-enabled SQLite, Packages/StrandAnalytics builds on Linux and all 1,424 tests pass. Standalone benefit regardless of the rest: Linux contributors can finally run swift test on the analytics package.
4. Differential harness — with both implementations runnable on one Linux machine, feed them the same inputs (curated edge cases + seeded random generators) and compare outputs directly. The implementations become each other's expected values: no golden-file curation, no macOS requirement. Sharpness is proven, not assumed: side-specific sabotage runs must go red, and the harness must reproduce the three known RMSSD divergences before any of them is fixed.
5. RMSSD convention decisions — pick one semantics per divergence, align, prove via the harness (red → green → red on revert).
6. Coverage ratchet — every twin function is differential-covered, platform-test-covered (with executed-line proof from coverage data, not a label), or explicitly exempted with a reason; exemption lists compare against the merge-base so they cannot be widened in the same PR that violates them. Armed module-by-module, never all at once.
7. Rollout — module by module (HRV → strain → recovery → sleep), up to whole-pipeline analyzeDay differential runs (both sides are pure functions with mirrored signatures: AnalyticsEngine.swift:12, AnalyticsEngine.kt:239).

Honest limits, stated up front: a direct comparison cannot see bugs both platforms share (that stays with independently-derived oracle cases), and Linux-Swift ↔ JVM is a proxy for the shipped Darwin/ART pair (mitigated, not eliminated, by occasional macOS runs).

Upstream intent

cc @ryanbr — no action needed on this issue; flagging it early so nothing lands as a surprise, and early feedback can still shape scope and conventions. The idea is to offer pieces upstream in digestible, single-concern order, each independently useful and each a local command (no CI required):

1. Bug-fix PRs first (from step 2/5 above) — smallest possible diffs, each with its red-before-fix test.
2. The twin ledger lint — same shape as the existing doc_comment_lint.py; it mechanizes the value-for-value discipline that docs/CROSS_PLATFORM.md already promises.
3. The Linux gates — ~10 lines, Darwin behavior untouched.
4. The differential harness only after a dedicated coordination issue, with the evidence table of what it caught by then — adopt or decline as you see fit.

Enforcement (required checks, workflow gating) stays on this fork unless there's appetite for it upstream.

Branch plan & checklist

Integration model: every work branch is cut from main (synced to upstream f40897a) so it stays clean for a later upstream PR; each merges into drift/meta (the integration branch) where everything is tested together. Fork PRs target drift/meta; upstream PRs are made later from the individual branches. Dependencies: 04 builds on 03; 05 builds on 04 (its red-proof needs the harness); 06+ integrate on drift/meta.

• drift/meta — integration branch (created first, collects everything)
• drift/01-twin-ledger — twin ledger lint + baselines (plan step 1)
• drift/02a-fix-backfill-steps — [P1] backfillWorkout rebuilds WorkoutRow and drops stored steps — silent data loss + Kotlin parity drift #16 backfillWorkout drops steps (plan step 2)
• drift/02b-avgsdnn — resolve the phantom sdnnIndex/avgSdnn twin: port or correct the claim (plan step 2)
• drift/02c-fix-restinghr-wiring — [P1] N-08: Production resting-HR path has no artifact gate — a single-sample bin can set daily RHR #10 production uses the ungated resting-HR path (plan step 2)
• drift/03-linux-enablement — #if gates + CI toolchain notes; StrandAnalytics builds & tests on Linux (plan step 3)
• drift/04-differential-harness — runners + comparator + generators + sabotage proofs; must reproduce the three RMSSD divergences (plan step 4)
• drift/05-rmssd-alignment — convention decisions + alignment, proven red → green → red-on-revert (plan step 5)
• drift/06-coverage-ratchet — coverage-proven ratchet with merge-base comparison, armed module-by-module (plan step 6)
• drift/07-rollout-<module> — one branch per module (hrv, strain, recovery, sleep, pipeline) as coverage expands (plan step 7)

Upstream offering happens per branch, in the order described above (fixes → lint → Linux gates → harness after coordination), each re-based cleanly because no work branch contains another branch's commits except where declared.

[bhelm]

Progress: three branches are merged into drift/meta (each with an independent read-only Codex review before merge, findings folded in):

• drift/03-linux-enablement (drift/03: StrandAnalytics builds and tests on Linux #19) — StrandAnalytics builds and tests on Linux: 1424 tests, 1 skipped, 0 failures. Known limit: the standalone WhoopStore test target references the gated raw-outbox API and doesn't compile on Linux (possible follow-up: gate those three test files).
• drift/02a-fix-backfill-steps (drift/02a: backfillWorkout no longer drops stored steps #20) — fixes [P1] backfillWorkout rebuilds WorkoutRow and drops stored steps — silent data loss + Kotlin parity drift #16; red-before-fix, integrated suite on drift/meta: 1426 tests, 0 failures.
• drift/02b-avgsdnn (drift/02b: retire the phantom sdnnIndex twin claim #21) — retires the phantom sdnnIndex twin claim; the port-or-declare decision is avgSdnn/sdnnIndex: Swift-only metric whose doc comment claims a nonexistent Kotlin twin #18.

drift/01-twin-ledger is built and already earning its keep — on today's tree it finds, beyond the known items (the sdnnIndex dead reference, the test-only hardened resting-HR path on both platforms, 4 duplicate Android-UI Pearson implementations, 12 dayString implementations), two previously unreported constant drifts: SleepStagerV2 padLo/padHi are 330/390 in Swift vs 360/420 in Kotlin. Its own review round returned findings on the ledger's parser coverage (expression constants like 48 * 3_600, trailing-comma arity, extension receivers, computed properties, CI path filters, inventory scope excluding WhoopProtocol); a fix round is running before it merges — the PR follows once it's green again.

Android toolchain on the build host is verified: testFullDebugUnitTest = 3974 tests, 0 failures, 5 skipped (baseline for 02c and the harness work). Next up: ledger fix round, then drift/02c-fix-restinghr-wiring and drift/04-differential-harness.

[bhelm]

Progress, part 2:

• drift/01-twin-ledger (drift/01: the twin-parity ledger lint #23) is merged. The first review round (independent Codex, verdict red) found real parser/enforcement gaps — expression constants read as their first literal, trailing-comma arity, extension receivers, computed properties, CI path filters missing the scanned sources, inventory scope excluding WhoopProtocol, baseline identities without value fingerprints. All eleven findings were fixed; the focused re-review came back green with zero findings. On the integrated branch the ledger now reports 3 baselined findings as disappeared (the sdnnIndex dead reference retired by 02b, the test-only resting-HR path wired by 02c) — the improve/ratchet semantics working on real history.
• drift/04-differential-harness (drift/04: differential parity pilot — the twins test each other #24) is merged. The pilot reproduces all three known RMSSD divergences as machine diffs (SUMMARY cases=14 diffs=3) with trimpToStrain green as control — verified end-to-end with the real Gradle runner on both negative-side sabotage proofs as well. Hardening from its review round: a parity run can no longer be up-to-date-skipped or served from Gradle's build cache, and a half-set env contract is a hard error instead of a silent skip. Honest limit, stated in the PR: output equality ≠ proof that production code ran — that attestation is step 6's coverage ratchet.
• Full suites at this point: Swift 1429 / 2 skipped / 0 failures; Android 3977 / 0 failed / 6 skipped.

drift/05-rmssd-alignment is building now (conventions: adopt (t−w,t], per-window cleaning, drop value rematching; proven red → green → red-on-revert through the pilot). drift/06-coverage-ratchet follows.

[bhelm]

Final progress for this build cycle — every planned branch (steps 1–6) is built, independently reviewed, merged into drift/meta, and measured on the integrated tree: #19 (Linux enablement), #20 (#16 steps fix), #21 (#18 phantom twin), #22 (#10 resting-HR wiring), #23 (twin ledger), #24 (differential harness), #25 (RMSSD alignment), #26 (coverage ratchet), plus drift/01b-callsite-defaults (ledger callsite matching learns defaulted parameters; merged directly, its PR had no delta left).

Integrated state of drift/meta:

• Swift suite 1429 / 2 skipped / 0 failures (Linux, incl. GRDB integration tests); Android testFullDebugUnitTest 3977 / 0 / 6.
• Twin ledger: exit 0, 274 baselined findings; on real history it retired the entries healed by 02b/02c on its own improve semantics.
• Differential pilot: 14 cases, 0 diffs — after reproducing all three known RMSSD divergences as its red proof, which drift/05: rollingRmssd aligned — (t−w,t], per-window cleaning, no rematching #25 then fixed ((t−w,t], per-window cleaning, no rematching; red → green → red-on-revert).
• Coverage ratchet armed for StrandAnalytics ↔ com.noop.analytics: differential=2, platform-test=0, exempt=2719 (every entry with reason + issue); verify green with fresh real coverage from both runners.

The fund table this cycle (all machine-found or machine-confirmed): steps data loss (#16, fixed), phantom sdnnIndex twin (#18, decision pending), production resting HR ungated on both platforms (#10, fixed — behavior change spelled out in #22), 3 RMSSD divergences (#9, fixed), 2 previously unreported constant drifts (SleepStagerV2 padLo/padHi 330/390 vs 360/420 — fix PR still to come), the genuinely test-only API surface now inventoried in the ledger baseline (e.g. BatteryPackInfo.decode, the WhoopStore highwater family), and the documented one-sided windowSec default.

Still open, deliberately: drift/07-rollout-<module> (module-by-module expansion as coverage grows), the padLo/padHi fix, the #18 decision, the WhoopStore-test-target Linux gates, and every upstream offering — each of those waits for its own go, per the branch plan above.