Sleep: "Bridge brief wakes" (experimental, default OFF) — one session per night across brief wakes - #1605
Open
vishk23 wants to merge 209 commits into
Open
Sleep: "Bridge brief wakes" (experimental, default OFF) — one session per night across brief wakes#1605vishk23 wants to merge 209 commits into
vishk23 wants to merge 209 commits into
Conversation
Off-by-default OAuth (BYO Oura app, authorization-code) one-time pull of the full Oura API v2 surface. Lossless raw archive (new ouraRaw table, migration v19) plus normalized projection into existing WhoopStore tables, incl. the sleep_phase_5_min hypnogram the file-import lane can't carry. Networking lives in the app target beside AICoach; StrandImport gains only pure API parsers and stays offline-pure. New DataSourceKind.ouraApi; honest-data doctrine preserved.
… store) TDD task breakdown for the network-free core: ouraRaw archive (migration v19) + OuraRawStore, the OuraHypnogram decoder, and the pure OuraApiParser (sleep/daily/events) reusing OuraExportParser semantics. Additive only.
Rebased target is now ryanbr/noop v8.5.2 (NoopApp/noop upstream went private). Study of v8.5.2 vs the v7.2.3-era design, verified against code: - ouraRaw migration is v24 (migrator is at v23, not v18); v19 already taken. - Provenance uses existing PairedDevice.sourceKind .cloudImport (non-day-owning via IntelligenceEngine:1369) — no new SourceKind; deviceId stays 'oura-api'. - motionJSON goes through persistSessionMotion(), not the sleep upsert. - Add oura-api to Repository.wearableImportSources; disconnect via store actor. - Citation shifts (parseISOWithOffset -> CSVParsing:509). Foundation design (models, 5 upsert APIs, tables, parsers) confirmed intact.
…ement, in-sleep HR
…m (Task 3 review)
…pt reference-only
Keychain token store, OuraCredentials (xcconfig->Info.plist), OAuth2 authorization-code (BYO Oura app) behind an AuthProvider seam, and the URLSession client (next_token paging, 429 backoff, 401 refresh, sandbox/prod). Networking confined to Strand/Oura/; packages stay network-free. Tested via xcodebuild StrandTests with a URLProtocol stub + pure builders/parsers.
… test was broken) The app PRODUCT_NAME is 'NOOP Staging' but the StrandTests TEST_HOST/BUNDLE_LOADER still pointed at NOOP.app/NOOP, so xcodebuild test failed with 'Could not find test host' for the whole macOS scheme. CI only runs build (never test) so it went unnoticed.
getWithRetry's 401 branch called validAccessToken(), which only refreshes when clock-expired -- a 401 for a revoked/invalidated-but-unexpired token re-served the same rejected token and the retry failed silently. Add refreshedAccessToken() to AuthProvider for an unconditional refresh; validAccessToken() now delegates to it on expiry. Cover both the 401 force-refresh path and 429 exhaustion with tests.
Wires the OURA_CLIENT_ID/SECRET/REDIRECT_URI Info.plist keys (read by OuraCredentials.fromBundle) and the noop:// CFBundleURLTypes scheme into both the Strand (macOS) and NOOPiOS targets via a new untracked Strand/Oura/OuraSecrets.xcconfig (gitignored; committed template at OuraSecrets.example.xcconfig). Also regenerates Strand/Resources/Info.plist and StrandiOS/Resources/Info.plist via xcodegen generate — both targets set GENERATE_INFOPLIST_FILE: NO, so xcodegen pre-materializes these tracked plists from project.yml's info.properties (same convention as the prior "regenerate Info.plist to match project.yml" commits). Escapes the redirect URI's "//" with the empty-macro $() trick in both xcconfigs: .xcconfig treats // as a start-of-comment anywhere on the line (not just at line-start), so a literal noop://oura/callback silently truncates to noop: at build time and would break the redirect_uri match against the app registered at the Oura developer portal. Verified: xcodegen generate + macOS build-for-testing succeeds, and the built app's Info.plist resolves all four keys correctly (not left as literal $(OURA_*)), including the full noop://oura/callback string. NOOPiOS gets the same wiring but isn't build-verified in this environment.
…ing base64 secrets)
OuraSyncWriter (persist backfill to WhoopStore as .cloudImport, honest-data + coalesce + persistSessionMotion + hrSample + raw archive), OuraSyncCoordinator (one-time backfill across all endpoints, readiness-RHR precedence), deleteOuraRaw for clean disconnect, the Connect Oura card + presentation-anchor helper, scoring source + docs. Verified against v8.5.2 signatures.
…nvariant) + harden test OuraSyncWriter.persist registered the Oura PairedDevice with status: .active, creating a second active row alongside the live WHOOP. DeviceRegistryStore.add does not enforce the at-most-one-active invariant (only setActive() does), and activeDeviceId() feeds day-owner priority-0 plus BLEManager's live-sample deviceId routing, so this was a real data-integrity violation. Every other PairedDevice construction in the app already uses .paired; an import source must never be the active device. Also hardens OuraSyncWriterTests: exact hrSamples count (2, not just >0), asserts the registered source is .paired, and reads the day back via dailyMetrics() to prove recovery/strain stay nil (honest data).
…nts, coalesced days
…se (Task 2 review)
…isconnect (Task 4 review)
… on disconnect; ouraRaw doc accuracy (final review)
`ppgWaveformSample`'s migration (v27) carries an explicit CONSUMER STATUS note ending "Do NOT 'clean up' the reader as dead code: the rows are the point, and the reader is how they are reachable". `v18AuxSample` (v31 / Room MIGRATION_24_25) is in exactly the same position — every `v18AuxSamples` call site on both platforms is a test — and carried no such note. A future tidy-up pass reading it cold would delete a reader that is deliberately unused, and with it the only way the banked rows are reachable. Adds the equivalent note on both platforms, and records the part that is easy to lose: before this migration those fields were not merely unread, they were DESTROYED. The strap trims its history the moment an offload is acked, so each one was unrecoverable and could never be censused. The migration converts permanent loss into retained-but-unread, which is the whole fix and is complete. Fifteen of the slots are unpinned bytes whose names deliberately assert nothing, so wiring them to a consumer before a census would be the overclaiming this project has already had to retract. Also names the four sibling columns the same migration added (`gravitySample.dynAccel`, `sleepStateSample.rawByte`, `skinTempSample.aux1Raw/aux2Raw`), which are SELECTed into their structs with no consumer touching the properties, on purpose. Comments only — no schema, no behaviour, no stored value changes. Verification: `swift build` (WhoopStore) clean; macOS `Strand` builds; Android `assembleFullDebug` + `testFullDebugUnitTest` — 3209 tests, 0 failures, counted from the JUnit XML.
# Conflicts: # Strand/Resources/Localizable.xcstrings
…tas, not snapshots Brings the liters page-replication integration and the fix that makes it worth anything onto fork main: * the xcframework link, the generated bindings, the trial switch and the per-push telemetry; * observable push outcomes, so a failing push and a push that never ran are distinguishable (build 218); * and the /ingest fallback no longer restarting the WAL underneath the replicator, which was holding the measured snapshot rate at 100%. Default is unchanged for anyone who has not switched the trial on: the trial flag is UserDefaults-absent (false), StoreReplication is never configured to .external, and every code path here reduces to what shipped before.
…on of the session (#930)
`SleepStagerV2.cyclePrior` suppressed REM while `c < 0.12` — 12 % of THAT session's
length. First-REM latency is an absolute physiological interval, so the guard's width
scaled with how long the wearer stayed in bed: 7.4–84.5 min, an 11.5x spread, across
one WHOOP 5 user's own 36 recorded nights.
Replaces the step with a graded penalty in minutes since a MEASURED sleep onset:
rem = 1.0 * c - K * clamp01(1 - m / M0) K = 3.0, M0 = 60 min
K is the incumbent step's own magnitude (e^-3 ~ 0.05 on the REM emission, strong
suppression but never a veto). The sleep-accel grid — {cliff, graded} x {fraction,
minutes} x K in 1..8 x threshold, 210 cells — ties on accuracy in every cell, so
nothing discriminates K, and the smallest correct change fixes the units and the shape
without also retuning a magnitude no measurement can justify. M0 = 60 sits inside the
measured-defensible band: against PSG truth a guard reaching 45 min costs 0.36 % of all
real REM (1 of 31 subjects) and one reaching 90 min costs 6.85 % (15 of 31).
The `1.0 * c` ramp and the deep term are untouched and stay fractions of the session —
both describe where in the night you are, which is inherently proportional. Removing
the ramp collapses REM to 1.4 % of night and kappa to 0.143; it is load-bearing.
THIS IS A ROBUSTNESS FIX, NOT AN ACCURACY WIN. On sleep-accel the step is measurably
inert (kappa 0.349 -> 0.349, median first-REM latency 142.0 -> 141.0 min when removed)
because that cohort spans only 2.4x in session length. Replayed over the 36-night
device database (15 stage-locked human-authored references, 9163 epochs): 4-class kappa
0.691 -> 0.689, sleep/wake 0.583 -> 0.582, accuracy 78.2 % both.
kappa is not the guardrail that matters here — #437 reverted #348 for exactly that
reason ("kappa doesn't guard stage-fraction calibration"), so stage fractions are
reported too. Per-night bias vs the human reference (predicted - truth, pp) improves on
three stages and is unchanged on deep: wake -7.26 -> -7.13, light -7.54 -> -6.10, deep
+6.65 -> +6.65, rem +8.15 -> +6.57. On the healthy stratum #348 broke (in-bed >= 5 h,
n = 20) nothing moves more than 0.42 pp and wake moves +0.03 pp; deep is byte-identical
on all 36 nights. 12 of 36 nights change at all, and the change is monotone in session
length — mean REM delta -20.9 pp under 2 h, -2.4 pp at 2-5 h, -0.4 pp at >= 5 h — i.e.
it lands on precisely the short sessions the fractional guard was mis-scaling.
Plumbing: `Epoch` gains `minutesSinceOnset`; `stageEpochs` stages once with the guard
disabled, takes the first sustained non-wake run as onset (5 min = 10 epochs; measured
against PSG onset at bias -3.8 min, MAE 7.4 min, n = 31), then re-applies the guard and
re-runs Viterbi. Only the guard differs between passes, so the cost is one extra
Viterbi, not a second featurisation. The guard is clamped to [0, K] so a pre-onset epoch
cannot be penalised harder than onset itself — #271 can place a window start hours early.
Android twin updated in lockstep per the parity contract, including the constants and
the frozen golden. Tests: Swift 1247 passed / 0 failed; Android 3238 passed / 0 failed
/ 5 skipped (393 suites, counts read from the JUnit XML). Both frozen goldens are
unchanged, so the recipe's pinned end-to-end shape is preserved.
Known and separately tracked, NOT addressed here: median first-REM latency runs ~54 min
late against PSG (142 vs 88.5 min truth; removing the step gives 141, so this term is
not the cause), and deep is under-called 2-3x (4.7-6.2 % predicted vs 14.8 % truth).
Refs #930.
… not just kappa
PR #348 fitted the stager to DREAMT. It raised kappa on all three benchmarks
with a held-out gap of -0.027 — clean by every ML criterion — and PR #437
reverted it 48 hours later because it re-scored a healthy night from 6% to 23%
awake. The revert's own words: "kappa doesn't guard stage-fraction
calibration."
SleepBench as it landed in #925 reproduces exactly that blind spot. Sections B
and C score agreement (accuracy, kappa, per-stage sensitivity/specificity) and
wake minutes, none of which constrain how much of the night a recipe spends at
each stage. A recipe can raise kappa while systematically reallocating stages,
because the epochs it newly gets right can outnumber the epochs it newly
mislabels.
Adds two sections. Nothing existing is replaced; the old section E is renamed G.
E. PER-STAGE FRACTION CALIBRATION. Predicted % of night vs reference % of
night for wake/light/deep/REM, as a signed bias in percentage points,
per night and aggregated, plus the unsigned MAE so a recipe that over- and
under-calls in equal measure cannot pass as well calibrated.
- E.0 prints accuracy and kappa on the SAME nights, so the number that is
not sufficient sits directly above the numbers that guard it.
- E.2 reports a HEALTHY stratum (in-bed >= 5 h) separately. The aggregate
hides the failure: #348's healthy-night blowout was ~17 pp of wake, which
pooling with short fragmented nights dilutes.
- E.3 reports the shipped population's stage fractions over every replayed
night with no reference at all, so a distribution shift is catchable on a
database with no human labels.
F. FIRST-REM LATENCY. Minutes from staged sleep onset to the first REM epoch,
per night and as a median/p10/p90/min/max, for each recipe and for the human
reference. Calibration pins how MUCH REM a recipe emits, not WHEN; REM in
the first minutes after onset is physiologically implausible in a healthy
adult. Nights that never reach REM are counted separately rather than folded
in as a zero.
Reference set. The calibration sections score the stage-locked rows, not
section B's set. B excludes an edited night whose stored hypnogram is a
byte-exact replay of the CURRENT V2 — right for B's question, but
version-dependent by construction: change the recipe and a night can enter or
leave the exclusion, silently swapping the denominator underneath a
before/after. The stagelock cursor set comes from `cursors` and does not move
when the recipe moves.
Read-only properties are unchanged: DB.swift is untouched, the open is still
SQLITE_OPEN_READONLY | immutable=1, there is no write surface, and the database
path is still a required argument.
Verification. 13 new unit tests over the pure label-array primitives, no
database needed (swift test in Tools/SleepBench). Measured on a real 36-session
database with 15 stage-locked human-authored references (9,163 epochs), against
the previous commit as a before/after:
V2 4-class kappa 0.691 -> 0.689, accuracy 78.2% -> 78.2%
stage bias, pp wake -7.26 -> -7.13
light -7.54 -> -6.10
deep +6.65 -> +6.65
rem +8.15 -> +6.57
first-REM latency median 72.2 -> 83.2 min, minimum 10.0 -> 43.0 min
healthy-stratum wake fraction 9.40% -> 9.43% (+0.03 pp)
Kappa moved -0.002 and would have called that change nothing. The latency
minimum moving off 10 minutes, and the healthy-stratum wake fraction holding to
+0.03 pp, are the evidence — and neither was measurable in this repository
before this commit.
Section E's reference set keys on `stagelock` so the denominator cannot move between two builds — the right call, and the reason section B's recipe-dependent exclusion is not reused there. But a `stagelock` cursor proves only that the stages arrived through `edit_sleep_stages`; it does not prove they differ from what V2 emits. On the database this was developed against, 3 of the 15 stage-locked reference nights replay BYTE-EXACT from V2 and a 4th agrees at 99.09%. Those rows carry no information about V2: they hand the incumbent a guaranteed perfect night and charge every alternative recipe for the same nights. The effect is large enough to invert a verdict. Scoring PR #348's parameter set on this database gives 4-class kappa 0.640 against the incumbent's 0.689 on the contaminated 15-night set — #348 loses — and 0.705 against 0.604 on the same set with the three byte-exact nights held out — #348 wins by +0.101. Section E exists precisely to judge a #348-style tune, so a 0.085-kappa thumb on the scale in favour of "change nothing" is the one bias it must not have. Section E cannot simply drop the rows the way section B does without reintroducing the version-dependence it was built to avoid, so the harness names them instead: a new E.-1 audit lists every stage-locked night whose stored hypnogram replays from V2 at >= 95%, flags the byte-exact ones, reports what fraction of the reference set they are, and prints the exact `--exclude` line to pin the same held-out set across both builds of a comparison. Excluding by explicit timestamp keeps the set frozen; excluding by "matches this build's V2" would not. The audit asserts no mechanism for the match, only that a row cannot be an independent reference for the recipe it replays. Verification: `swift test` in Tools/SleepBench = 13 tests, 0 failures; release build clean; run against a 37-session device database copy, E.-1 names the four nights above and every downstream section is unchanged when `--exclude` is not passed. Tools-only — no analytics, no stager, no Android twin, no goldens. Refs #348, #437.
`HealthAlertBanner` was mounted only in the classic `TodayView`. Liquid Today ships as the iOS default (`noop.liquidTodayEnabled` = true), so when `AppModel.applyIllnessSignal` raised `healthAlert` on `.raised` / `.alreadyUnwell`, the screen the user actually opens showed nothing — the signal survived only in the Health tab's HeadsUpCard and the once-a-day local notification. Mount the same leaf in the Liquid section column, directly under the header scene, matching the classic placement under its top bar. It sits above the reorderable block so the Arrange sheet can't move it out of sight, and it renders nothing when there's no alert. Same amber, non-diagnostic treatment (unchanged `HealthAlertBanner` / `NoopCard` + `StrandPalette.statusWarning`). Same class of regression as #992 runtime, B1 backfill and #543 Charge carry. Verified: NOOPiOS + Strand (macOS) both build; screenshotted the seeded `--demo-screen liquidtoday` simulator build with and without an alert — banner renders under the header above the hero, and the no-alert case leaves the wordmark-to-hero spacing unchanged (no phantom gap).
…8 that survives measurement
PR #348 re-tuned seven things about SleepStagerV2 on DREAMT; PR #437 reverted all of them 48 h later
because one healthy night went from 6 % to 23 % awake ("kappa doesn't guard stage-fraction calibration").
Re-litigating that revert component-by-component, on a reference set de-contaminated with the
`--exclude` flag added in af44563, six of the seven components measure neutral-to-negative and stay
reverted. One does not: the AWAKE transition row.
Restored from #348, on both platforms:
"awake": deep 0.01 -> 0.0, rem 0.02 -> 0.0, light 0.27 -> 0.10, awake 0.70 -> 0.90
This is a physiological claim, not a fitted constant: sleep onset descends through N1/N2, so wake never
transitions straight into N3 or REM, and the freed mass makes a WASO episode span several epochs rather
than flicker back to sleep after one. The `ln(max(v, 1e-9))` viterbi floor that makes a zeroed entry
safe already landed with #348 and survived #437, so no other change is needed to support it.
MEASUREMENT (Tools/SleepBench, one wearer's 36 recorded nights).
The only reference here that V2 cannot contaminate is the strap's own band `sleep_state` — the v18 @81
nibble, WHOOP's verdict rather than a re-derivation of NOOP's. Over 21 banded nights / 15 554 epochs:
sleep/wake kappa 0.105 -> 0.118 wake sensitivity 16.0 % -> 17.6 %
accuracy 79.1 % -> 79.2 %
The #437 guard holds — this is the number that must not blow out, and does not:
healthy-stratum wake fraction (n = 20) 9.43 % -> 9.96 % (#348 entire: 32.66 %)
healthy deep 22.09 % -> 22.11 %
first-REM latency MAE 53.9 -> 41.6 min
Why the other six stay reverted, each measured alone against the same band reference / wake guard:
base priors (deep .15, awake .34) healthy wake 9.43 % -> 17.76 % the #437 blow-out, confirmed
motion gate (jerk 75/35, boost 4) healthy wake 9.43 % -> 15.92 % a SECOND wake channel, not free
emission coefficients band kappa 0.105 -> 0.094
deep gate .25 -> .40 healthy deep 22.09 % -> 25.47 % worsens an existing over-call
awake dead-zone 0.30 band kappa 0.105 -> 0.099
deep/rem/light transition rows band kappa 0.105 -> 0.101, healthy REM 29.58 % -> 33.11 %
LIMITS, stated plainly. n = 1 wearer. #348's out-of-cohort evidence was +0.028 (AAUWSS) and +0.006
(Walch); its headline +0.17 is DREAMT in-sample, on the cohort its constants were tuned to. 13 of this
database's 15 stage-locked nights are byte-exact or >= 95 % V2 replays and cannot serve as an independent
reference for V2 at all, leaving 2 usable hand-labelled nights (kappa 0.857 -> 0.863 there, a tie); the
band comparison above is the load-bearing one precisely because it does not depend on those labels.
Verification: StrandAnalytics `swift test` 1248 tests, 0 failures. Android
`:app:testFullDebugUnitTest --rerun-tasks` 3239 tests, 0 failures, 0 errors (counted from JUnit XML),
SleepStagerV2Test 12/12. Both frozen goldens pass UNCHANGED — they are insensitive to this row, which
is why this adds a direct pin for it on both platforms (twin of the existing deep-row pin), asserting
the zeros and that a zeroed entry still reaches the lattice as a finite log-weight.
…2, in the repo this time
Every reference the sleep work can currently reach is either the recipe's own output handed back to it or
another vendor's black box. A harness that scored SleepStagerV2 against real polysomnography existed once,
settled the most decisive question this project has asked — whether REM detection reads physiology or the
clock — and then vanished, because it lived in a session scratchpad. The next investigation had to run on
one instrument instead of two and said so. This is that harness, committed, with tests.
`sleeppsg --dataset <path>` replays SleepStagerV2 over PhysioNet sleep-accel (Walch, Huang, Forger &
Goldstein, SLEEP 42(12) zsz180, 2019) — 31 subjects of wrist accelerometer + heart rate recorded alongside
human-scored PSG hypnograms — and scores it epoch-for-epoch. Read-only; the dataset path is always an
argument and the dataset is never committed. README.md carries the download step and the attribution the
Open Data Commons licence and the MIT companion code require.
THE PORT, AND WHY IT CANNOT DRIFT. Baseline numbers come from StrandAnalytics.SleepStagerV2.stageSession
itself. Variant numbers cannot: the recipe holds its constants as `static let`s and keeps Epoch/features()
internal, so asking "what would this score with one transition row moved" needs the recipe restated with
knobs. RecipePort.swift does that, and PortValidation MEASURES the equivalence instead of asserting it —
48 randomised nights (off-grid starts, channel dropout, arousals, motion bursts, unsorted input, R-R
present so the RSA term is exercised, which this dataset cannot do) plus 7 degenerate cases: all-HR-
missing, zero-variance HR, 1- and 2-epoch nights, saturated motion gate, no coverage at all, motion
absent. 12 377 of 12 377 epoch labels identical. It runs in `swift test`, needs no dataset, and names the
night and epoch when it fails.
REPORTED IN THIS SHAPE ON PURPOSE. Stage fractions are a first-class result, because kappa does not
constrain them and #348 -> #437 is the proof. Splits are leave-one-SUBJECT-out, never epoch-level, and a
test fails if a held-out subject's rows can reach its own training fold. Night length is a stratification
variable rather than a footnote.
WHAT IT FOUND, on 26 773 PSG-scored epochs.
1. #987's awake row survives PSG. It was landed on band-state evidence alone. Against truth: kappa
0.356 -> 0.363, REM F1 0.569 -> 0.575, wake sensitivity 30.4 % -> 30.8 %, and the #437 guard holds
(wake 4.34 % -> 4.15 %, deep unmoved at 18.94 %). Its first-REM latency claim does NOT reproduce and
reverses: MAE 45.1 -> 49.9 min against the band's 53.9 -> 41.6.
2. #348's six rejected components: PSG agrees on five, disagrees on one. Emissions -0.012 kappa, motion
gate -0.004, dead-zone -0.001; the deep gate worsens an already +5.2 pp deep over-call to +8.9, and
the deep/rem/light rows push REM from +5.0 pp to +9.2. But the BASE PRIORS improve calibration here
(+0.002 kappa, wake bias -4.74 -> -1.20 pp, deep +5.18 -> +1.21), because against PSG this recipe
UNDER-calls wake by 4.7 pp. The #437 blow-out is a property of that wearer's nights, not a universal.
3. The REM/night-length coupling does not reproduce: predicted r = +0.037 where PSG truth itself carries
+0.398. Weak test — sleep-accel nights cluster at 476 min.
4. Physiology beats the clock, by much less than reported. LOSO logistic: clock-only 0.431, physiology
0.479, both 0.522; physiology - clock positive in 22/31 subjects and both - clock in 26/31. The
direction reproduces; the +0.329 does not, because a clock model with a train-tuned threshold is a
far stronger baseline than 0.241.
The self-check in section 3 reproduces every truth-side figure exactly (26 773 epochs, deep 14.76 %, truth
first-REM 88.5 min) and predicted deep to 0.01 pp. The REM prediction does not; the obvious explanation
(#930's guard) was tested with a dedicated variant and FALSIFIED, and the residual is left open rather
than tuned away.
The CI wiring is deliberately not here: it extends the `tools:` matrix that upstream #943 added, which
this fork has not synced yet. It rides in with the upstream sync.
Source Hygiene has been red on fork main with 5 detached doc comments, all in
Strand/CloudSync/Generated/liters_ffi.swift. That file is uniffi-bindgen output:
Rust/build-ios.sh `rm -rf`s the bindings directory and re-emits it on every build,
its header says "GENERATED by Rust/build-ios.sh ... Do not edit by hand", and it
already carries `// swiftlint:disable all`.
So the two options the gate's own message allows -- fix the comment, or scope the
path -- reduce to one. A fix does not survive: the next `Rust/build-ios.sh` deletes
it and the gate goes red again on a file nobody touched. That is the failure mode
#514 already fixed once for i18n, where a whole-tree gate red-checked every
contributor on a line none of them wrote.
Scoping is also the honest reading of what this lint is for. Its docstring names
the failure mode as editing by INSERTION rather than in place -- a hand-editing
mistake. A generator does not edit, it re-emits, so the premise does not hold for
this path. `/Generated/` sits in SKIP_PARTS alongside /build/, /.build/ and
/DerivedData/, which are the same category: output, not source. It is NOT a
baseline raise -- Tools/doc_comment_lint_baseline.txt is byte-identical, and the
25 grandfathered hand-written sites still ratchet down exactly as before.
Two of uniffi's emissions defeat a fix even ignoring regeneration:
* 4 sites are `/** ... */`, blank line, `public enum` -- uniffi's own spacing;
* 1 site (:3325) is a parameter doc inlined into an argument list,
`public init(/** Whether a full restore ran. */restored: Bool, ...)`.
That block IS correctly bound, to its parameter. The scanner walks to the
first line ending in `*/`, which that line does not, so it runs on to an
unrelated close and reports a site that is not there. Nothing to repair.
`/Generated/` currently matches exactly one directory, Strand/CloudSync/Generated,
verified against all four globbed roots.
Verification: `python3 Tools/doc_comment_lint.py` exits 0 --
"OK no NEW detached doc comments (1467 files, 25 baselined site(s) remaining)",
against 5 regressions before. Baseline file untouched.
Not fork-specific: any repo that checks in generator output hits this, so this is
a candidate to send upstream rather than carry.
The tool has always printed both conventions, labelled, one table above the other. The README quoted
one of them in the self-check and the other in the variant section without ever saying so, which read
as two different answers to the same question:
section 3, pooled over all scored epochs deep truth 13.76 % predicted 18.94 % bias +5.18 pp
section 3, mean of per-subject percentages deep truth 14.76 % predicted 19.24 % bias +4.48 pp
Both are right and both are measured on the same 26 773 epochs. The denominator WITHIN a night is
identical either way — all scored epochs, wake included. What differs is the weighting ACROSS the 31
subjects: pooled gives a 490-minute night more weight than a 208-minute one, the per-subject mean gives
every subject one vote. For deep that is a ~1 pp gap, which is large enough to look like a contradiction
and small enough to go unnoticed.
The self-check is stated in the per-subject mean because that is the convention the previous harness
reported in, and reproducing it on any other convention would not be reproducing it. Section 6's variant
table and every `bias pp` column in it are pooled, which is why the incumbent's deep bias reads +5.18 pp
there. Neither changes; both are now named at the point of use.
Adds a convention table to the README, labels every percentage in the self-check section, and records
the same thing on `Variants.preNine30Guard`, whose docstring quotes the per-subject figures. Also
un-rounds "REM 27.0 %" to 26.96 % — 27.0 was ambiguous between the two conventions (26.96 per-subject,
27.02 pooled) precisely where the distinction matters.
Documentation only: no measurement, no threshold and no reported number changes. Verified against the
full 31-subject cohort locally; `swift test` in Tools/SleepPSG 33 tests, 0 failures.
# Conflicts: # .gitignore # Packages/StrandAnalytics/Sources/StrandAnalytics/SleepDebt.swift # Packages/StrandAnalytics/Sources/StrandAnalytics/SleepStagerV2.swift # Packages/StrandAnalytics/Tests/StrandAnalyticsTests/SleepDebtTests.swift # Packages/StrandDesign/Sources/StrandDesign/NoopMotion.swift # Packages/StrandDesign/Tests/StrandDesignTests/QuietMotionCoverageTests.swift # Packages/WhoopProtocol/Sources/WhoopProtocol/R22Disable.swift # Packages/WhoopProtocol/Tests/WhoopProtocolTests/R22DisableTests.swift # Strand/BLE/BLEManager.swift # Strand/Screens/SettingsView.swift # Strand/Screens/SleepView.swift # Strand/System/ScheduledDebugExport.swift # StrandiOS/App/StrandiOSApp.swift # StrandiOS/Health/HealthKitBridge.swift # android/app/src/main/java/com/noop/analytics/SleepStagerV2.kt # android/app/src/main/java/com/noop/ble/WhoopBleClient.kt # android/app/src/main/java/com/noop/protocol/R22Disable.kt # android/app/src/main/java/com/noop/ui/SettingsScreen.kt # android/app/src/main/res/values-de/strings.xml # android/app/src/main/res/values-es/strings.xml # android/app/src/main/res/values-fr/strings.xml # android/app/src/main/res/values-pt-rPT/strings.xml # android/app/src/main/res/values-zh/strings.xml # android/app/src/main/res/values/strings.xml # android/app/src/test/java/com/noop/protocol/R22DisableTest.kt
…rs is live
Every /ingest resets the server-side liters lineage (correct and unavoidable
once the mirror is replaced wholesale), which forces the phone's next push to
a full ~390 MB snapshot — which cannot complete in a background window, which
fell back to /ingest again. Measured on-device 2026-08-03: 27 consecutive
full-size segments; the one lineage that survived produced deltas of 125 KB
and 198 B. The old rule (any non-push falls through to /ingest) even fired on
liters' healthiest outcome: 'already in sync, nothing to ship' was answered
with a 208 MB re-upload that destroyed the lineage that made the no-op
possible.
uploaded == 0 is four states, not one, so classify instead of guessing:
- pushed -> success, retry streak resets
- inSync -> verified (txid > 0, remote == local): success, 0 bytes.
txid == 0 'synced' is vacuous (both sides agree on
nothing, observed 17:16 right after a reset) and is
NOT blessed as success — that would stamp
lastUploadToken and make the next sync skip the
upload entirely.
- retryable -> throw litersRetryPending; the next sync retries the
cheap delta with the lineage intact
- unavailable -> trial off / not compiled / no destination: /ingest
is simply the path, unchanged
After litersRetryThreshold (3) consecutive retryables the fallback opens: a
genuinely dead liters path (sink secret lost, rotated token, FFI regression)
degrades to the proven whole-database upload rather than silence. The streak
persists across relaunches and resets only on a liters success, so a
persistently dead path ships every sync instead of every third.
Verified: StrandTests (LitersFallbackPolicyTests + CloudSyncUploaderTests,
14/14) via xcodebuild test on macOS, CI-parity unsigned; NOOPiOS compile with
LITERS validated separately. Deep-buffer drain and the edit pull are
unaffected by the new throw — both run outside the upload's do/catch.
…icting them The reject archive is the only surviving copy of a historical record NOOP cannot decode — the strap frees it the instant the trim is acked. Two ways an on-demand WHOOP MG ECG record could still be lost. 1. The archive filter asked the wrong question for an unmapped layout. It kept a type-47 record only when the DECODE produced nothing usable (no unix, or neither heart rate nor motion), so a record from a layout with no field map that happened to yield those fields passed the screen and was kept nowhere at all. That has held so far only by accident: the unmapped branch of decodeWhoop5Historical reads no offsets, so nothing but hist_version ever reaches `parsed` today. One static schema field for type 47, or one partially-mapped new version, would silently reopen it. The decision is now structural: mappedWhoop5HistoricalVersions ([18, 20, 21, 26]) is the single source of truth behind both the decoder's dispatch and the archive filter, and any 5/MG record outside it is archived whatever it decoded. v18/v20/v21/v26 are unaffected, so nothing already understood starts being archived wholesale. WHOOP 4 is untouched: its unmapped versions go through the schema's v24 fallback, which keeps a record only when it validates physiologically and otherwise drops the biometrics, so those records already reach the archive by the decode-outcome route. 2. Retention evicted the evidence. With enable_raw_data_w_ecg armed the strap banks one 1584-byte record per second whose every byte from offset 21 to the CRC32 trailer is zero (invariant across 1,307 captured frames). At ~3.2 KB per hex JSONL line that is the whole 5 MB archive in ~25 minutes, so a real contact-window record was evicted by placeholders within the half hour. Eviction is now weakest-value-first rather than purely oldest-first: malformed lines, then surplus lines whose payload region is ENTIRELY zero, then surplus informative lines. The floor splits with it — the newest 64 informative lines per version stay protected, all-zero lines get a floor of 8, enough to date the artefact without owning the archive. The test is strictly all-zero: one non-zero byte anywhere makes a frame informative, because a small populated block inside an empty buffer is exactly what the archive is for. The cap stays 5 MB (volume was never the missing thing), and an over-cap rewrite now evicts to 7/8 of the cap so a 1 Hz stream no longer rewrites the whole file on the main actor once per record. Adds an operator affordance for checking after an experiment whether the record landed: Devices -> MG ECG -> "Check archived raw records" renders counts per layout version split by informative vs all-zero payload, plus the newest captures' phone-clock timestamps. Local file read, no strap traffic. The ECG opcode sequences are untouched. Tests: swift test (WhoopProtocol, 537 pass) covers the layout rule including a lockstep check that no version outside the mapped set decodes anything; StrandTests (1054 pass) covers zero-payload ranking, the strict all-zero test, and the summary. Both app targets build. Android twin not updated: evictLines/hasZeroPayload and the widened filter need the same change in com.noop before the platforms agree.
The 5/MG emits its own computed SpO2 percentage on @82 of the v18 record. It has been captured since v31 — but only inside v18AuxSample.fields, a table capped at 604,800 rows/device (~7 days at 1 Hz) because the aux blob costs ~30 MB/day. So every reading rolled off after a week and was gone for good: the strap trims its own history the moment an offload is acked, so there was no second copy anywhere. Raising the aux cap is the wrong fix — it buys a year of SpO2 at the price of ~11 GB of unrelated aux bytes. The in-band signal is ~2,000 samples/week (~100k rows/year), so it gets a narrow sibling table that is never pruned while the wide aux capture stays capped and unchanged. v34-spo2-pct-durable: spo2PctSample(deviceId, ts, pct), PK (deviceId, ts). Numbered v34 rather than v32 because upstream already holds v32-rr-src-channel and v33-workout-steps, which this branch has not merged. A colliding identifier is not a merge conflict anyone sees — GRDB keys migrations by name, so it would silently skip the second body and one table would simply never be created. Declared in schema_oracle.json under a new fork-ahead-of-upstream lineage with that reasoning written down. Storage is lossless: the raw in-band byte, verbatim. Run grouping and ramp trimming are read-time policies — they have already changed once on the cloud reader, and baking one in would destroy the evidence needed to change it again. The one thing applied on the way in is the 70...100 demultiplex, which is not a policy: @82 also carries bit-7 status sentinels, sub-70 diagnostic codes and 0-for-not-emitted on the same byte. The write reads s.auxByte82 off the decoded struct, never fields[23] of the packed blob. Byte 23 is correct only because every slot ahead of @82 happens to be present in production; V18AuxCodec omits absent slots entirely, so one missing earlier slot slides the read onto opticalAmpB — which still lands in 70...100 and so fails silently as a plausible percentage. Both halves pinned in V18AuxCodecOffsetTests. Nightly aggregate now fills dailyMetric.spo2Pct, which was unconditionally nil for WHOOP while the same field rendered fine for imported Oura rows. The statistic is a per-run ramp-trimmed median: the strap samples in runs of ~30 whose first ~5 are the optical front end settling and read low, biasing a naive mean down (1.5 points on the pinned case). Median over trimmed survivors matches what the cloud reader already reports for the same rows. Note this field also feeds HealthKit's oxygenSaturation, so it stays nil whenever the night has no in-band reading rather than carrying a guess. Retention/registry wired consistently: excluded from the aux sweep by construction, added to deviceScopedTables (a never-pruned table makes delete-means-gone more important), storage stats, and the timestamp heal. Reaches the cloud with no extra wiring — .noopbak archives the whole .sqlite and liters replicates at page level; neither enumerates tables. No Room twin yet; recorded as ios_only in the oracle. Tests: 26 new (14 store/codec, 12 analytics). Package suites green (WhoopStore 409, StrandAnalytics 1324, WhoopProtocol, StrandImport, NoopLocalAccess). Both app targets build; StrandTests 1049 pass.
Populating dailyMetric.spo2Pct for WHOOP nights has a consequence that is easy to miss: HealthKitBridge writes that field back as .oxygenSaturation. This export loop reads only the computed/imported NOOP device rows — Oura never passes through it — so while spo2Pct was nil the branch had never fired. Filling the field does not resume an export, it starts one. What it would start writing is a number nobody has checked against a reference oximeter. The bar is higher than for the in-app card: a HealthKit sample lands in the system health record under Apple's Blood Oxygen heading, sits beside clinically-sourced readings with nothing distinguishing it, syncs to iCloud, and is readable by every other authorised app. Deleting it later does not recall what already propagated. The evidence supports the decode — distribution, run structure, sleep gating, night-to-night stability, agreement with this user's own Oura-era medians — but all of it establishes that the byte is a real oxygen measurement, not that 97 means 97. Upstream declines to promote the same metric for the same reason: two straps checked against the WHOOP app moved in opposite directions. Gated behind one named constant with the reasoning at the declaration, so enabling it after paired-oximeter nights is a one-line change and a deliberate one. Everything else the agent built — the durable table, the decode, the nightly aggregate, the in-app card — is untouched and live.
Two features landed on Swift with their Kotlin twins explicitly outstanding. Both are now written, and the platforms agree again. DURABLE SpO2 — Room 25 -> 26, MIGRATION_25_26, twin of GRDB v34-spo2-pct-durable. The 5/MG emits its own computed SpO2 % on @82 of the v18 record. Android has banked that byte since MIGRATION_24_25 — inside v18AuxSample.fields, capped at 604,800 rows/device (~7 days at 1 Hz) because the aux blob costs ~30 MB/day. So every reading rolled off after a week and was gone: the strap trims its history the moment an offload is acked. spo2PctSample(deviceId, ts, pct), PK (deviceId, ts), is the narrow never-pruned sibling; the wide aux capture stays capped and unchanged. Room's version and GRDB's identifier are independent counters that drifted apart with the fork's iOS-only lineage, so 25 -> 26 pairs with v34 rather than v32/v33. Room's is a dense integer sequence and takes the next free value; GRDB's is a string numbered ahead of upstream on purpose. schema_oracle.json moves spo2PctSample from ios_only to both, bumps roomVersion to 26, and drops the "no Room twin yet" reason rather than leaving it to rot. Both byte-identical copies updated; both halves of the oracle suite pass. The write reads V18AuxRow.auxByte82 off the decoded row, never fields[23] of the packed blob. Byte 23 is correct only because every slot ahead of @82 happens to be present in production; V18AuxCodec omits absent slots entirely, so one missing earlier slot slides the read onto opticalAmpB — which still lands in 70..100 and so fails silently as a plausible percentage. Both halves pinned in V18AuxCodecOffsetTest, the twin of the Swift file. Storage is lossless: the raw in-band byte, verbatim. The only thing applied on the way in is the SPO2_CANDIDATE_IN_BAND (70..100) demultiplex, which is not a policy — @82 also carries bit-7 sentinels, sub-70 diagnostic codes and 0. That range is now a named constant the store fork, nightlySpo2Pct and the existing nightlySpo2CandidateMean all read, instead of three matching literals. Nightly aggregate fills DailyMetric.spo2Pct, which was unconditionally null: a per-run ramp-trimmed median, runs split on gaps > 60 s, dropping the leading 5 of each run but never more than half a run. Same fixtures and expected values as Spo2PctNightlyTests. Retention/registry wired consistently: excluded from the aux sweep by construction (there is deliberately no prune query), added to the deviceScoped delete + adopt-serial re-key, and to the timestamp heal — a table that is never pruned on age would otherwise carry a bad-clock row forever. HealthConnectWriter.EXPORTS_SPO2_TO_HEALTH_CONNECT = false, the twin of Swift's exportsSpo2ToHealthKit. Filling spo2Pct does not resume that export, it starts one: the loop reads only computed rows, so while the field was null the branch had never fired. What it would start writing is a number nobody has checked against a reference oximeter, landing in the system health record beside clinically-sourced readings. RAW HISTORY ARCHIVE — twin of the Swift hardening. The archive filter asked the wrong question for an unmapped layout: it kept a type-47 record only when the DECODE produced nothing. On Android that gives the right answer today only because decodeWhoop5Historical returns null for every version but 18 — an accident of the decoder's current shape. The decision is now structural: MAPPED_WHOOP5_HISTORICAL_VERSIONS is the single source of truth, and any 5/MG record outside it is archived whatever it decoded. That set is {18, 26} here, NOT Swift's {18, 20, 21, 26}. Swift dispatches v20 (raw optical) and v21 (raw IMU) through decodeWhoop5HistoricalV2021; the Kotlin historical path has no such branch — Whoop5RawOptical/Whoop5RawImu exist but are wired to the live deep-buffer route. Claiming 20/21 would assert a field map this platform does not have, which is the exact class of bug the constant closes. The divergence is written down at the declaration and asserted in the test. Retention is now weakest-value-first: malformed lines, then surplus lines whose payload region is ENTIRELY zero, then surplus informative lines. The floor splits with it — 64 informative per version, 8 all-zero. The test is strictly all-zero: one non-zero byte anywhere makes a frame informative, because a small populated block inside an empty buffer is exactly what the archive is for. payloadStart is 21 on 5/MG and 17 on WHOOP 4, deliberately past the header, or a region carrying the non-zero seq/ts bytes would never test as empty. An over-cap rewrite evicts to 7/8 of the cap so a 1 Hz stream no longer rewrites the whole file per record. evictLines takes zeroFloor defaulted, matching Swift, so no call site churns. Tests: android/./gradlew testFullDebugUnitTest — 3483 pass, 0 fail, 5 skipped (baseline was green at 3441). 32 new: 14 nightly, 10 store/migration, 2 codec offset, 6 layout rule; 5 added to the eviction suite. swift test --filter SchemaOracleTests: 7 pass, confirming the shared fixture still satisfies both halves. Three fake DAOs and DeviceRegistryTest's expected-table set updated for the new device-scoped table. Not twinned, with reasons: the Swift LocalAccessCore storage-stats entry (no NoopLocalAccess on Android and no table-enumerating storage readout), and the Devices -> MG ECG "Check archived raw records" operator screen (no MG ECG screen exists on Android; the ECG probe surface is iOS-only).
`Whoop5Ecg.ControlSignal` carries stop/start/restart = 0/1/2, and only 0 and 1 have ever been put on a wire by any build in this tree. That leaves `restart` as the one untried argument inside the opcode family recorded as non-destructive (123/124/125/139), and therefore the cheapest remaining experiment against #891's open question. `BLEManager.ecgRestartCapture()` sends the SAME preamble as `ecgStartCapture()` — toggleRealtimeFilteredECG(on), toggleSaveRawECG(on) — and changes exactly one byte, the control value. What the strap does with `2` is unknown: the name is read off the client enum's order, like the wrist mapping, and no capture attests it, so there is no basis for claiming it re-arms the channel toggles itself nor for claiming it needs them already on. Issuing 124 alone would change two things at once and leave a null result unattributable to either. `ecgStartCapture()` / `ecgStopCapture()` are untouched; the ordinary stop sequence reverses this, because the strap state it leaves behind is the state a normal start leaves behind. Gating is unchanged and unwidened: the `noopWhoop5Ecg` Experimental opt-in, `ecgGatesAllow()` (positively attested MG, connected), the send() allowlist's double gate, and the same confirmation dialog every other ECG action sits behind. No new gate, no weaker gate. The probe report now annotates each command with the argument it was sent with (`TOGGLE_LABRADOR_DATA_GENERATION(124) arg=2`). Without it a start run and a restart run render character-for-character identically, and the report is the artefact that gets copied into an issue long after the strap log is gone. It is display-only, never feeds a verdict, and stays `nil` for an UNSOLICITED reply whose argument nothing here knows. Mirrored in Kotlin with the twin tests, per the parity contract. The Android client has no ECG app layer, so there is no Kotlin twin of the restart send path — only of the probe formatting it produces. i18n: the new button label is left out of the string catalog, as the branch's existing ECG strings are. Appending it untranslated would turn one failing focus-locale check into five by breaking the four non-focus locales that currently pass, and this copy sits beside medical-safety framing that must not be machine-translated. Tracked as debt.
The live ECG probe reported "no frame passed the structural triage" on every
run. `Whoop5Ecg.plausibleFilteredPayload` ended in a length agreement that
hardcoded TWO bytes per sample:
let end = headerLength + n * 2
A populated RAW flash record read off hardware carries numberOfECGSamples =
500 against a 1500-byte sample blob — THREE bytes per sample. Nothing attests
that the filtered stream matches the raw one, and nothing rules it out; what
the hardcoded 2 did was make the question unanswerable, because a 3-byte frame
misses the agreement, blows the pad4 budget, and is dropped before anyone can
look at it. The raw side has enumerated widths from the start
(`rawBytesPerSampleCandidates`); the filtered triage never got the same
treatment.
Two changes:
1. `filteredBytesPerSampleCandidates` replaces the fixed width with the
enumerated set [2, 3, 4] (2 first, so the old behaviour is a strict subset)
and RETURNS which widths agreed, so a width-3 candidate is not reported as
if it were a width-2 one. `plausibleFilteredPayload` is now "any width
agreed". Nothing else is loosened: the four Bool-typed bytes, the two
classifier enums, the signal-quality range and n > 0 all still reject. The
offset math is overflow-checked, as decodeRaw's is.
2. `Whoop5EcgProbe.FrameCensus` records EVERY unclassified 5/MG frame while a
probe is armed — the triage's misses as well as its hits. Logging only the
hits made the hunt for the (unattested) ECG type byte unfalsifiable: a
rejected frame left no trace, the report truthfully said nothing passed, and
the bytes that would have shown the mistake were gone. That is exactly how
the 2-byte assumption stayed invisible. Per frame it keeps the type byte,
frame and payload lengths, numberOfECGSamples when the header parses, the
widths that agreed, and the first 64 bytes as hex; bounded at 3 recorded
frames per type byte and 16 distinct type bytes, with everything past a cap
COUNTED rather than dropped. It renders directly under the triage result in
the probe report, where the operator can copy it out of the same sheet.
The candidate line now reads the status header directly instead of going
through decodeFilteredFrame, which decodes samples and so still assumes 2 bytes
each — a width-3 candidate would otherwise lose its line to a decoder that
cannot read its blob.
No BLE command, opcode, argument or send sequence is touched:
ecgStartCapture / ecgRestartCapture / ecgStopCapture are byte-for-byte
unchanged, as are the Experimental opt-in, ecgGatesAllow() and the MG-family
check.
Kotlin twins mirrored (Whoop5Ecg.kt, Whoop5EcgProbe.kt) per the parity
contract. Android has no ECG app layer, so only the protocol-level halves
exist there, covered by their own tests.
Verified: swift test (555 pass), xcodebuild Strand macOS test (1054 pass),
xcodebuild NOOPiOS build, gradlew testFullDebugUnitTest (3501 pass).
Two complete 1584-byte type-47 layout-16 records captured off an MG on
2026-08-06 with the clasp electrodes held, embedded as hex and run through
the existing public decode API. No production behaviour changes.
What the real bytes establish:
* rawBytesPerSampleCandidates resolves unambiguously to [3], and the
RawLabradorPacket shape accounts for every byte of the record body
with no remainder -- but only when the payload is taken from frame
offset 17.
* headerLength = 17 is four bytes too long for this record. The header's
last nine bytes (heartKeyProgress through numberOfECGSamples) land
correctly and carry real values; the first eight read the record's u32
unix second and two unmapped bytes instead of signal quality, the four
booleans and the two classifier enums. The record's real signal-quality
byte sits at frame[21], which the header maps onto
heartKeyIsStoppedAndComplete.
* The leads-off block is fixed-size -- eleven slots each for I and Q --
not packed to numberOfLeadsOffSamples. When the count is 10 the Q array
is read two bytes early, yielding a spurious leading zero and losing the
last real value, and the five-byte remainder exceeds defaultMaxPadding,
so 226 of the 350 populated records in the capture (65%) are discarded
before any field is read.
* The sample blob is 18 signed bits big-endian inside a 24-bit field whose
top six bits are a tag. Proven by continuity rather than assumed: on the
first record the electrode offset leaves the 16-bit signed range, so a
trailing-16-bit reading wraps 65428 counts between adjacent 500 Hz
samples while the 18-bit reading stays within 3556.
No sample SCALE is asserted -- the package states the unit and scale are
not attested, and nothing in the capture establishes one.
The raw flash record's leads-off diagnostic block is FIXED-SIZE — the count byte at frame[1534] is followed by eleven i16 I slots at frame[1535] and eleven i16 Q slots at frame[1557], present in full whether or not numberOfLeadsOffSamples fills them, with the unused tail slots zeroed. Both counts in the 2026-08-06 MG capture (10 and 11) put the Q array at the same offset and leave the same single trailing byte, which is what identifies the block as fixed rather than packed. decodeRaw treated the two arrays as packed to the count. Whenever the count was below capacity that was wrong twice over: Q started two bytes early per unused slot, so it gained a spurious leading value and pushed its last real value out into the "padding"; and the misplaced end-of-record left a 5-byte remainder against defaultMaxPadding = 3, so rawBytesPerSampleCandidates returned [] and the record was discarded before any field was read. On the capture's populated records (type 47, layout 16, numberOfECGSamples > 0) that took the decode-success rate from 124/351 to 350/351. The remainder is charged correctly by the fixed layout, not by widening the tolerance: a real record still leaves exactly one padding byte, and 17 + 1500 + 1 + 22 + 22 + 1 = 1563 accounts for the whole body. A count above the block capacity now fails closed rather than reading past the block. Sample decode, commands and gates are untouched. No scale or unit is claimed for the sample values and none is applied; the 18-in-24 big-endian packing is recorded as measured from this capture, not as an attested vendor fact. The synthetic fixtures on both platforms encoded the packed assumption and are rebuilt against the real layout via a shared leadsOffBlock() helper. The Kotlin twin carries the same two real frames so the parity contract is proved against hardware bytes rather than against a shared assumption. Still open, deliberately not papered over: the capture's short final record (numberOfECGSamples = 245) keeps its leads-off block at the same fixed frame[1534], with the sample region zero-filled from frame[769] — so the sample REGION is fixed-size too, and the blob-length-implies-width premise rawBytesPerSampleCandidates rests on does not hold for a partly filled record. It is the one populated record that still does not decode. Verified: Packages/WhoopProtocol swift test 570/570 pass; android testFullDebugUnitTest 3507/3507 pass (JDK 17).
The capture's one partly-filled raw record (numberOfECGSamples == 245) was the last populated record `Whoop5Ecg` still discarded. It is the same 1584-byte type-47 layout-16 frame as every full 500-sample record: its sample data stops after 245 * 3 bytes, frame[769..1534) is zeroed, and its leads-off block sits at the SAME fixed frame offset 1534 as both full records'. So the SAMPLE region is a fixed-size container too — `numberOfECGSamples` says how many of its slots are valid, exactly the way `numberOfLeadsOffSamples` does one field over. `decodeRaw` therefore locates the leads-off block from the END of the payload rather than from `headerLength + n * bytesPerSample`, and carries only the valid `n * width` bytes as samples. Under the old reading the count byte landed on a zero at payload[752], which yielded an empty leads-off block and 766 trailing bytes against `defaultMaxPadding = 3` — rejected before any field was read. No region length is hardcoded: the block's own fixed size anchors it, candidate ends are tried closest-to-the-end first, and the span between the declared samples and the block must be all zero. That zero-fill check is what keeps the width enumeration honest — a width that under-reads leaves real sample bytes in what it calls unused capacity and is rejected there. Full records are unaffected: their region is exactly full, so every existing decode is byte-for-byte the same. `RawLabradorPacket` gains `unusedSampleBytes` (counted, not carried) so the record still accounts for every byte, plus `sampleRegionBytes`. STILL OPEN, and reported rather than papered over: a partly-filled record cannot determine its own sample WIDTH. 245 samples inside a 1500-byte region fits 3 bytes per sample and 4 alike, so `rawBytesPerSampleCandidates` returns [3, 4] for it and the auto-width overload declines. The width is a property of the stream — the 350 full records resolve to [3] unaided, and all 351 populated records decode at that explicit width. No scale or unit is claimed for the sample values, and none is applied. No encoding claim is made for the new record's samples: it is the first half second of a reading and the input is still settling, so its tag bytes are not the uniform 0b100000 the full records carry (pinned as a census, not an interpretation). Verification: the n=245 record is embedded verbatim in both hardware test files, byte-identical across platforms. `swift test` 576 passed; `./gradlew testFullDebugUnitTest` 3511 passed. Kotlin twin mirrored per the parity contract. No hardware retest needed — this is pure decode over bytes already captured.
…idge brief wakes The gravity-stillness spine reads a restless but genuinely asleep night as "active", so whole hours of physiologically unambiguous sleep (sustained sub-baseline HR) never become candidate runs: a real 5/MG user's 6-8 h nights collapsed to 1-3 h quiescent morning fragments (hourly HR 47-58 from ~01:00 to ~09:00 local scored nothing), and an ~8-min bathroom get-up — smeared past mergeMin by the centered 15-min rolling stillness window — split the remaining night into separate stored sessions at a 16-min seam. The dense path has no HR-led recovery at all (the #308 rescue is gated on gravity SPARSENESS, which a dense restless night never trips), and detection emits one stored session per still-run with no session-level re-merge (the #561/#861 bridges only group later, at selection/aggregation). New experimental toggle "HR-first sleep detection" (default OFF, PuffinExperiment.hrFirstSleep on both platforms), applied between the run spine and the gate ladder in SleepStager.detectSleep: 1. hrRescueActiveRuns — an "active" run whose MEDIAN in-run HR sits inside the SAME sleep band the ladder already trusts (confirmSleepWithHR's baseline x 1.05, same median statistic, same sleepHRBaseline override) is reclassified sleep: motion alone must not veto sleep the cardiac signal confirms. 2. bridgeWakeGaps — adjacent sleep runs separated by <= 45 min (wakeBridgeMaxMin, below the selector-side gapBridgeMaxMin of 60) are assembled into ONE candidate run, so a brief get-up yields one session with the wake staged inside it, and sub-minSleepMin fragments between arousals are no longer dropped outright. Every downstream gate (minSleepMin, H4 span cap, hrConfirm, off-wrist, daytime/morning guards) still applies to the assembled runs, so a rescued daytime stretch still needs a real cardiac dip. Default OFF keeps every existing caller and golden test byte-identical (locked by an explicit byte-identity test); the flag is folded into both platforms' detectSleep memo keys. Trace gates hrRescue/wakeBridge emit under the Sleep test mode. Cross-platform parity: identical constants, functions, and threading in Swift (SleepStager, AnalyticsEngine.analyzeDay, IntelligenceEngine, PuffinExperiment, SettingsView) and Kotlin (SleepStager, AnalyticsEngine, IntelligenceEngine, AppViewModel, WhoopBleClient, PuffinExperiment, SettingsScreen). Tests run without a strap and follow the varying-input rule (#194/#345): the rescue must TRACK injected sleep windows at different positions/lengths and refuse the identical motion pattern with elevated HR (SleepStagerHrFirstTests.swift / SleepStagerHrFirstTest.kt, same scenarios, same numbers). Verified: swift test (StrandAnalytics, 1342 passing incl. 9 new), macOS Strand build, iOS NOOPiOS build, Android compileFullDebugKotlin + testFullDebugUnitTest (all passing, 9 new). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… bridge, span-guarded A 5-night replay of the reporting wearer's raw mirror streams (Aug 2026) through detectSleep falsified the HR-rescue half of the previous commit: on a low-HR wearer the quiet awake EVENING median (~58 bpm) also sits inside the x1.05 sleep band off the ~64 window baseline, so whole evenings were rescued and chained through the wake bridge - the assembled run then either exceeded the 16 h H4 cap and dropped the night to ZERO (worse than baseline), or squeaked under it and scored a 15.4 h "night" starting at 18:00. The strap's own banked band sleep_state cannot arbitrate: it is motion-led too, and read "wake" across hours of HR-proven sleep. Median HR LEVEL cannot separate that wearer's quiet wakefulness from sleep; only dip DEPTH (sleep-only minima 42-48 bpm) looked separable, and deriving an onset boundary from dips needs a properly validated design (#194/#345). The restless-night DROPPING failure stays OPEN; a history note above `wakeBridgeMaxMin` records the evidence so it is not re-attempted naively. What remains is the safe half, renamed "Bridge brief wakes" (`noopSleepWakeBridge`, default OFF): `bridgeWakeGaps` assembles adjacent sleep runs <= 45 min apart into one candidate run before the gate ladder, now REFUSING any merge whose assembled span would exceed maxMainSleepSpanS - without that guard a pathological chain would assemble and then be dropped whole by the H4 cap, converting a partially-scored night into a zero-score night (the exact bimodal failure the replay exposed). The tests are rewritten around the surviving feature and pick up a REAL-DATA pin: the exact stored fragment layouts of five of the wearer's split nights, asserting which seams the bridge merges (the 16/12/28/25/ 19-min arousal seams) and which it leaves split (2.2-3.9 h genuine wake gaps; a 90-min morning gap left to the selector's #861 night-tail bridge). The bridge reproduces 7/7 of the sub-45-min seam-fills that an independent HR-evidence-led correction pass applied to the same nights on the wearer's mirror; the two disagreements (a kept-separate morning doze it folds in, and the 90-min restless gap it refuses to guess) are documented in the pin. A restless-night test now pins that the bridge deliberately does NOT reclassify motion-vetoed sleep. Cross-platform parity: identical rename (wakeBridge / useWakeBridge / PuffinExperiment.wakeBridgeEnabled / sleepWakeBridge, key noopSleepWakeBridge), identical span guard, identical tests (SleepStagerWakeBridgeTests.swift / SleepStagerWakeBridgeTest.kt). Verified: swift test (StrandAnalytics, 1332 passing incl. 8 rewritten), macOS Strand build, iOS NOOPiOS build, Android compileFullDebugKotlin + testFullDebugUnitTest (all passing incl. 8 rewritten). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…estigation The MG motion/gravity decode investigation (2026-08-24) closed with the decode verified CLEAN: the alarming server-side "postureVar" is a per-axis RANGE (max-min of the unit gravity vector, in g), not a variance, so the "RMS |g| ~1.5 g during sleep, physically implausible" inference that motivated a decode-bug suspicion was invalid. More importantly, it showed the rescue's premise was itself partly wrong: on the flagship "dropped" night the strap's own band sleep_state read wake across the disputed hours WITH corroborating 70-75 bpm stretches and continuous posture change - likely genuine in-bed wakefulness the stillness spine was RIGHT to reject (the wearer's awake-resting HR sits in the low 50s, RHR ~44, so "HR 47-58 = unambiguous sleep" does not hold for them). Comment-only: rewrite the history note above wakeBridgeMaxMin (both platforms) to drop the "band state was fooled / HR-proven sleep" framing and record the decode-clean finding, so the next attempt at the open restless-night problem starts from the corrected evidence. No code change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ryanbr
added a commit
that referenced
this pull request
Aug 28, 2026
…analyzeRecentOnCpu (#1696) analyzeRecentOnCpu had SEVENTEEN bytes of headroom: 61,518 against the 61,535-byte JaCoCo budget its own guard pins. Not a margin, a wall - and one that announces itself at test time rather than at review time, so the next person to touch that method for any reason hit it with no hint it had been full since before they arrived. analyzeRecentOnCpu instrumented Code length 61,518 -> 54,087 margin to the JVM's 65,535 method limit 4,017 -> 11,448 7,431 bytes freed by lifting the per-day skin-temp, SpO2 and off-wrist reads into a private suspend helper. IT IS A MOVE, AND THE DIFF PROVES IT. All 44 removed lines reappear verbatim in the additions; the extra lines are only the signature, the holder, the doc and the call site re-binding the same six names it used inline. Side-effect order is unchanged - the four per-owner memo maps are passed by reference and mutated in the same sequence, so WHEN the WHOOP 4.0 ADC anchor is learned and WHAT FROM are identical. The call sits inside the same prep timing bracket, so the analyzeRecent cost split still attributes these reads exactly as before. WHY EXTRACT RATHER THAN RAISE THE BUDGET. The guard exists because JaCoCo instrumentation already blew the JVM method limit once (bhelm#102), and this file already answered the question: persistFitnessVitalityAndSteps was extracted for exactly this reason, with a second test pinning that it stays extracted and at its required call site. Raising a budget to fit a change is how the method got to 17 bytes. WHY THIS BLOCK. Best ratio of bytes freed to boundary crossed - six values out against eleven in, all already local - and self-contained. The full 115-line read region was the blunt alternative: sixteen values crossing instead of six, for a refactor whose only job is to make room. THE BUDGET IS NOW A RATCHET, 61,535 -> 56,000. Leaving it would have left 7,448 bytes to be spent the same silent way, which is the mechanism this removes rather than merely resets. The ~1.9 K margin is deliberate: enough for an ordinary change (the pass-1 sliding read windows need about 825), small enough that a large regression fails at the test rather than at the 64 KB wall. 56,000 rather than the 55,000 first floated, because 55,000 would have left 88 bytes after the wiring and rebuilt the wall inside one PR. The comment states the discipline: lower it whenever an extraction frees space, never raise it to fit a change. persistFitnessVitalityAndSteps is left at 12,000 against 6,415 - slack, but not creeping, and tightening a method nobody is pressing against would be tightening for its own sake. MARKED DELIBERATELY ONE-SIDED. The Swift engine keeps this block inline, because the constraint is a JVM one and Swift has neither the limit nor the guard. The doc says do not mirror it, so a parity audit that finds a Kotlin helper with no Swift counterpart has found the intended state rather than a gap. REGRESSION SURFACE, CHECKED RATHER THAN ASSERTED. The three open PRs touching this file (#1605, #1587, #1572) conflict identically against main and against this branch - 44, 4 and 8 files, with IntelligenceEngine.kt already among #1605's against main. This adds no conflict. Nor can the ratchet block them: all three pass CI today, which is itself proof they add nothing to analyzeRecentOnCpu, since any addition would already fail at 17 bytes. They go from 17 bytes of room to 1,913. Android 4604 tests 0 failures, the same count as main. doc_comment_lint OK, i18n green. No Swift touched, so no app-build exposure.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The on-device sleep detector splits a real night into separate stored sessions around brief mid-night get-ups. Verified against a WHOOP MG wearer's data (Aug 2026): an ~8-min bathroom trip (HR 42→75, walk ticks, then back to 38–40 within minutes) cut the night at a 16-min seam (06:17 session end / 06:33 next session start); seven nights in a two-week span split into 2–4 sessions at 12–28 min arousals, and sub-1h sleep fragments between arousals were dropped entirely by
minSleepMin.Mechanism: the CENTERED 15-min rolling stillness window smears a brief get-up past
mergeMin(15 min) into its own "active" run; detection emits one stored session per still-run and nothing at the detection layer re-merges them — the existing #561/#861 bridges only group sessions later, at selection/aggregation, afterminSleepMinhas already discarded short fragments. Industry convention (and the selector's own <60-min "same sleep period" threshold) treats a brief wake as belonging INSIDE the night.Change
New experimental toggle "Bridge brief wakes" (
PuffinExperimentkeynoopSleepWakeBridge, default OFF, Settings → Sleep staging on both platforms). When ON,SleepStager.bridgeWakeGapsassembles adjacent sleep runs ≤ 45 min apart (wakeBridgeMaxMin, deliberately below the selector-sidegapBridgeMaxMinof 60) into ONE candidate run between the run spine and the gate ladder:minSleepMinfragments between arousals stop being dropped outright;maxMainSleepSpanS, so a pathological chain can never assemble and then be dropped whole by the H4 cap.Default OFF is byte-identical to current behavior (pinned by an explicit test); the flag is folded into both platforms'
detectSleepmemo keys.What was tried and withdrawn (documented in-code)
An HR-led rescue of motion-vetoed restless runs (median in-run HR inside the
confirmSleepWithHRband → reclassify as sleep) was built alongside the bridge and withdrawn after a 5-night replay of the wearer's raw streams: on a low-HR wearer, quiet awake HR (median ~58 bpm; awake-resting low-50s, RHR ~44) also sits inside the ×1.05 band off the ~64 window baseline, so whole awake evenings were rescued and chained — nights then either exceeded the 16-h cap and dropped to zero, or scored a 15-h "night" starting at 18:00.A follow-up decode investigation then verified the MG gravity decode clean (unit-norm throughout) and showed the rescue's premise was itself partly wrong: on the flagship "dropped" night, the strap's own band sleep-state read wake across the disputed hours with corroborating 70–75 bpm stretches and continuous posture change — likely genuine in-bed wakefulness that the stillness spine was RIGHT to reject. Median HR level cannot separate that wearer's quiet wakefulness from sleep; only dip depth (sleep-only minima 42–48 bpm) looked separable, and deriving an onset boundary from dips needs a properly validated design (#194/#345) plus per-night ground truth HR alone cannot supply. A history note above
SleepStager.wakeBridgeMaxMinrecords the evidence so this is not re-attempted naively. This PR deliberately ships only the bridge.Verification
SleepStagerWakeBridgeTests.swift/SleepStagerWakeBridgeTest.kt, asserting precisely which seams the bridge merges (16/12/28/25/19-min arousal seams) and which it leaves split (2.2–3.9 h genuine wakes; a 90-min morning gap left to the selector's fix(protocol): v18 @36 is a flag byte, @106/@107 two u8 channels #861 night-tail bridge). The bridge reproduces 7/7 of the sub-45-min seam-fills an independent HR-evidence-led correction pass applied to the same nights; the two disagreements are documented in the pin.swift test(StrandAnalytics): 1332 passing. AndroidcompileFullDebugKotlin+testFullDebugUnitTest: all passing. App targets compiled locally per the CI-gap rule: macOSStrandand iOSNOOPiOSboth build.Cross-platform parity: identical constants, functions, threading, and tests in Swift (
SleepStager,AnalyticsEngine.analyzeDay,IntelligenceEngine,PuffinExperiment,SettingsView) and Kotlin (SleepStager,AnalyticsEngine,IntelligenceEngine,AppViewModel,WhoopBleClient,PuffinExperiment,SettingsScreen).🤖 Generated with Claude Code