sleepypod/training — offline sleep-stage model training and an
OKF knowledge base. Available publicly at
sleepypod/training.
Watch training progress, compare held-out models with the pod rules and Watch labels, and reopen previous runs. See the sleepypod documentation for the workflow and how it relates to personal models on the pod.
These are unmodified captures of the running app with deterministic synthetic test data. The displayed scores illustrate the interface, not real-world accuracy.
You need a Mac (or Linux machine) on the same network as your pod, an Apple Watch that tracks sleep, and about ten minutes.
-
Install the tools. With Homebrew:
brew install node@24 git jq. (jqis only needed for the model-upload commands.) The quickstart offers to install uv, which brings its own Python. -
Get this repository:
git clone https://github.com/sleepypod/training.git && cd training. -
Find your pod's address. The sleepypod iOS app shows it under Settings.
-
Export your Watch sleep data (skip this if the iOS app already syncs Watch data to the pod): on the iPhone open Health, tap your profile picture, then Export All Health Data. AirDrop or save
export.zipto the Mac. Large exports take a few minutes. -
Run the guided script:
scripts/quickstart.sh
It checks Node.js and uv, installs the pinned dependencies, confirms it can reach the pod, asks which side the Watch wearer sleeps on and where
export.zipis, then trains. Pass--pod,--sideand--watch-exportto skip the questions, or--dry-runto check everything without training. -
Read the results in the dashboard link the script prints, or later with
uv run --locked python scripts/dashboard.py. Held-out macro F1 and κ for the candidate, the existing baseline and the pod's rules are the numbers to compare; see interpreting model metrics.
Training only reads from the pod. It needs at least five nights with Watch sleep stages, and more nights help most. To put a model on the pod, see Use a trained model on the pod.
Install Node.js 24 and uv, then from this repository:
npm ci
uv sync --lockedpackage-lock.json pins the TypeScript tooling and SQLite dependency. uv.lock
pins Python 3.11 numerical dependencies. Production features, rules, and portable
inference are vendored from a pinned core revision, with checksums;
no sibling core checkout or unpublished commit is needed to run training.
uv run --locked python scripts/train.py --pod 192.168.1.88 --side left --dashboard --verboseReplace the example IP with your pod. The fetcher uses unauthenticated, read-only
HTTP on port 3000; --pod hostname:port also works. It reads the pod timezone,
completed sessions from the last 30 days, vitals, movement, imported Watch labels,
and the current stage rules. Use --days 14 to change the window or --timezone
to explicitly override the pod setting. No MCP client or SSH is needed.
On macOS, Local Network privacy can refuse Python's connection to the pod (No route to host) although curl reaches it; pod reads then retry once through /usr/bin/curl,
which is exempt. Grant your terminal Local Network access in System Settings to avoid it.
If Watch labels are not already imported on the pod, add
--watch-export data/export.zip or --watch-json data/watch.json. Fetching does
not import labels, modify settings, or activate a model.
Terminal progress shows each phase, completed night folds, model fits, and elapsed
time. --verbose keeps every event on its own terminal line. All runs also save
events.jsonl with the complete structured history. The dashboard shows:
- Response-body bytes, average download throughput, API calls, and unique input rows.
- Labeled/unlabeled feature epochs, class balance, and per-feature min/mean/max/std.
- Per-fit input rows (before/after oversampling), fit time, rows/s, and actual tree/node counts.
- A fit-time chart and per-night held-out macro F1 for nested selection versus the baseline.
- A scrolling activity feed with inner-validation scores and current evaluation phase.
Throughput measures wall-clock work for this run, not sensor cadence. Feature
matrix bytes describe float64 storage, not process memory. Per-night scores are
not a loss/learning curve, and partial results are not a final quality estimate.
Dashboard history is bounded (120 events, 80 fit timings); events.jsonl retains
all events. Feature summaries are descriptive only and do not affect fitting.
--dashboard prints a loopback URL for a live browser view with progress,
model comparisons, selectable metrics, and confusion matrices. The sleep comparison
at the top aligns Pod / New model / Apple Watch stages by session. Select a night,
hover an interval, or use Previous/Next to inspect exact stages. Orange marks
Pod/model disagreements. Held-out predictions appear as each night finishes;
the final fitted-model view is explicitly in-sample. It stays open after
training until Ctrl-C. Without this flag the command exits when training finishes.
Both modes save index.html in the run directory; open it directly for an offline
interactive report. A failed or interrupted run retains its status and partial
files; retry into a new directory.
API reads are not an atomic database snapshot. The fetcher limits acquisition to
completed sessions, splits saturated API windows, checks side identity, detects
configuration/calibration changes, and retains response hashes in
acquisition.json. Concurrent historical backfills can still alter data between
reads. Use a consistent SQLite backup when atomic input consistency is required.
Timeline Pod stages come from GET /api/biometrics/sleep-stages for each record,
with record identity and stage-view consistency checked. The five-minute comparison
samples pod/Watch stages at epoch midpoints; it does not display their original
sub-epoch boundaries. Snapshot-only runs label the pod lane as a local rules replay.
Aggregate “Rules (replay)” metrics still use the pinned local rules engine.
The current rules are applied retrospectively; they do not reconstruct historical
per-sample rule versions. Raw responses and reports remain under ignored data/.
uv run --locked python scripts/dashboard.py --port 8765Open http://127.0.0.1:8765. This starts only the viewer; it never retrains.
The workspace discovers earlier run directories under this repository's data/
and its Git worktrees, and keeps a persistent SQLite catalog at the primary
checkout's ignored data/run-history.sqlite. New training runs register
there automatically. --out still requires a fresh directory; the default is
now the primary checkout's data/runs/, shared across worktrees.
Use --run data/runs/example to open a specific existing run, or add
--import-root /path/to/old/runs (repeatable) to discover runs stored elsewhere.
Importing registers paths; it does not move, overwrite, or fabricate artifacts.
Keep imported folders in place. --catalog data/another.sqlite selects a
separate catalog. The viewer serves only selected report artifacts and never
serves raw snapshots or arbitrary files. Historical runs with stale running
state and no known live process are labeled interrupted, without editing their
saved state. Runs with a report but no pipeline status are archived, not assumed
to have passed parity.
Actions (bottom of the sidebar) runs the repository's own scripts from the
workspace: train.py --snapshot, stage-backtest.py, fusion.py,
vitals_reference.py, movement_model.py train and epochs, archive_index.py,
search.py plan, run and report, and the archive pull that scripts/install-archive-agent.sh schedules (core's read-only
biometrics:archive pull, resolved by that script's --print-command from
data/workspace-settings.json). One job runs at a time; starting another while one
runs is refused (HTTP 409) rather than queued, so a job never starts on inputs that
changed after it was requested. Each job writes data/web-jobs/<id>/command.json
(the validated parameters, the exact argv and working directory, status and exit
code) and log.txt in the primary checkout's data/, is registered in the run
catalog, and adds its output to the run list when that output is a run. Parameters
are checked against fixed schemas: input and output paths must resolve inside
data/, outputs must be new, and nothing passes through a shell. No action uploads,
selects or activates a model, or writes to the pod. Stopping the workspace
interrupts a running job.
POST /api/jobs exists only in dashboard.py (not train.py --dashboard), listens
on 127.0.0.1, and requires the random token embedded in that launch's page as an
X-Training-Token header. Requests with another Host, a cross-site Origin or
Sec-Fetch-Site, or a non-JSON body are refused, and every page is served only to
the 127.0.0.1 and localhost host names, so a rebound DNS name cannot read the token.
Search in the sidebar lists model searches (scripts/search.py, kind search). A
search shows its pre-registered primary comparison with the night-bootstrap interval and
sign test, every candidate's pooled walk-forward scores on development nights, the
exploratory comparisons against the baseline, layer scores, exclusions and limitations,
and, with more than one search, a table comparing their primary results and winners. The
view reads only each run's report.json. A search's final reads the locked holdout once
and exports model/set.json. It is not a workspace action, and final.json, where its
scores live, is never read or served; the view shows only how many holdout reads there
have been.
A Sleep / Movement / Snore switch at the top of the sidebar picks the kind of
research run; the run list shows only that kind. Movement runs (scripts/movement.py train) and snore runs (scripts/piezo_bandwidth.py analyze and night) register
themselves when their output is under data/.
- Movement: windows, validation scheme and sessions; the pod movement score and the classifier side by side (moving recall, false alarms while still, balanced accuracy, and how often each movement type was flagged as moving); the classifier's confusion matrix and per-class recall; and a per-session timeline of labels, classifier predictions and pod-score flags. The pod score is one value per minute and cannot type movements; one-session validation is optimistic.
- Snore: a bandwidth session shows each band's rise over lying still against the 6 dB pass bar, the outcome, knock counts (which gate clock sync, not the outcome), pump speed and per-step band energy. A night shows 30 s band energy for every channel in both snore bands, the pump, coverage and storage, and, when a session from the same pod exists, its lying-still and imitated-snore levels as reference lines. This is vibration energy, not detected snoring.
Sleep runs are organized around a run sidebar, sticky pipeline header, and five tabs:
- Overview: held-out scores, review checks, four-lane stage comparison, small hypnograms for every night, sleep architecture, fold results, per-stage F1, confusion matrices, feature importance, resource measurements, and notes.
- Nights: Pod / candidate / baseline / Watch aligned on one axis, a night picker, candidate-vs-Watch or candidate-vs-Pod disagreement navigation, per-epoch inputs, exact stages, and matched stage-duration comparisons.
- Weeks: stage minutes per night grouped into ISO weeks by local night date, a source selector, and weekly averages per source with differences from Watch. Each night counts only intervals shared by every displayed source; nights without shared data are gaps, never zero.
- Data: intake, label coverage, per-night balance, feature histograms and descriptive distribution comparisons. Missing raw sensor readings cannot be inferred from finite feature values because some features use fallback values.
- Log: filterable activity, full event loading/download, captured failure details, configuration/provenance, and a candidate download. No activation.
Sidebar, compact-header and wide-screen inspector layouts are available. Every technical term has contextual help or a searchable Learn the terms entry, including macro F1, kappa, depth, leaves, cross-validation, and all input features. The same help, tabs and plots work in exported self-contained HTML reports. Offline reports contain one run; the running workspace supplies history.
New runs record baseline traces, fold details, environment versions, command, notes, per-fit CPU and peak trainer RSS, and three-repeat held-out permutation importance using the same sequential postprocessing as scoring. These descriptive measurements never affect candidate selection. Historical missing measurements are shown as unavailable, not reconstructed through new fits. Feature distributions can be derived from retained feature files without retraining.
Sleep architecture uses the intersection of epochs present in every displayed source, including Watch. All bars therefore share a denominator; gaps are not filled as wake. Hypnograms show stage sequences, not validated cycle boundaries. Final-fit views are explicitly in-sample. Bootstrap intervals concern the mean night-level improvement, not the pooled score difference. Review checks are not release gates: promotion thresholds have not been agreed.
Add --name 'Leaf experiment' --notes 'Hypothesis and changes' to train.py
to label a run. Failure tracebacks and subprocess output remain in ignored
error.txt / process.log. There is no fabricated resume or promote command;
correct a failure and retry into a new output directory.
Put local inputs in data/. Only data/.gitkeep is tracked; all other contents,
including subfolders and generated outputs, are ignored. Raw data and fitted
artifacts must not be force-added. Git ignore rules do not encrypt local files.
Required inputs:
- A consistent
biometrics.dbSQLite backup containing sleep records, vitals, movement, runtime configuration, and calibration tables. See snapshot instructions. - An Apple Health export ZIP,
export.xml, or directory containingexport.xml. The parser selects Watch sleep stages and always excludes SleepyPod writeback. - The Watch wearer's pod side and the pod's IANA timezone.
- At least five independent night groups. Each retained session must last 3–14 hours and have at least 24 labeled five-minute epochs. Training folds need enough class diversity to fit the classifiers. This workflow currently evaluates one participant; do not mix people into its night-only split scheme.
uv run --locked python scripts/train.py \
--snapshot data/biometrics.db \
--watch-export data/export.zip \
--side left --timezone America/Los_Angeles \
--out data/runs/experiment-001--contract podContext trains on the 33-column contract (the ten podNative
features plus context, movement-bout and arousal features from
scripts/feature-ablation.py). Pods need core with podContext support (core#907)
to accept and evaluate such a model; podNative remains the default.
The output directory must not already exist. Use --source 'Watch' to choose a
case-insensitive Health source-name substring. To reuse extracted labels, replace
--watch-export with --watch-json data/watch.json (schema in input data).
The command checks the vendor snapshot, parses labels, exports production runtime features, performs nested whole-night model selection, writes the candidate, and checks all exported predictions against the TypeScript evaluator. It never uploads or activates the model. It produces:
watch.jsonwhen parsing an export or fetching pod labels, plusfeatures.json.progress.json, the fullevents.jsonllog, and an offline interactiveindex.htmlreport. Pod acquisition also createsbiometrics.db,responses/, andacquisition.json.model/report.jsonandmodel/protocol.jsonwith evaluation and provenance.- Private
model/timelines.jsonwith aligned Watch, captured pod, held-out, and final-model predictions. Pod acquisition also savespod-stages.json. model/candidate.json(pod-compatible model),model/candidate.joblib(Python fit), andmodel/parity-inputs.json(private verification rows).
Do not use these scores as prospective validation. Features use whole-night and centered-window context, and this is a retrospective classifier. See the validation protocol and latest historical findings.
uv run --locked python scripts/parse-watch-export.py data/export.zip data/watch.json
npm run features -- data/biometrics.db data/watch.json data/features.json left America/Los_Angeles
uv run --locked python scripts/retrain-models.py data/features.json --out data/runs/manual-model
npm run parity -- data/runs/manual-modelLiterature-derived feature ablation (context, movement bouts, arousal coupling, and a wake-first hierarchical forest) on an existing run's features and the snapshot they came from. It compares arms only and never exports a model:
uv run --locked python scripts/feature-ablation.py data/runs/RUN/features.json \
--snapshot data/biometrics.db --out data/runs/ablationChronological backtest of the same feature sets: refit before each night on earlier nights only (walk-forward) and fit once on the first nights (frozen), scored against the stages the pod recorded live:
uv run --locked python scripts/stage-backtest.py data/runs/RUN/features.json \
--snapshot data/biometrics.db --out data/runs/backtestFrom the night of 2026-11-14, every fourth night group (11-14, 11-18, 11-22, …) is
locked (rule, scripts/holdout.py).
train.py, retrain-models.py, stage-backtest.py, feature-ablation.py,
fusion.py and sequence-decoding.py drop locked nights before any fitting,
selection or feature analysis, and record the rule and the dropped groups under
holdout in their protocol. The dashboard shows "Locked holdout" in place of
scores for those nights.
--holdout-final is the only way a development run reads them, and only train.py
and retrain-models.py accept it. Training and selection still use development
nights only. After export, the run appends the candidate's SHA-256 and the locked
nights to data/holdout-access.jsonl in the primary checkout, then scores the
candidate on them once (holdoutFinal in model/report.json). Every read spends
the holdout, so make one per finished candidate. The other scripts fit, tune or
compare on the nights they score, so they refuse the flag.
uv run --locked python scripts/retrain-models.py data/runs/RUN/features.json \
--out data/runs/RUN-final --holdout-finalstage-backtest.py --trial scores two frozen pod models against the Watch on the same
new nights (night groups after 2026-11-13, plan E1): the stages the pod shows
(pod_live, read with GET /api/biometrics/sleep-stages; the pod restages on read with
its selected model, library id 3) and library id 2 staged by the read-only
POST /api/sleep-stages/model/preview. The library API lists models without their
JSON, so id 2 comes from a local copy, checked against the library's version. Nights
come from an archive snapshot (--snapshot, Watch stages from reference_stages)
or a run's features.json (--run), aligned to the exporter's Watch 5-minute epochs;
both models are scored on the same labelled epochs.
uv run --locked python scripts/stage-backtest.py --trial --pod 192.168.1.88 --side left \
--snapshot data/archive/<podId>/biometrics.db \
--baseline-model data/runs/<id-2 run>/model/candidate.json \
--live-model data/runs/<id-3 run>/model/candidate.jsonEach run appends one line per (night, model) to data/runs/trial-id3/weekly.jsonl
with the model id, version, SHA-256 (core's stageModelSha256: canonical JSON with
exact decimal numbers, so a pod copy and a local file agree), training cutoff, outOfSample, locked, metrics and
scoredAt. Re-scoring a night replaces its lines. A night counts as out of sample
only after a model's training cutoff, taken from --trained-through ID=YYYY-MM-DD,
the model's metrics.trainedThrough, or the last night group of the run the local
copy came from. Locked holdout nights are included, since both models are frozen,
and each run logs them for both models in data/holdout-access.jsonl (trial)
before reading. The summary gives per-night macro F1 and κ differences id 3 − id 2,
a night-level bootstrap 95% interval (10,000 resamples, seed 42) and a sign-test
count, over every night in the file that is out of sample for both. Pod reads are
GETs and the preview only.
Watch signals teach the pod models; they are never stage-model inputs. Watch stages are the stage labels, Watch motion labels the movement model, and Watch heart rate, HRV and respiratory rate check the pod's own vitals. Every model input comes from the pod, so a model never needs a Watch at night.
# Watch vitals, and how far pod vitals are from them epoch by epoch
uv run --locked python scripts/parse-watch-export.py data/export.zip data/watch-vitals.json --vitals
uv run --locked python scripts/vitals_reference.py data/runs/RUN/features.json \
--snapshot data/biometrics.db --watch-vitals data/watch-vitals.json --out data/runs/vitals
# Pod movement model, labeled by Watch motion (windows.json from movement.py train --watch-motion)
uv run --locked python scripts/movement_model.py train --windows data/runs/movement-001/windows.json --out data/runs/movement-model
uv run --locked python scripts/movement_model.py epochs data/runs/RUN/features.json --snapshot data/biometrics.db \
--model data/runs/movement-model/model.joblib --export SESSION_ID=data/raw/session.tar.gz --out data/runs/movement-epochs.json
# Portable forest for the pod from an existing model.joblib (train writes model/movement.json itself), checked in TypeScript
uv run --locked python scripts/movement_model.py export data/runs/movement-model/model.joblib --out data/runs/movement-model/model/movement.json
npm run movement-parity -- data/runs/movement-model/model/movement.json
# Stage model with pod features, HR quality and movement-model features
uv run --locked python scripts/fusion.py data/runs/RUN/features.json --snapshot data/biometrics.db \
--movement-epochs data/runs/movement-epochs.json --pod-model-trained-through 2026-11-03 --out data/runs/fusionvitals_reference.py reports pod-versus-Watch bias, error and correlation for heart
rate, breathing rate and HRV (rank only: the pod and Watch compute different HRV
statistics), how each signal moves with Watch stage, and which pod-only quality
signals (hrCoverage, hrJitter, brCoverage) predict heart-rate error.
movement_model.py train holds out one Watch-motion night at a time and compares the
pod movement score, the movement.py classifier and the new model on 2 s windows and
5-minute epochs. The model adds the pod score to the raw piezo and capacitance
features, and its moving threshold is chosen on out-of-bag training predictions.
epochs needs one raw export per sleep session (GET /api/export/archive, read-only;
the pod prunes old archives), or --archive data/archive/<podId> to read every session's
raw from the off-pod archive; an --export still wins for its session. Its features are cross-fitted: each session is scored by a
model fitted without every Watch-motion night whose windows fall inside it, and once more
per other Watch-motion night T by a model also fitted without T. model.joblib keeps the
teaching windows so these models can be refitted; one trained by an earlier version is refused.
Watch motion on locked holdout nights never teaches the movement model; epochs and fusion.py
refuse a model that was taught on one.
Moving windows are those strictly above the strict threshold, so tied probabilities at the
cutoff never count as moving. Raw-piezo features need a core port before a pod can compute them.
train also writes model/movement.json, the final model as a portable forest
(movement-model-v1, contract movementRaw-v1) with up to 64 test vectors and a SHA-256 of
its canonical body. Its trees use the stage-model arrays plus float32 inputs and the routing of a
missing pod score, so it reproduces scikit-learn exactly. The file is refused unless it matches
predict_proba within 1e-9 on every training window, with and without the pod score. export
writes it from an existing model.joblib; one saved before teaching windows needs the --windows
it was trained on. The movementRaw-v1 contract defines every
input for the core port. A 200-tree model is about 9 MB of JSON. Nothing is uploaded.
fusion.py evaluates the frozen ablation forest on the nights that have raw exports:
base (podNative), all (the podContext columns), all plus HR quality, plus
movement-model features, both (fused), and a wake-first hierarchical_fused. It
scores every arm leave-one-night-out and chronologically, against the stages the pod
recorded live. When night T is held out, every night's movement features come from models
that never saw T's Watch motion. Chronological scores use only labelled epochs that also have
a live pod stage, for every arm, so pod and model share one population (podCoverage per
night). The input must use the podNative contract, since the arms add the podContext columns. --pod-model-trained-through reports nights after the pod model's
training separately, because earlier nights are in-sample for it. Nothing is exported.
The pod prunes about one night of raw piezo a day. Core's archiver
(pnpm biometrics:archive pull, in a core checkout) keeps everything it exports in
data/archive/<podId>/: a merged biometrics.db, content-addressed raw/*.RAW.gz and
one pulls/*.json record per window. The pull is read-only on the pod. Install a daily
12:30 pull as a launchd agent; set coreCheckout in data/workspace-settings.json (else
../sleepypod-core next to this checkout is used) and check the plist first:
scripts/install-archive-agent.sh --dry-run
scripts/install-archive-agent.sh # logs to data/archive/pull.log
scripts/install-archive-agent.sh --uninstallscripts/install-archive-agent.sh --print-command prints the same pull (working directory,
PATH, argv) without installing anything; the training workspace's archive pull runs it.
In the training workspace (scripts/dashboard.py), Archive in the sidebar shows each
data/archive/<podId>/index.json: archived nights, Watch coverage per teacher, vitals
lineage, locked holdout nights (listed, never scored), nights at risk, and the age of the
last successful pull (the newest pulls/*.json, which core writes only when a pull
completes). It turns red when that pull is more than 3 days old, which is how a stopped
nightly agent gets noticed before raw is pruned. Pull now and Rebuild index start
the pull and archive_index.py as workspace jobs. While the workspace is open it also
pulls when it starts and every 6 hours, as a job; it skips a round when core's
.biometrics-archive.lock is held by a live pull, waits while another job runs, and never
makes the first pull, which waits for the other sleeper's consent. --no-sync turns this
off. It only protects nights while the workspace is open; the launchd agent is what keeps
raw from being pruned unarchived.
Then index the archive by night. The index reads local files only:
uv run --locked python scripts/archive_index.py --timezone America/Los_Angeles \
--local-exports-dir data/raw --storage data/storage.json
uv run --locked python scripts/lineage.py data/archive/<podId> --timezone America/Los_Angelesarchive_index.py writes data/archive/<podId>/index.json, one row per sleep record per
side: in-bed hours, raw coverage of the in-bed seconds and its source (archive, or
local-export:<path> for /api/export/archive tarballs saved before the archive), Watch
teachers per type, vitals lineage, whether the night is in the locked holdout, and whether
its raw is at risk (only on the pod, within 3 days of the pod's oldest raw). At-risk nights
print first. --storage is a saved GET /api/system/storage; without it the oldest raw is
estimated from the first pull. Decoded coverage is cached by RAW SHA-256 in
data/cache/raw-coverage/. The archive never deletes, so a sleep record edited on the pod
stays as two rows; the index marks the old copy supersededBy the newest one, and
export-runtime-features.ts trains on the newest copy only. overlaps lists other records
on the same side that share in-bed time. A record deleted on the pod stays in the archive and
can't be told apart.
lineage.py reports which vitals engine and profile produced each night's stored vitals,
with core's identity: each row's live processing run (manifest hash and config hash). A night
with more than one identity, or only partly recorded, is mixed; one with no recorded run is
unknown. It also reports the core build from the first pull after the night,
and the stage-model selection from an optional saved GET /api/sleep-stages/models?side=
(--stage-models left=FILE); the pod restages on read.
Nights whose Watch data never reached the pod can take a sidecar from an Apple Health export. The index prefers the pod's rows and marks sidecar nights in lower case:
uv run --locked python scripts/parse-watch-export.py data/export.zip data/watch.json
uv run --locked python scripts/parse-watch-export.py data/export.zip data/watch-vitals.json --vitals
uv run --locked python scripts/watch_sidecar.py import --archive data/archive/<podId> --side left \
--timezone America/Los_Angeles --stages data/watch.json --vitals data/watch-vitals.jsonThe archive database works as --snapshot for train.py, export-runtime-features.ts and
the stage scripts, and its raw as movement_model.py epochs --archive. Archived RAW files
for a session's window are chosen like the pod's export (mtime within 20 min before and
45 min after) and decode to the same records as a tarball of those files. Each pull rewrites
biometrics.db, and the scripts check the snapshot's SHA-256 against the features, so copy
the database into the run folder first to keep a run reproducible:
cp data/archive/<podId>/biometrics.db data/runs/RUN/biometrics.db
uv run --locked python scripts/train.py --snapshot data/runs/RUN/biometrics.db \
--watch-export data/export.zip --side left --timezone America/Los_Angeles --out data/runs/RUN/train
uv run --locked python scripts/movement_model.py epochs data/runs/RUN/train/features.json \
--snapshot data/runs/RUN/biometrics.db --model data/runs/movement-model/model.joblib \
--archive data/archive/<podId> --out data/runs/RUN/movement-epochs.jsonStage models trained on database rows learn from whatever vitals profile was live each night. A derived snapshot replaces those rows with vitals replayed from raw through one pinned engine and profile (plan C5). Core does the replay; this repo only copies its readings into a database the stage scripts already read.
# In a core checkout: replay archived nights with one profile (defaults, or a profile JSON)
pnpm biometrics:local-fit replay --archive <training>/data/archive/<podId> --nights all --side left \
--timezone America/Los_Angeles --config defaults --compare-stored <training>/data/replays/defaults
# Here: cache the readings, then derive a snapshot from them
uv run --locked python scripts/derive_snapshot.py derive data/archive/<podId>/biometrics.db \
--replay data/replays/defaults --out data/derived/defaults/biometrics.db
npm run features -- data/derived/defaults/biometrics.db data/watch.json data/runs/replayed/features.json left America/Los_Angeles podContextderive copies the database and, for each replayed night and side, replaces the vitals
and vitals_quality rows inside the night's in-bed window with the replay's readings; the
other side and every other table are unchanged. The derived_snapshot table records each
night's source database SHA-256, engine (biometrics manifest hash), profile (resolved config
hash), readings SHA-256 and replay.json SHA-256. Readings are cached once per
(night, engine, profile) in data/cache/vitals/<night>-<engine12>-<profile12>.json; derive --engine E --profile P --side left --timezone TZ rebuilds from the cache alone, and a night
whose sleep records changed since its replay is refused.
Replayed readings replace a night's rows only when core's training gate passed that night.
The gate comes from replay --compare-stored, in nights[].parity.trainingGate, and it
judges level, scatter and coverage against the stored rows, only on nights whose rows ran
the replayed profile. --allow-ungated also accepts nights the gate could not judge
(not_gated_*, no_heart_rate_to_compare, or a replay without --compare-stored). That
is the normal case when replaying a profile other than the one stored. A failed night is
never used. Every other night keeps its stored rows. Each night's verdict, core's check
values and the reason are recorded in derived_snapshot. The thresholds are core's and rest
on two real nights so far, so treat a pass as a coarse check, not proof of parity.
scripts/search.py searches over vitals profiles, movement models and stage models on one
fixed protocol: locked holdout, walk-forward, cross-fitting, paired night bootstrap and one
pre-registered comparison. Each node's output is cached once per version, and the single
locked-holdout read ends in RUN/model/set.json. See docs/search.md
for specs, the protocol and the prepared E2 run. The synthetic final above uses its own
access log, so synthetic reads never count against the real holdout.
scripts/modelset.py writes RUN/model/set.json (modelset-v1) next to an exported model:
the stage model pinned by core's own hash (modelset.py hash candidate.json; the canonical JSON
is ported once, in scripts/canonical_json.py, and pinned to vectors core computed), the vitals engine and profile it
expects, and a movement model when one is used. training.inputs is replayed or live.
Every successful run automatically saves model/candidate.json (portable pod
model) and model/candidate.joblib (fitted Python model) inside its run folder.
The Log tab also offers Download candidate JSON. Keep the complete run
folder to retain evaluation results, feature contracts, provenance, and parity
checks. These local artifacts are Git-ignored; back them up separately.
Saving or downloading a candidate does not activate it on the pod.
Checks whether the pod's raw piezo channels carry 16–30 Hz and 60–100 Hz energy before any snoring capture is built (section 0 of the overnight capture plan).
uv run --locked python scripts/piezo_bandwidth.py session --side left --pod 192.168.1.88 --out data/bandwidth/steps.json
uv run --locked python scripts/piezo_bandwidth.py analyze --steps data/bandwidth/steps.json --pod 192.168.1.88 --out data/bandwidth/session
uv run --locked python scripts/piezo_bandwidth.py night --start 2026-10-22T23:00-07:00 --end 2026-10-23T07:00-07:00 --pod 192.168.1.88 --out data/bandwidth/nightsession walks through seven one-minute steps (empty bed with pump off and on,
lying still, humming, imitated snoring, knocks at the head and foot) and saves
each step's UTC window plus a coarse pod clock offset read from the HTTP Date
of GET /api/settings. Run analyze after the session. It downloads the
pod's raw files with GET /api/export/archive (read-only), shifts step windows
by the measured offset, and scores the pass bar fixed in PASS_BAR: a band
passes when humming or snoring raises it at least 6 dB over lying still on the
occupant's channel. Each step used for scoring needs 80% of its samples. The
outcome is both-bands, low-band-only, high-band-only, neither, or
incomplete when a needed step lacks data. Knocks are scored separately: both
knock steps need at least three transients on every channel present. They gate
the knock clock sync, not the band outcome.
Finished pod frames can wait up to about 30 minutes before the export sees
them; both commands warn when data ends early, so fetch again into a fresh
--out. night reports channel count, coverage, archive gzip ratio and
hourly band-power percentiles for one night; run it the next morning, before
the pod's disk-usage pruner can remove the night. Add --export path.tar.gz to
either command to reanalyse a saved download. A failed or truncated download is
deleted. Downloads and reports stay under ignored data/; reports hold spectra
summaries, not samples.
Compares a movement classifier built from raw piezo and capacitance with the pod's
own per-minute movement score. Labels come from scripted sessions: the computer
speaks each cue aloud (say on macOS) and records when it asked for the movement.
uv run --locked python scripts/movement.py session --side left --pod 192.168.1.88 --out data/movement/session-1.json
# about 45 minutes later, once the pod has written the data:
uv run --locked python scripts/movement.py train --session data/movement/session-1.json --pod 192.168.1.88 --out data/runs/movement-001The session takes about seven minutes: lie still, then move an arm, a leg, turn over
and sit up when prompted, then lie still again. Training cuts the session into 2 s
windows. Each window gets the class of a movement it half covers, still when it is
clear of every movement, or is skipped as a transition. Repeat --session to add
sessions; with two or more, each session is held out in turn, and with one the
folds keep each movement's windows together, which is optimistic. The report gives
moving recall and false alarms while still for both the classifier and the pod
score, plus per-class recall for the classifier. The pod score is one value per
minute, so it is scored on the minute around each window and cannot tell movement
types apart. Cued movements are deliberate, not natural sleep movements.
Training never uploads or activates a model. These steps change what the pod shows,
so do them deliberately, one side at a time. They need a pod build with personal
stage models (core dev); <pod> is your pod's address and RUN your run folder.
# 1. Preview: stages recent nights with the candidate and compares it with the rules. Stores nothing.
jq -n --slurpfile m RUN/model/candidate.json '{side:"left", nights:7, model:$m[0]}' |
curl -s -X POST -H 'Content-Type: application/json' -d @- http://<pod>:3000/api/sleep-stages/model/preview
# 2. Add it to the side's model library. It is stored, not selected, not active.
jq -n --slurpfile m RUN/model/candidate.json '{side:"left", model:$m[0]}' |
curl -s -X POST -H 'Content-Type: application/json' -d @- http://<pod>:3000/api/sleep-stages/models
# 3. List the library, then select the new entry. "active": true makes it stage nights.
curl -s 'http://<pod>:3000/api/sleep-stages/models?side=left'
curl -s -X PUT -H 'Content-Type: application/json' -d '{"side":"left","id":ID,"active":true}' \
http://<pod>:3000/api/sleep-stages/models/selectedThe pod refuses a model whose feature contract it does not know or whose embedded
test vectors it does not reproduce. To go back to the rules, send {"side":"left","active":false}
to PUT /api/sleep-stages/model/active, or use the switch in the pod's Settings. Do not use
PUT /api/sleep-stages/model to try a model: on a side whose model is active it replaces
the model that is staging nights immediately.
The preview stages nights the candidate was trained on, so its agreement is in-sample and reads higher than the held-out scores in the report. Held-out scores are retrospective too. Treat activation as a trial: compare the next nights with the Watch before keeping it.
uv run --locked python scripts/validate_knowledge.py
uv run --locked python scripts/check-vendor.py
uv run --locked python -m unittest discover -s scripts -p 'test_*.py'
npm run typecheck
node --check scripts/ui/dashboard.js
node --test scripts/test-dashboard.mjs
uv run --locked python scripts/make-synthetic-inputs.py data/smoke-inputs
uv run --locked python scripts/train.py \
--snapshot data/smoke-inputs/biometrics.db --watch-export data/smoke-inputs/export.zip \
--side left --timezone UTC --out data/smoke-run
uv run --locked python scripts/make-synthetic-inputs.py data/smoke-graph --nights 12 --replay data/smoke-graph/replay
uv run --locked python scripts/derive_snapshot.py derive data/smoke-graph/biometrics.db --replay data/smoke-graph/replay \
--cache data/smoke-graph/cache --out data/smoke-graph/derived/biometrics.db
uv run --locked python scripts/parse-watch-export.py data/smoke-graph/export.zip data/smoke-graph/watch.json
npm run features -- data/smoke-graph/derived/biometrics.db data/smoke-graph/watch.json data/smoke-graph/features.json left UTC podContext
uv run --locked python scripts/stage-backtest.py data/smoke-graph/features.json \
--snapshot data/smoke-graph/derived/biometrics.db --out data/smoke-graph/backtest
uv run --locked python scripts/make-synthetic-inputs.py data/smoke-search --nights 14 --start 2026-11-03 \
--watch-vitals --replay data/smoke-search/replay
uv run --locked python scripts/derive_snapshot.py ingest data/smoke-search/replay --cache data/smoke-search/cache/vitals
uv run --locked python scripts/parse-watch-export.py data/smoke-search/export.zip data/smoke-search/watch.json
uv run --locked python scripts/parse-watch-export.py data/smoke-search/export.zip data/smoke-search/watch-vitals.json --vitals
uv run --locked python scripts/search.py plan docs/search/synthetic.json --out data/smoke-search/run --cache data/smoke-search/cache
uv run --locked python scripts/search.py run data/smoke-search/run --jobs 2
uv run --locked python scripts/search.py final data/smoke-search/run --candidate replayed+none+podContext \
--access-log data/smoke-search/holdout-access.jsonlTo also exercise pod fetching against a synthetic local HTTP server:
uv run --locked python scripts/smoke-pod.py data/smoke-inputs data/smoke-run/features.json data/pod-smoke-runSynthetic fixtures are invented and deterministic. Use fresh directories to repeat the smoke run. Their scores demonstrate plumbing only, not model quality. CI runs these checks with synthetic inputs.
The knowledge/ bundle follows Google's OKF v0.2 specification.
Concepts are agent-authored drafts unless actually reviewed; structure validation
does not verify scientific claims. Update source provenance, generated metadata,
directory indexes, and knowledge/log.md when changing concepts. Add verified
only after a real review, naming the reviewer and scope.
Aggregate evidence still describes personal data. Review it before public release.
The importer follows iOS docs/watch-motion-format.md and
Shared/WatchMotion/WatchMotionFile.swift on dev: watch-motion-v1, 50 Hz
acceleration in g, 2-second bins (not per-second), jerk in g/s, variance in
g², Unix milliseconds, IANA zone and its offset at capture start. Unknown
versions, unsupported bin/rate/jerk settings, malformed columns, duplicate bins,
invalid counts and inconsistent zones are refused. Optional raw samples remain
under ignored data/; no health data belongs in fixtures or CI.
watch-motion-v2 adds per-bin meanX, meanY and meanZ; labels from v2
captures carry angleDeg, the van Hees arm angle, with no threshold applied yet.
Bins wholly outside the recorded span (leftovers that older Watch builds
appended from a previous recording) are trimmed and counted in trimmedBins.
watch_motion.py also takes the iOS Export all file (watch-motion-all-*.json,
an array of captures); a night that appears in it and in a single-night file is
imported once. movement.py --watch-motion still pairs one single-night file
with one archive.
uv run --locked python scripts/watch_motion.py data/watch/night1.json data/watch/night2.json --out data/watch/labels.json
uv run --locked python scripts/movement.py train \
--watch-motion data/watch/night1.json=data/raw/night1.tar.gz \
--watch-motion data/watch/night2.json=data/raw/night2.tar.gz \
--side left --movement data/pod-movement.json --out data/watch-movement-runUse --pod <address> instead of saved exports/movement to fetch with archive
slack. Existing --session inputs can be mixed with Watch captures; specify
--timezone for scripted night grouping. Watch captures use their recorded zone.
Each window is grouped by its source Watch time and recorded zone (before clock
alignment), so captures spanning local noon are split between nights. Windows
straddling noon are excluded to avoid sharing samples across folds.
All captures from a local noon-to-noon night stay in the same held-out fold;
a single Watch night cannot produce a validation score. Overlapping windows
are deduplicated and conflicting labels refused. The report compares the
classifier with the existing per-minute pod movement score on identical windows.
Thresholds are fixed in watch_motion.py before observing results: at least 80
of the expected 100 samples per bin; moving when at least 5% exceed the exported
1 g/s jerk threshold; vigorous at 20%, otherwise light movement or still.
These are provisional heuristics, not measured classifier accuracy. Missing or
under-covered bins remain unknown, including when aggregating a window; they
never become zero movement. Keep thresholds fixed for an evaluation; any change
needs a new label version and fresh held-out evaluation.
Watch motion is a movement label only. Never use it as a sleep-stage feature when Watch stages are the target: the Watch's own stages use acceleration, so that would leak the label.
Both devices use wall time; NTP alone does not verify sensor alignment. When
knock or turn events are available, pass --alignment to the importer or
--watch-alignment to movement.py train for one capture. The JSON contract is
{"version":"capture-alignment-v1","confirmation":"schedule","pairs":[{"sourceMs":...,"podMs":...},...]}.
Use at least three matched events near each end, confirmed against the schedule
or raw energy (confirmation: "energy-fit"). Ending knocks were sharper in the
2026-10-22 movement session: inspect those first, then check the opening group.
The end-anchored linear fit records offset, drift and residual uncertainty;
missing checks are explicitly reported as NTP-only and unverified.
The importer follows iOS docs/snore-capture.md and Services/Snore/:
sleepypod-snore-capture-v1, units: "unix-ms", recorded IANA zone,
iphone-microphone, and the known Apple classifier/hysteresis version.
It validates the microphone format, classifier parameters, bounds, predicted
event provenance, confidences, coverage, interruptions and knock timestamps.
Export a stopped session; unfinished captures and unknown versions are refused.
No audio is read, stored or required.
uv run --locked python scripts/snore_labels.py \
--session data/snore/phone-night1.json --export data/raw/night1.tar.gz \
--side left --out data/snore-night1
# Or --pod <address> in place of --export; fetch waits are not implicit.
uv run --locked python scripts/snore_labels.py \
--reports data/snore-night1/report.json data/snore-night2/report.json \
--out data/snore-exitFetch after finished frames have reached the archive (about 30–45 minutes),
with one hour of end slack. Decoding and band energy reuse raw_piezo and
piezo_bandwidth; source SHA-256 hashes and decode errors accompany each run.
Three-knock schedules at both ends are matched to sharp piezo transients,
ending first, with a residual-energy timing fit; ambiguous matches refuse sync.
Use --alignment data/snore/matches.json for independently confirmed pairs as
above. Offsets beyond the automatic ±5 s search need explicit matched pairs.
Without matched knocks the timeline uses wall time, says unsynchronized,
and is excluded from event-level evaluation. A full night on real hardware is
still needed to validate this detector and uncertainty estimate.
Reports have kind: "snore-labels" and register in the catalog's Snore view.
Phone events overlay the raw night band-energy timeline with confidence tooltips,
knock markers, phone coverage, and the exit-test result/per-night AUCs. Green
means recording with no detection, gray means not recording, gold means a phone
prediction. A prediction is not verified snoring or a diagnosis.
snore_exit.py preregisters section 4 of
docs/overnight-capture-implementation-plan.md: one 3 s log-power score in the
lower passing band on the preregistered occupant channel, normalized by that
night's median occupied pump-off windows; single occupant only; negatives below
half the calibrated threshold within 30 minutes of positives; movement/pump
exclusions with a 10 s guard; median scores per bout (positive windows less than
30 s apart merge). Negative windows also merge into control bouts to avoid
window pseudoreplication. At least five eligible nights, at least three with
five positive bouts, are required. Report per-night AUCs and pooled AUC with a
95% interval from 2,000 night-resampled bootstrap draws (seed 42). Pass is AUC
≥0.7 and lower bound >0.5; fail is upper bound <0.7; otherwise inconclusive.
Duplicate nights and mixed calibration/threshold/band contracts are refused.
Current iOS v1 saves merged events and peak confidence, not all detector windows. Therefore no event does not establish confidence below threshold/2. These imports remain inconclusive for the exit test; the importer never invents negative confidences. Real overnight exports alone will not fix this limitation: a calibration session and a future per-window confidence export are needed.
For synthetic checks or a separately produced calibrated window export, optional
--window-evidence data/snore/windows.json accepts snore-window-evidence-v1:
captureId, registeredBeforeCaptureMs, calibrationId, threshold,
passingBands (ordered 16-30Hz, 60-100Hz, or one passing band),
primaryChannel, occupants, windows (startMs, endMs, confidence in the
phone clock), and occupied, pumpOff, pumpActive, movement arrays of
{startMs,endMs} in the pod clock. Unknown pump/occupancy coverage is excluded.
This companion contract is not claimed to be an existing iOS file format.
The code computes scores directly from raw piezo; it never accepts supplied
scores. Missing coverage, evidence or synchronization yields inconclusive.
All implementation tests use deterministic synthetic files. Still needed on real files: overnight Watch coverage and heuristic quality, phone calibration, clock-match robustness/drift, recording interruption behavior, and at least the preregistered night/bout minimum. No new empirical findings are claimed here.

