Skip to content
sleepypodPublic

About

Biometric model training knowledge, evaluation protocols, and aggregate research evidence in Open Knowledge Format.

Resources

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

Training

sleepypod/training — offline sleep-stage model training and an OKF knowledge base. Available publicly at sleepypod/training.

Training workspace

Watch training progress, compare held-out models with the pod rules and Watch labels, and reopen previous runs. See the sleepypod documentation for the workflow and how it relates to personal models on the pod.

Training overview with run history, pipeline status, and held-out scores

Sleep-stage comparison showing Pod, candidate, baseline, and Watch timelines

These are unmodified captures of the running app with deterministic synthetic test data. The displayed scores illustrate the interface, not real-world accuracy.

Run it yourself

You need a Mac (or Linux machine) on the same network as your pod, an Apple Watch that tracks sleep, and about ten minutes.

  1. Install the tools. With Homebrew: brew install node@24 git jq. (jq is only needed for the model-upload commands.) The quickstart offers to install uv, which brings its own Python.

  2. Get this repository: git clone https://github.com/sleepypod/training.git && cd training.

  3. Find your pod's address. The sleepypod iOS app shows it under Settings.

  4. Export your Watch sleep data (skip this if the iOS app already syncs Watch data to the pod): on the iPhone open Health, tap your profile picture, then Export All Health Data. AirDrop or save export.zip to the Mac. Large exports take a few minutes.

  5. Run the guided script:

    scripts/quickstart.sh

    It checks Node.js and uv, installs the pinned dependencies, confirms it can reach the pod, asks which side the Watch wearer sleeps on and where export.zip is, then trains. Pass --pod, --side and --watch-export to skip the questions, or --dry-run to check everything without training.

  6. Read the results in the dashboard link the script prints, or later with uv run --locked python scripts/dashboard.py. Held-out macro F1 and κ for the candidate, the existing baseline and the pod's rules are the numbers to compare; see interpreting model metrics.

Training only reads from the pod. It needs at least five nights with Watch sleep stages, and more nights help most. To put a model on the pod, see Use a trained model on the pod.

Setup

Install Node.js 24 and uv, then from this repository:

npm ci
uv sync --locked

package-lock.json pins the TypeScript tooling and SQLite dependency. uv.lock pins Python 3.11 numerical dependencies. Production features, rules, and portable inference are vendored from a pinned core revision, with checksums; no sibling core checkout or unpublished commit is needed to run training.

Query the pod and watch training

uv run --locked python scripts/train.py --pod 192.168.1.88 --side left --dashboard --verbose

Replace the example IP with your pod. The fetcher uses unauthenticated, read-only HTTP on port 3000; --pod hostname:port also works. It reads the pod timezone, completed sessions from the last 30 days, vitals, movement, imported Watch labels, and the current stage rules. Use --days 14 to change the window or --timezone to explicitly override the pod setting. No MCP client or SSH is needed. On macOS, Local Network privacy can refuse Python's connection to the pod (No route to host) although curl reaches it; pod reads then retry once through /usr/bin/curl, which is exempt. Grant your terminal Local Network access in System Settings to avoid it.

If Watch labels are not already imported on the pod, add --watch-export data/export.zip or --watch-json data/watch.json. Fetching does not import labels, modify settings, or activate a model.

Terminal progress shows each phase, completed night folds, model fits, and elapsed time. --verbose keeps every event on its own terminal line. All runs also save events.jsonl with the complete structured history. The dashboard shows:

  • Response-body bytes, average download throughput, API calls, and unique input rows.
  • Labeled/unlabeled feature epochs, class balance, and per-feature min/mean/max/std.
  • Per-fit input rows (before/after oversampling), fit time, rows/s, and actual tree/node counts.
  • A fit-time chart and per-night held-out macro F1 for nested selection versus the baseline.
  • A scrolling activity feed with inner-validation scores and current evaluation phase.

Throughput measures wall-clock work for this run, not sensor cadence. Feature matrix bytes describe float64 storage, not process memory. Per-night scores are not a loss/learning curve, and partial results are not a final quality estimate. Dashboard history is bounded (120 events, 80 fit timings); events.jsonl retains all events. Feature summaries are descriptive only and do not affect fitting. --dashboard prints a loopback URL for a live browser view with progress, model comparisons, selectable metrics, and confusion matrices. The sleep comparison at the top aligns Pod / New model / Apple Watch stages by session. Select a night, hover an interval, or use Previous/Next to inspect exact stages. Orange marks Pod/model disagreements. Held-out predictions appear as each night finishes; the final fitted-model view is explicitly in-sample. It stays open after training until Ctrl-C. Without this flag the command exits when training finishes. Both modes save index.html in the run directory; open it directly for an offline interactive report. A failed or interrupted run retains its status and partial files; retry into a new directory.

API reads are not an atomic database snapshot. The fetcher limits acquisition to completed sessions, splits saturated API windows, checks side identity, detects configuration/calibration changes, and retains response hashes in acquisition.json. Concurrent historical backfills can still alter data between reads. Use a consistent SQLite backup when atomic input consistency is required. Timeline Pod stages come from GET /api/biometrics/sleep-stages for each record, with record identity and stage-view consistency checked. The five-minute comparison samples pod/Watch stages at epoch midpoints; it does not display their original sub-epoch boundaries. Snapshot-only runs label the pod lane as a local rules replay. Aggregate “Rules (replay)” metrics still use the pinned local rules engine. The current rules are applied retrospectively; they do not reconstruct historical per-sample rule versions. Raw responses and reports remain under ignored data/.

Reopen runs and explore the training workspace

uv run --locked python scripts/dashboard.py --port 8765

Open http://127.0.0.1:8765. This starts only the viewer; it never retrains. The workspace discovers earlier run directories under this repository's data/ and its Git worktrees, and keeps a persistent SQLite catalog at the primary checkout's ignored data/run-history.sqlite. New training runs register there automatically. --out still requires a fresh directory; the default is now the primary checkout's data/runs/, shared across worktrees.

Use --run data/runs/example to open a specific existing run, or add --import-root /path/to/old/runs (repeatable) to discover runs stored elsewhere. Importing registers paths; it does not move, overwrite, or fabricate artifacts. Keep imported folders in place. --catalog data/another.sqlite selects a separate catalog. The viewer serves only selected report artifacts and never serves raw snapshots or arbitrary files. Historical runs with stale running state and no known live process are labeled interrupted, without editing their saved state. Runs with a report but no pipeline status are archived, not assumed to have passed parity.

Actions (bottom of the sidebar) runs the repository's own scripts from the workspace: train.py --snapshot, stage-backtest.py, fusion.py, vitals_reference.py, movement_model.py train and epochs, archive_index.py, search.py plan, run and report, and the archive pull that scripts/install-archive-agent.sh schedules (core's read-only biometrics:archive pull, resolved by that script's --print-command from data/workspace-settings.json). One job runs at a time; starting another while one runs is refused (HTTP 409) rather than queued, so a job never starts on inputs that changed after it was requested. Each job writes data/web-jobs/<id>/command.json (the validated parameters, the exact argv and working directory, status and exit code) and log.txt in the primary checkout's data/, is registered in the run catalog, and adds its output to the run list when that output is a run. Parameters are checked against fixed schemas: input and output paths must resolve inside data/, outputs must be new, and nothing passes through a shell. No action uploads, selects or activates a model, or writes to the pod. Stopping the workspace interrupts a running job.

POST /api/jobs exists only in dashboard.py (not train.py --dashboard), listens on 127.0.0.1, and requires the random token embedded in that launch's page as an X-Training-Token header. Requests with another Host, a cross-site Origin or Sec-Fetch-Site, or a non-JSON body are refused, and every page is served only to the 127.0.0.1 and localhost host names, so a rebound DNS name cannot read the token.

Search in the sidebar lists model searches (scripts/search.py, kind search). A search shows its pre-registered primary comparison with the night-bootstrap interval and sign test, every candidate's pooled walk-forward scores on development nights, the exploratory comparisons against the baseline, layer scores, exclusions and limitations, and, with more than one search, a table comparing their primary results and winners. The view reads only each run's report.json. A search's final reads the locked holdout once and exports model/set.json. It is not a workspace action, and final.json, where its scores live, is never read or served; the view shows only how many holdout reads there have been.

A Sleep / Movement / Snore switch at the top of the sidebar picks the kind of research run; the run list shows only that kind. Movement runs (scripts/movement.py train) and snore runs (scripts/piezo_bandwidth.py analyze and night) register themselves when their output is under data/.

  • Movement: windows, validation scheme and sessions; the pod movement score and the classifier side by side (moving recall, false alarms while still, balanced accuracy, and how often each movement type was flagged as moving); the classifier's confusion matrix and per-class recall; and a per-session timeline of labels, classifier predictions and pod-score flags. The pod score is one value per minute and cannot type movements; one-session validation is optimistic.
  • Snore: a bandwidth session shows each band's rise over lying still against the 6 dB pass bar, the outcome, knock counts (which gate clock sync, not the outcome), pump speed and per-step band energy. A night shows 30 s band energy for every channel in both snore bands, the pump, coverage and storage, and, when a session from the same pod exists, its lying-still and imitated-snore levels as reference lines. This is vibration energy, not detected snoring.

Sleep runs are organized around a run sidebar, sticky pipeline header, and five tabs:

  • Overview: held-out scores, review checks, four-lane stage comparison, small hypnograms for every night, sleep architecture, fold results, per-stage F1, confusion matrices, feature importance, resource measurements, and notes.
  • Nights: Pod / candidate / baseline / Watch aligned on one axis, a night picker, candidate-vs-Watch or candidate-vs-Pod disagreement navigation, per-epoch inputs, exact stages, and matched stage-duration comparisons.
  • Weeks: stage minutes per night grouped into ISO weeks by local night date, a source selector, and weekly averages per source with differences from Watch. Each night counts only intervals shared by every displayed source; nights without shared data are gaps, never zero.
  • Data: intake, label coverage, per-night balance, feature histograms and descriptive distribution comparisons. Missing raw sensor readings cannot be inferred from finite feature values because some features use fallback values.
  • Log: filterable activity, full event loading/download, captured failure details, configuration/provenance, and a candidate download. No activation.

Sidebar, compact-header and wide-screen inspector layouts are available. Every technical term has contextual help or a searchable Learn the terms entry, including macro F1, kappa, depth, leaves, cross-validation, and all input features. The same help, tabs and plots work in exported self-contained HTML reports. Offline reports contain one run; the running workspace supplies history.

New runs record baseline traces, fold details, environment versions, command, notes, per-fit CPU and peak trainer RSS, and three-repeat held-out permutation importance using the same sequential postprocessing as scoring. These descriptive measurements never affect candidate selection. Historical missing measurements are shown as unavailable, not reconstructed through new fits. Feature distributions can be derived from retained feature files without retraining.

Sleep architecture uses the intersection of epochs present in every displayed source, including Watch. All bars therefore share a denominator; gaps are not filled as wake. Hypnograms show stage sequences, not validated cycle boundaries. Final-fit views are explicitly in-sample. Bootstrap intervals concern the mean night-level improvement, not the pooled score difference. Review checks are not release gates: promotion thresholds have not been agreed.

Add --name 'Leaf experiment' --notes 'Hypothesis and changes' to train.py to label a run. Failure tracebacks and subprocess output remain in ignored error.txt / process.log. There is no fabricated resume or promote command; correct a failure and retry into a new output directory.

Train with your data

Put local inputs in data/. Only data/.gitkeep is tracked; all other contents, including subfolders and generated outputs, are ignored. Raw data and fitted artifacts must not be force-added. Git ignore rules do not encrypt local files.

Required inputs:

  • A consistent biometrics.db SQLite backup containing sleep records, vitals, movement, runtime configuration, and calibration tables. See snapshot instructions.
  • An Apple Health export ZIP, export.xml, or directory containing export.xml. The parser selects Watch sleep stages and always excludes SleepyPod writeback.
  • The Watch wearer's pod side and the pod's IANA timezone.
  • At least five independent night groups. Each retained session must last 3–14 hours and have at least 24 labeled five-minute epochs. Training folds need enough class diversity to fit the classifiers. This workflow currently evaluates one participant; do not mix people into its night-only split scheme.
uv run --locked python scripts/train.py \
  --snapshot data/biometrics.db \
  --watch-export data/export.zip \
  --side left --timezone America/Los_Angeles \
  --out data/runs/experiment-001

--contract podContext trains on the 33-column contract (the ten podNative features plus context, movement-bout and arousal features from scripts/feature-ablation.py). Pods need core with podContext support (core#907) to accept and evaluate such a model; podNative remains the default.

The output directory must not already exist. Use --source 'Watch' to choose a case-insensitive Health source-name substring. To reuse extracted labels, replace --watch-export with --watch-json data/watch.json (schema in input data).

The command checks the vendor snapshot, parses labels, exports production runtime features, performs nested whole-night model selection, writes the candidate, and checks all exported predictions against the TypeScript evaluator. It never uploads or activates the model. It produces:

  • watch.json when parsing an export or fetching pod labels, plus features.json.
  • progress.json, the full events.jsonl log, and an offline interactive index.html report. Pod acquisition also creates biometrics.db, responses/, and acquisition.json.
  • model/report.json and model/protocol.json with evaluation and provenance.
  • Private model/timelines.json with aligned Watch, captured pod, held-out, and final-model predictions. Pod acquisition also saves pod-stages.json.
  • model/candidate.json (pod-compatible model), model/candidate.joblib (Python fit), and model/parity-inputs.json (private verification rows).

Do not use these scores as prospective validation. Features use whole-night and centered-window context, and this is a retrospective classifier. See the validation protocol and latest historical findings.

Individual steps

uv run --locked python scripts/parse-watch-export.py data/export.zip data/watch.json
npm run features -- data/biometrics.db data/watch.json data/features.json left America/Los_Angeles
uv run --locked python scripts/retrain-models.py data/features.json --out data/runs/manual-model
npm run parity -- data/runs/manual-model

Literature-derived feature ablation (context, movement bouts, arousal coupling, and a wake-first hierarchical forest) on an existing run's features and the snapshot they came from. It compares arms only and never exports a model:

uv run --locked python scripts/feature-ablation.py data/runs/RUN/features.json \
  --snapshot data/biometrics.db --out data/runs/ablation

Chronological backtest of the same feature sets: refit before each night on earlier nights only (walk-forward) and fit once on the first nights (frozen), scored against the stages the pod recorded live:

uv run --locked python scripts/stage-backtest.py data/runs/RUN/features.json \
  --snapshot data/biometrics.db --out data/runs/backtest

Locked holdout

From the night of 2026-11-14, every fourth night group (11-14, 11-18, 11-22, …) is locked (rule, scripts/holdout.py). train.py, retrain-models.py, stage-backtest.py, feature-ablation.py, fusion.py and sequence-decoding.py drop locked nights before any fitting, selection or feature analysis, and record the rule and the dropped groups under holdout in their protocol. The dashboard shows "Locked holdout" in place of scores for those nights.

--holdout-final is the only way a development run reads them, and only train.py and retrain-models.py accept it. Training and selection still use development nights only. After export, the run appends the candidate's SHA-256 and the locked nights to data/holdout-access.jsonl in the primary checkout, then scores the candidate on them once (holdoutFinal in model/report.json). Every read spends the holdout, so make one per finished candidate. The other scripts fit, tune or compare on the nights they score, so they refuse the flag.

uv run --locked python scripts/retrain-models.py data/runs/RUN/features.json \
  --out data/runs/RUN-final --holdout-final

Weekly trial of the deployed model

stage-backtest.py --trial scores two frozen pod models against the Watch on the same new nights (night groups after 2026-11-13, plan E1): the stages the pod shows (pod_live, read with GET /api/biometrics/sleep-stages; the pod restages on read with its selected model, library id 3) and library id 2 staged by the read-only POST /api/sleep-stages/model/preview. The library API lists models without their JSON, so id 2 comes from a local copy, checked against the library's version. Nights come from an archive snapshot (--snapshot, Watch stages from reference_stages) or a run's features.json (--run), aligned to the exporter's Watch 5-minute epochs; both models are scored on the same labelled epochs.

uv run --locked python scripts/stage-backtest.py --trial --pod 192.168.1.88 --side left \
  --snapshot data/archive/<podId>/biometrics.db \
  --baseline-model data/runs/<id-2 run>/model/candidate.json \
  --live-model data/runs/<id-3 run>/model/candidate.json

Each run appends one line per (night, model) to data/runs/trial-id3/weekly.jsonl with the model id, version, SHA-256 (core's stageModelSha256: canonical JSON with exact decimal numbers, so a pod copy and a local file agree), training cutoff, outOfSample, locked, metrics and scoredAt. Re-scoring a night replaces its lines. A night counts as out of sample only after a model's training cutoff, taken from --trained-through ID=YYYY-MM-DD, the model's metrics.trainedThrough, or the last night group of the run the local copy came from. Locked holdout nights are included, since both models are frozen, and each run logs them for both models in data/holdout-access.jsonl (trial) before reading. The summary gives per-night macro F1 and κ differences id 3 − id 2, a night-level bootstrap 95% interval (10,000 resamples, seed 42) and a sign-test count, over every night in the file that is out of sample for both. Pod reads are GETs and the preview only.

Multi-source training

Watch signals teach the pod models; they are never stage-model inputs. Watch stages are the stage labels, Watch motion labels the movement model, and Watch heart rate, HRV and respiratory rate check the pod's own vitals. Every model input comes from the pod, so a model never needs a Watch at night.

# Watch vitals, and how far pod vitals are from them epoch by epoch
uv run --locked python scripts/parse-watch-export.py data/export.zip data/watch-vitals.json --vitals
uv run --locked python scripts/vitals_reference.py data/runs/RUN/features.json \
  --snapshot data/biometrics.db --watch-vitals data/watch-vitals.json --out data/runs/vitals
# Pod movement model, labeled by Watch motion (windows.json from movement.py train --watch-motion)
uv run --locked python scripts/movement_model.py train --windows data/runs/movement-001/windows.json --out data/runs/movement-model
uv run --locked python scripts/movement_model.py epochs data/runs/RUN/features.json --snapshot data/biometrics.db \
  --model data/runs/movement-model/model.joblib --export SESSION_ID=data/raw/session.tar.gz --out data/runs/movement-epochs.json
# Portable forest for the pod from an existing model.joblib (train writes model/movement.json itself), checked in TypeScript
uv run --locked python scripts/movement_model.py export data/runs/movement-model/model.joblib --out data/runs/movement-model/model/movement.json
npm run movement-parity -- data/runs/movement-model/model/movement.json
# Stage model with pod features, HR quality and movement-model features
uv run --locked python scripts/fusion.py data/runs/RUN/features.json --snapshot data/biometrics.db \
  --movement-epochs data/runs/movement-epochs.json --pod-model-trained-through 2026-11-03 --out data/runs/fusion

vitals_reference.py reports pod-versus-Watch bias, error and correlation for heart rate, breathing rate and HRV (rank only: the pod and Watch compute different HRV statistics), how each signal moves with Watch stage, and which pod-only quality signals (hrCoverage, hrJitter, brCoverage) predict heart-rate error.

movement_model.py train holds out one Watch-motion night at a time and compares the pod movement score, the movement.py classifier and the new model on 2 s windows and 5-minute epochs. The model adds the pod score to the raw piezo and capacitance features, and its moving threshold is chosen on out-of-bag training predictions. epochs needs one raw export per sleep session (GET /api/export/archive, read-only; the pod prunes old archives), or --archive data/archive/<podId> to read every session's raw from the off-pod archive; an --export still wins for its session. Its features are cross-fitted: each session is scored by a model fitted without every Watch-motion night whose windows fall inside it, and once more per other Watch-motion night T by a model also fitted without T. model.joblib keeps the teaching windows so these models can be refitted; one trained by an earlier version is refused. Watch motion on locked holdout nights never teaches the movement model; epochs and fusion.py refuse a model that was taught on one. Moving windows are those strictly above the strict threshold, so tied probabilities at the cutoff never count as moving. Raw-piezo features need a core port before a pod can compute them.

train also writes model/movement.json, the final model as a portable forest (movement-model-v1, contract movementRaw-v1) with up to 64 test vectors and a SHA-256 of its canonical body. Its trees use the stage-model arrays plus float32 inputs and the routing of a missing pod score, so it reproduces scikit-learn exactly. The file is refused unless it matches predict_proba within 1e-9 on every training window, with and without the pod score. export writes it from an existing model.joblib; one saved before teaching windows needs the --windows it was trained on. The movementRaw-v1 contract defines every input for the core port. A 200-tree model is about 9 MB of JSON. Nothing is uploaded.

fusion.py evaluates the frozen ablation forest on the nights that have raw exports: base (podNative), all (the podContext columns), all plus HR quality, plus movement-model features, both (fused), and a wake-first hierarchical_fused. It scores every arm leave-one-night-out and chronologically, against the stages the pod recorded live. When night T is held out, every night's movement features come from models that never saw T's Watch motion. Chronological scores use only labelled epochs that also have a live pod stage, for every arm, so pod and model share one population (podCoverage per night). The input must use the podNative contract, since the arms add the podContext columns. --pod-model-trained-through reports nights after the pod model's training separately, because earlier nights are in-sample for it. Nothing is exported.

Off-pod archive

The pod prunes about one night of raw piezo a day. Core's archiver (pnpm biometrics:archive pull, in a core checkout) keeps everything it exports in data/archive/<podId>/: a merged biometrics.db, content-addressed raw/*.RAW.gz and one pulls/*.json record per window. The pull is read-only on the pod. Install a daily 12:30 pull as a launchd agent; set coreCheckout in data/workspace-settings.json (else ../sleepypod-core next to this checkout is used) and check the plist first:

scripts/install-archive-agent.sh --dry-run
scripts/install-archive-agent.sh              # logs to data/archive/pull.log
scripts/install-archive-agent.sh --uninstall

scripts/install-archive-agent.sh --print-command prints the same pull (working directory, PATH, argv) without installing anything; the training workspace's archive pull runs it.

In the training workspace (scripts/dashboard.py), Archive in the sidebar shows each data/archive/<podId>/index.json: archived nights, Watch coverage per teacher, vitals lineage, locked holdout nights (listed, never scored), nights at risk, and the age of the last successful pull (the newest pulls/*.json, which core writes only when a pull completes). It turns red when that pull is more than 3 days old, which is how a stopped nightly agent gets noticed before raw is pruned. Pull now and Rebuild index start the pull and archive_index.py as workspace jobs. While the workspace is open it also pulls when it starts and every 6 hours, as a job; it skips a round when core's .biometrics-archive.lock is held by a live pull, waits while another job runs, and never makes the first pull, which waits for the other sleeper's consent. --no-sync turns this off. It only protects nights while the workspace is open; the launchd agent is what keeps raw from being pruned unarchived.

Then index the archive by night. The index reads local files only:

uv run --locked python scripts/archive_index.py --timezone America/Los_Angeles \
  --local-exports-dir data/raw --storage data/storage.json
uv run --locked python scripts/lineage.py data/archive/<podId> --timezone America/Los_Angeles

archive_index.py writes data/archive/<podId>/index.json, one row per sleep record per side: in-bed hours, raw coverage of the in-bed seconds and its source (archive, or local-export:<path> for /api/export/archive tarballs saved before the archive), Watch teachers per type, vitals lineage, whether the night is in the locked holdout, and whether its raw is at risk (only on the pod, within 3 days of the pod's oldest raw). At-risk nights print first. --storage is a saved GET /api/system/storage; without it the oldest raw is estimated from the first pull. Decoded coverage is cached by RAW SHA-256 in data/cache/raw-coverage/. The archive never deletes, so a sleep record edited on the pod stays as two rows; the index marks the old copy supersededBy the newest one, and export-runtime-features.ts trains on the newest copy only. overlaps lists other records on the same side that share in-bed time. A record deleted on the pod stays in the archive and can't be told apart.

lineage.py reports which vitals engine and profile produced each night's stored vitals, with core's identity: each row's live processing run (manifest hash and config hash). A night with more than one identity, or only partly recorded, is mixed; one with no recorded run is unknown. It also reports the core build from the first pull after the night, and the stage-model selection from an optional saved GET /api/sleep-stages/models?side= (--stage-models left=FILE); the pod restages on read.

Nights whose Watch data never reached the pod can take a sidecar from an Apple Health export. The index prefers the pod's rows and marks sidecar nights in lower case:

uv run --locked python scripts/parse-watch-export.py data/export.zip data/watch.json
uv run --locked python scripts/parse-watch-export.py data/export.zip data/watch-vitals.json --vitals
uv run --locked python scripts/watch_sidecar.py import --archive data/archive/<podId> --side left \
  --timezone America/Los_Angeles --stages data/watch.json --vitals data/watch-vitals.json

The archive database works as --snapshot for train.py, export-runtime-features.ts and the stage scripts, and its raw as movement_model.py epochs --archive. Archived RAW files for a session's window are chosen like the pod's export (mtime within 20 min before and 45 min after) and decode to the same records as a tarball of those files. Each pull rewrites biometrics.db, and the scripts check the snapshot's SHA-256 against the features, so copy the database into the run folder first to keep a run reproducible:

cp data/archive/<podId>/biometrics.db data/runs/RUN/biometrics.db
uv run --locked python scripts/train.py --snapshot data/runs/RUN/biometrics.db \
  --watch-export data/export.zip --side left --timezone America/Los_Angeles --out data/runs/RUN/train
uv run --locked python scripts/movement_model.py epochs data/runs/RUN/train/features.json \
  --snapshot data/runs/RUN/biometrics.db --model data/runs/movement-model/model.joblib \
  --archive data/archive/<podId> --out data/runs/RUN/movement-epochs.json

Replayed vitals and model sets

Stage models trained on database rows learn from whatever vitals profile was live each night. A derived snapshot replaces those rows with vitals replayed from raw through one pinned engine and profile (plan C5). Core does the replay; this repo only copies its readings into a database the stage scripts already read.

# In a core checkout: replay archived nights with one profile (defaults, or a profile JSON)
pnpm biometrics:local-fit replay --archive <training>/data/archive/<podId> --nights all --side left \
  --timezone America/Los_Angeles --config defaults --compare-stored <training>/data/replays/defaults
# Here: cache the readings, then derive a snapshot from them
uv run --locked python scripts/derive_snapshot.py derive data/archive/<podId>/biometrics.db \
  --replay data/replays/defaults --out data/derived/defaults/biometrics.db
npm run features -- data/derived/defaults/biometrics.db data/watch.json data/runs/replayed/features.json left America/Los_Angeles podContext

derive copies the database and, for each replayed night and side, replaces the vitals and vitals_quality rows inside the night's in-bed window with the replay's readings; the other side and every other table are unchanged. The derived_snapshot table records each night's source database SHA-256, engine (biometrics manifest hash), profile (resolved config hash), readings SHA-256 and replay.json SHA-256. Readings are cached once per (night, engine, profile) in data/cache/vitals/<night>-<engine12>-<profile12>.json; derive --engine E --profile P --side left --timezone TZ rebuilds from the cache alone, and a night whose sleep records changed since its replay is refused.

Replayed readings replace a night's rows only when core's training gate passed that night. The gate comes from replay --compare-stored, in nights[].parity.trainingGate, and it judges level, scatter and coverage against the stored rows, only on nights whose rows ran the replayed profile. --allow-ungated also accepts nights the gate could not judge (not_gated_*, no_heart_rate_to_compare, or a replay without --compare-stored). That is the normal case when replaying a profile other than the one stored. A failed night is never used. Every other night keeps its stored rows. Each night's verdict, core's check values and the reason are recorded in derived_snapshot. The thresholds are core's and rest on two real nights so far, so treat a pass as a coarse check, not proof of parity.

scripts/search.py searches over vitals profiles, movement models and stage models on one fixed protocol: locked holdout, walk-forward, cross-fitting, paired night bootstrap and one pre-registered comparison. Each node's output is cached once per version, and the single locked-holdout read ends in RUN/model/set.json. See docs/search.md for specs, the protocol and the prepared E2 run. The synthetic final above uses its own access log, so synthetic reads never count against the real holdout.

scripts/modelset.py writes RUN/model/set.json (modelset-v1) next to an exported model: the stage model pinned by core's own hash (modelset.py hash candidate.json; the canonical JSON is ported once, in scripts/canonical_json.py, and pinned to vectors core computed), the vitals engine and profile it expects, and a movement model when one is used. training.inputs is replayed or live.

Save and reuse a model

Every successful run automatically saves model/candidate.json (portable pod model) and model/candidate.joblib (fitted Python model) inside its run folder. The Log tab also offers Download candidate JSON. Keep the complete run folder to retain evaluation results, feature contracts, provenance, and parity checks. These local artifacts are Git-ignored; back them up separately. Saving or downloading a candidate does not activate it on the pod.

Piezo bandwidth gate

Checks whether the pod's raw piezo channels carry 16–30 Hz and 60–100 Hz energy before any snoring capture is built (section 0 of the overnight capture plan).

uv run --locked python scripts/piezo_bandwidth.py session --side left --pod 192.168.1.88 --out data/bandwidth/steps.json
uv run --locked python scripts/piezo_bandwidth.py analyze --steps data/bandwidth/steps.json --pod 192.168.1.88 --out data/bandwidth/session
uv run --locked python scripts/piezo_bandwidth.py night --start 2026-10-22T23:00-07:00 --end 2026-10-23T07:00-07:00 --pod 192.168.1.88 --out data/bandwidth/night

session walks through seven one-minute steps (empty bed with pump off and on, lying still, humming, imitated snoring, knocks at the head and foot) and saves each step's UTC window plus a coarse pod clock offset read from the HTTP Date of GET /api/settings. Run analyze after the session. It downloads the pod's raw files with GET /api/export/archive (read-only), shifts step windows by the measured offset, and scores the pass bar fixed in PASS_BAR: a band passes when humming or snoring raises it at least 6 dB over lying still on the occupant's channel. Each step used for scoring needs 80% of its samples. The outcome is both-bands, low-band-only, high-band-only, neither, or incomplete when a needed step lacks data. Knocks are scored separately: both knock steps need at least three transients on every channel present. They gate the knock clock sync, not the band outcome.

Finished pod frames can wait up to about 30 minutes before the export sees them; both commands warn when data ends early, so fetch again into a fresh --out. night reports channel count, coverage, archive gzip ratio and hourly band-power percentiles for one night; run it the next morning, before the pod's disk-usage pruner can remove the night. Add --export path.tar.gz to either command to reanalyse a saved download. A failed or truncated download is deleted. Downloads and reports stay under ignored data/; reports hold spectra summaries, not samples.

Movement experiments

Compares a movement classifier built from raw piezo and capacitance with the pod's own per-minute movement score. Labels come from scripted sessions: the computer speaks each cue aloud (say on macOS) and records when it asked for the movement.

uv run --locked python scripts/movement.py session --side left --pod 192.168.1.88 --out data/movement/session-1.json
# about 45 minutes later, once the pod has written the data:
uv run --locked python scripts/movement.py train --session data/movement/session-1.json --pod 192.168.1.88 --out data/runs/movement-001

The session takes about seven minutes: lie still, then move an arm, a leg, turn over and sit up when prompted, then lie still again. Training cuts the session into 2 s windows. Each window gets the class of a movement it half covers, still when it is clear of every movement, or is skipped as a transition. Repeat --session to add sessions; with two or more, each session is held out in turn, and with one the folds keep each movement's windows together, which is optimistic. The report gives moving recall and false alarms while still for both the classifier and the pod score, plus per-class recall for the classifier. The pod score is one value per minute, so it is scored on the minute around each window and cannot tell movement types apart. Cued movements are deliberate, not natural sleep movements.

Use a trained model on the pod

Training never uploads or activates a model. These steps change what the pod shows, so do them deliberately, one side at a time. They need a pod build with personal stage models (core dev); <pod> is your pod's address and RUN your run folder.

# 1. Preview: stages recent nights with the candidate and compares it with the rules. Stores nothing.
jq -n --slurpfile m RUN/model/candidate.json '{side:"left", nights:7, model:$m[0]}' |
  curl -s -X POST -H 'Content-Type: application/json' -d @- http://<pod>:3000/api/sleep-stages/model/preview

# 2. Add it to the side's model library. It is stored, not selected, not active.
jq -n --slurpfile m RUN/model/candidate.json '{side:"left", model:$m[0]}' |
  curl -s -X POST -H 'Content-Type: application/json' -d @- http://<pod>:3000/api/sleep-stages/models

# 3. List the library, then select the new entry. "active": true makes it stage nights.
curl -s 'http://<pod>:3000/api/sleep-stages/models?side=left'
curl -s -X PUT -H 'Content-Type: application/json' -d '{"side":"left","id":ID,"active":true}' \
  http://<pod>:3000/api/sleep-stages/models/selected

The pod refuses a model whose feature contract it does not know or whose embedded test vectors it does not reproduce. To go back to the rules, send {"side":"left","active":false} to PUT /api/sleep-stages/model/active, or use the switch in the pod's Settings. Do not use PUT /api/sleep-stages/model to try a model: on a side whose model is active it replaces the model that is staging nights immediately.

The preview stages nights the candidate was trained on, so its agreement is in-sample and reads higher than the held-out scores in the report. Held-out scores are retrospective too. Treat activation as a trial: compare the next nights with the Watch before keeping it.

Validate without personal data

uv run --locked python scripts/validate_knowledge.py
uv run --locked python scripts/check-vendor.py
uv run --locked python -m unittest discover -s scripts -p 'test_*.py'
npm run typecheck
node --check scripts/ui/dashboard.js
node --test scripts/test-dashboard.mjs
uv run --locked python scripts/make-synthetic-inputs.py data/smoke-inputs
uv run --locked python scripts/train.py \
  --snapshot data/smoke-inputs/biometrics.db --watch-export data/smoke-inputs/export.zip \
  --side left --timezone UTC --out data/smoke-run
uv run --locked python scripts/make-synthetic-inputs.py data/smoke-graph --nights 12 --replay data/smoke-graph/replay
uv run --locked python scripts/derive_snapshot.py derive data/smoke-graph/biometrics.db --replay data/smoke-graph/replay \
  --cache data/smoke-graph/cache --out data/smoke-graph/derived/biometrics.db
uv run --locked python scripts/parse-watch-export.py data/smoke-graph/export.zip data/smoke-graph/watch.json
npm run features -- data/smoke-graph/derived/biometrics.db data/smoke-graph/watch.json data/smoke-graph/features.json left UTC podContext
uv run --locked python scripts/stage-backtest.py data/smoke-graph/features.json \
  --snapshot data/smoke-graph/derived/biometrics.db --out data/smoke-graph/backtest
uv run --locked python scripts/make-synthetic-inputs.py data/smoke-search --nights 14 --start 2026-11-03 \
  --watch-vitals --replay data/smoke-search/replay
uv run --locked python scripts/derive_snapshot.py ingest data/smoke-search/replay --cache data/smoke-search/cache/vitals
uv run --locked python scripts/parse-watch-export.py data/smoke-search/export.zip data/smoke-search/watch.json
uv run --locked python scripts/parse-watch-export.py data/smoke-search/export.zip data/smoke-search/watch-vitals.json --vitals
uv run --locked python scripts/search.py plan docs/search/synthetic.json --out data/smoke-search/run --cache data/smoke-search/cache
uv run --locked python scripts/search.py run data/smoke-search/run --jobs 2
uv run --locked python scripts/search.py final data/smoke-search/run --candidate replayed+none+podContext \
  --access-log data/smoke-search/holdout-access.jsonl

To also exercise pod fetching against a synthetic local HTTP server:

uv run --locked python scripts/smoke-pod.py data/smoke-inputs data/smoke-run/features.json data/pod-smoke-run

Synthetic fixtures are invented and deterministic. Use fresh directories to repeat the smoke run. Their scores demonstrate plumbing only, not model quality. CI runs these checks with synthetic inputs.

Knowledge updates

The knowledge/ bundle follows Google's OKF v0.2 specification. Concepts are agent-authored drafts unless actually reviewed; structure validation does not verify scientific claims. Update source provenance, generated metadata, directory indexes, and knowledge/log.md when changing concepts. Add verified only after a real review, naming the reviewer and scope.

Aggregate evidence still describes personal data. Review it before public release.

Import Watch movement

The importer follows iOS docs/watch-motion-format.md and Shared/WatchMotion/WatchMotionFile.swift on dev: watch-motion-v1, 50 Hz acceleration in g, 2-second bins (not per-second), jerk in g/s, variance in g², Unix milliseconds, IANA zone and its offset at capture start. Unknown versions, unsupported bin/rate/jerk settings, malformed columns, duplicate bins, invalid counts and inconsistent zones are refused. Optional raw samples remain under ignored data/; no health data belongs in fixtures or CI.

watch-motion-v2 adds per-bin meanX, meanY and meanZ; labels from v2 captures carry angleDeg, the van Hees arm angle, with no threshold applied yet. Bins wholly outside the recorded span (leftovers that older Watch builds appended from a previous recording) are trimmed and counted in trimmedBins. watch_motion.py also takes the iOS Export all file (watch-motion-all-*.json, an array of captures); a night that appears in it and in a single-night file is imported once. movement.py --watch-motion still pairs one single-night file with one archive.

uv run --locked python scripts/watch_motion.py data/watch/night1.json data/watch/night2.json --out data/watch/labels.json
uv run --locked python scripts/movement.py train \
  --watch-motion data/watch/night1.json=data/raw/night1.tar.gz \
  --watch-motion data/watch/night2.json=data/raw/night2.tar.gz \
  --side left --movement data/pod-movement.json --out data/watch-movement-run

Use --pod <address> instead of saved exports/movement to fetch with archive slack. Existing --session inputs can be mixed with Watch captures; specify --timezone for scripted night grouping. Watch captures use their recorded zone. Each window is grouped by its source Watch time and recorded zone (before clock alignment), so captures spanning local noon are split between nights. Windows straddling noon are excluded to avoid sharing samples across folds. All captures from a local noon-to-noon night stay in the same held-out fold; a single Watch night cannot produce a validation score. Overlapping windows are deduplicated and conflicting labels refused. The report compares the classifier with the existing per-minute pod movement score on identical windows.

Thresholds are fixed in watch_motion.py before observing results: at least 80 of the expected 100 samples per bin; moving when at least 5% exceed the exported 1 g/s jerk threshold; vigorous at 20%, otherwise light movement or still. These are provisional heuristics, not measured classifier accuracy. Missing or under-covered bins remain unknown, including when aggregating a window; they never become zero movement. Keep thresholds fixed for an evaluation; any change needs a new label version and fresh held-out evaluation.

Watch motion is a movement label only. Never use it as a sleep-stage feature when Watch stages are the target: the Watch's own stages use acceleration, so that would leak the label.

Both devices use wall time; NTP alone does not verify sensor alignment. When knock or turn events are available, pass --alignment to the importer or --watch-alignment to movement.py train for one capture. The JSON contract is {"version":"capture-alignment-v1","confirmation":"schedule","pairs":[{"sourceMs":...,"podMs":...},...]}. Use at least three matched events near each end, confirmed against the schedule or raw energy (confirmation: "energy-fit"). Ending knocks were sharper in the 2026-10-22 movement session: inspect those first, then check the opening group. The end-anchored linear fit records offset, drift and residual uncertainty; missing checks are explicitly reported as NTP-only and unverified.

Import iPhone snore sessions

The importer follows iOS docs/snore-capture.md and Services/Snore/: sleepypod-snore-capture-v1, units: "unix-ms", recorded IANA zone, iphone-microphone, and the known Apple classifier/hysteresis version. It validates the microphone format, classifier parameters, bounds, predicted event provenance, confidences, coverage, interruptions and knock timestamps. Export a stopped session; unfinished captures and unknown versions are refused. No audio is read, stored or required.

uv run --locked python scripts/snore_labels.py \
  --session data/snore/phone-night1.json --export data/raw/night1.tar.gz \
  --side left --out data/snore-night1
# Or --pod <address> in place of --export; fetch waits are not implicit.
uv run --locked python scripts/snore_labels.py \
  --reports data/snore-night1/report.json data/snore-night2/report.json \
  --out data/snore-exit

Fetch after finished frames have reached the archive (about 30–45 minutes), with one hour of end slack. Decoding and band energy reuse raw_piezo and piezo_bandwidth; source SHA-256 hashes and decode errors accompany each run. Three-knock schedules at both ends are matched to sharp piezo transients, ending first, with a residual-energy timing fit; ambiguous matches refuse sync. Use --alignment data/snore/matches.json for independently confirmed pairs as above. Offsets beyond the automatic ±5 s search need explicit matched pairs. Without matched knocks the timeline uses wall time, says unsynchronized, and is excluded from event-level evaluation. A full night on real hardware is still needed to validate this detector and uncertainty estimate.

Reports have kind: "snore-labels" and register in the catalog's Snore view. Phone events overlay the raw night band-energy timeline with confidence tooltips, knock markers, phone coverage, and the exit-test result/per-night AUCs. Green means recording with no detection, gray means not recording, gold means a phone prediction. A prediction is not verified snoring or a diagnosis.

Milestone-2 exit test and the current export limit

snore_exit.py preregisters section 4 of docs/overnight-capture-implementation-plan.md: one 3 s log-power score in the lower passing band on the preregistered occupant channel, normalized by that night's median occupied pump-off windows; single occupant only; negatives below half the calibrated threshold within 30 minutes of positives; movement/pump exclusions with a 10 s guard; median scores per bout (positive windows less than 30 s apart merge). Negative windows also merge into control bouts to avoid window pseudoreplication. At least five eligible nights, at least three with five positive bouts, are required. Report per-night AUCs and pooled AUC with a 95% interval from 2,000 night-resampled bootstrap draws (seed 42). Pass is AUC ≥0.7 and lower bound >0.5; fail is upper bound <0.7; otherwise inconclusive. Duplicate nights and mixed calibration/threshold/band contracts are refused.

Current iOS v1 saves merged events and peak confidence, not all detector windows. Therefore no event does not establish confidence below threshold/2. These imports remain inconclusive for the exit test; the importer never invents negative confidences. Real overnight exports alone will not fix this limitation: a calibration session and a future per-window confidence export are needed.

For synthetic checks or a separately produced calibrated window export, optional --window-evidence data/snore/windows.json accepts snore-window-evidence-v1: captureId, registeredBeforeCaptureMs, calibrationId, threshold, passingBands (ordered 16-30Hz, 60-100Hz, or one passing band), primaryChannel, occupants, windows (startMs, endMs, confidence in the phone clock), and occupied, pumpOff, pumpActive, movement arrays of {startMs,endMs} in the pod clock. Unknown pump/occupancy coverage is excluded. This companion contract is not claimed to be an existing iOS file format. The code computes scores directly from raw piezo; it never accepts supplied scores. Missing coverage, evidence or synchronization yields inconclusive.

All implementation tests use deterministic synthetic files. Still needed on real files: overnight Watch coverage and heuristic quality, phone calibration, clock-match robustness/drift, recording interruption behavior, and at least the preregistered night/bout minimum. No new empirical findings are claimed here.

About

Biometric model training knowledge, evaluation protocols, and aggregate research evidence in Open Knowledge Format.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages