Skip to content

docs: record E2 and the model-graph plan's implementation status - #39

Merged
ng merged 1 commit into
mainfrom
docs/e2-and-plan-status
Nov 16, 2026
Merged

ng merged 1 commit into
mainfrom
docs/e2-and-plan-status

Conversation

@ng

@ng ng commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • New knowledge finding: E2, podContext trained on vitals replayed through one pinned engine and profile versus the mixed-version live rows. Walk-forward macro F1 ties at 0.524 (+0.000, night bootstrap [−0.036, +0.037], 9/16 nights better). Replayed breathing tracks the Watch more closely (MAE 1.11 vs 1.80/min). Nothing is exported.
  • The plan gains an implementation status table: what is merged, what is open for review in core and iOS, what is gated. The C5 gate is updated to core #928's thresholds, recalibrated on 17 archived nights.

Test plan

  • uv run --locked python scripts/validate_knowledge.py (22 concepts)
  • E2 reproduced from docs/search/e2-replayed-vs-live.json, with the archive pod id and the left side's saved profile filled in before the first run: search.py plan/run/report

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Added a report comparing replayed and live measurement results, including performance and agreement with Watch measurements. The comparison found no performance difference and did not result in a model being exported or activated.
    • Updated evaluation guidance with revised replay acceptance criteria and results from recalibration across archived nights.
    • Added links to the new evaluation in the documentation index and knowledge log.

…t step starts from what landed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Explain Change →

Caution

Review failed

The pull request is closed.

Walkthrough

The changes update the model-graph plan with implementation status and revised C5 replay criteria. They also document a staging comparison of live and replayed vitals, report its results and limitations, and add evaluation index and log entries.

Changes: Replay evaluation and plan records

Layer / File(s) Summary
Implementation status and replay criteria
docs/plans/model-graph-plan.md
The plan records implementation status and findings. The C5 criteria specify replay comparison requirements, updated thresholds, and recalibration results from 17 archived nights.
Staging evaluation report and index
knowledge/evaluation/e2-replayed-vitals-2026-11-16.md, knowledge/evaluation/index.md, knowledge/log.md
The draft report gives both candidates a walk-forward macro F1 of 0.524 across 24 night groups and classifies the difference as a tie. It also records Watch comparisons and evaluation limitations. The index and log summarize the evaluation. The report states that no model was exported, uploaded, or activated.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Possibly related PRs

  • sleepypod/training#20: Adds Watch vitals extraction and a comparison tool; the evaluation report compares results with Watch measurements.
Merge Risk: Minimal · up to e4a8f

This documentation-only change records an evaluation tie and the plan's implementation status. It has no runtime impact and is ready to merge.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@ng
ng merged commit 84ce788 into main Nov 16, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant