Skip to content

Add a clinician demo app for the MIMIC-IV forecasting model - #250

Open
amrit110 wants to merge 6 commits into
mainfrom
clinician-demo
Open

amrit110 wants to merge 6 commits into
mainfrom
clinician-demo

Conversation

@amrit110

@amrit110 amrit110 commented Sep 11, 2026

Copy link
Copy Markdown
Member

Summary

A clinician-facing web app for the MIMIC-IV flagship model (full_run_v10). It replays one admission and shows, moment by moment:

  • the risk of ICU admission, vasopressors, AKI, Sepsis-3 and death within 8, 24 and 72 h, against an alert line;
  • when that alert came on, and how many hours of warning it gave;
  • what the model thinks is going on (29 concept beliefs) and what it expects next;
  • What if: change a recent value (for example blood pressure) and re-score;
  • Why: which recorded items the forecast leans on (occlusion);
  • How good is it?: a scorecard with CIs against the tuned GBM, computed from the banked evaluation files.

Runbook: docs/clinician_demo.md.

Commits

  1. Shared single-patient streaming loop and per-patient helpers (odyssey/). The loop copied in case_study and counterfactual now lives once in odyssey/inference/patient_stream.py. Also adds ordered_sequence_rows, load_meds_subject, code_metadata.load_code_descriptions, a target-agnostic occlude_codes, and moves concept_display_name into concepts. Behaviour is unchanged: the existing tests pass unmodified.
  2. The app (apps/clinician_demo/), with its tests (tests/apps/) and the runbook. pyproject.toml only adds apps to coverage.

Design notes

  • Standard library HTTP only. The GPU host's environment is pinned and has no web framework.
  • Locked down. Loopback bind only, Host allowlist, a required X-Odyssey-Demo header, no CORS, Cache-Control: no-store, same-origin CSP, static files from an allowlisted directory with path containment.
  • Two data modes. credentialed serves held-out MIMIC-IV and is for PhysioNet-credentialed viewers only. open serves the MIMIC-IV Clinical Database Demo. 90 of those 100 patients were in the model's training split, and every chart says so.
  • Honest by construction.
    • The gallery always shows misses and false alarms with their rates.
    • Lead time counts from the alert episode still on at the event, not from the first touch of the line.
    • Risk is hidden after onset.
    • Readmission, steering, label overrides and rollouts are never shown, and a test enforces those imports.
  • Frontend. Vanilla ES modules, no build step, no third-party JS.

Test plan

  • Full suite: 1,908 passed, 7 skipped. Ruff and mypy clean over odyssey scripts apps.
  • 200+ new CPU tests on synthetic data: every module, and server security over a real socket. Spies prove the run's chunk_size reaches every model call. The trace equals score_record_at at every bundle end.
  • Self-check on the VM against the real checkpoint, in both modes. The demo's 24 h risk matches the banked landmark scores with a median gap of 7e-6 (n=425; max 0.27 near chunk boundaries, because the banked run packed 64 lanes and the demo streams one).
  • Every screen driven in Chrome on open data: gallery, replay and play, what-if, Why, scorecard. No console errors.
  • Light theme and narrow screens are not yet checked.

CI fixes (commits 3 and 4)

  1. Fix CI (ab3368b):
    • Static files are served from a map built at startup, so no path is ever built from request data. This clears the three CodeQL path-injection alerts.
    • thr is renamed to threshold, since the typos hook read it as "the".
    • Hooks never rewrite scripts/gemini/out/.
    • mypy and typos skip the two cohort producers committed verbatim. pyproject.toml excludes them from mypy, but pre-commit passes files by name, which bypasses that exclude. This is also why main's code check has failed since 0de877f. Merging this fixes main too.
  2. Revert pre-commit.ci's autofix (35b5e9a). The bot added a trailing newline to 12 verbatim GEMINI exports on this branch. They are byte-identical to main again.

Caveats

  • The what-if edit method was validated on an earlier model (v8), not v10. The UI says so.
  • Deployed on the VM from a tarball in ~/odyssey_demo, not from main. After merge it can move to the normal git reset path.

🤖 Generated with Claude Code

https://claude.ai/code/session_012zuuVqnBfTsozfaj5F4y3r

amrit110 and others added 3 commits September 11, 2026 07:32
Three readers streamed one patient through the model with their own
copy of the same PackedLaneSampler loop. It now lives once in
odyssey/inference/patient_stream.py (stream_patient, risk_within), and
case_study and counterfactual use it. Behaviour is unchanged: the
existing case-study and counterfactual tests pass unmodified.

Also added, each with tests:
- extract_patient_case reports risk at 8/24/72 h (event_risk_by_horizon);
  event_risk_24h is kept for the report that reads it.
- ordered_sequence_rows: the exact row order build_patient_sequence
  tokenizes, so a position maps back to its raw row.
- load_meds_subject: one subject from one shard, filter pushed into the scan.
- code_metadata.load_code_descriptions, split out of steering so readers
  can name codes without importing steering.
- occlude_codes: target-agnostic occlusion with a progress callback;
  occlusion_attribution is now a thin wrapper, plus
  event_occlusion_attribution for an event's risk.
- concept_display_name moves from make_readout_table into concepts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012zuuVqnBfTsozfaj5F4y3r
A web app that replays one admission and shows, moment by moment, the
model's risk of ICU admission, vasopressors, AKI, Sepsis-3 and death
against an alert line, what it thinks is going on, what it expects
next, what-if value edits, occlusion evidence, and a scorecard against
the tuned GBM. Runbook: docs/clinician_demo.md.

- Standard library HTTP only (the GPU host's environment is pinned);
  loopback bind, Host allowlist, custom API header, no CORS, no-store,
  same-origin CSP, static path containment.
- Two data modes: credentialed (held-out MIMIC-IV) and open (MIMIC-IV
  Clinical Database Demo). Every chart says whether the model trained
  on that patient.
- Honest by construction: the gallery shows misses and false alarms with
  their rates; lead time counts from the alert episode still on at the
  event; risk is hidden after onset; readmission, steering, label
  overrides and rollouts are never shown (a test enforces the imports).
- Vanilla ES modules, no build step, no third-party JS.
- 200+ CPU tests on synthetic data: every module, server security over
  a real socket, and spies proving the run's chunk_size reaches every
  model call.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012zuuVqnBfTsozfaj5F4y3r
Comment thread apps/clinician_demo/server.py Fixed
Comment thread apps/clinician_demo/server.py Fixed
Comment thread apps/clinician_demo/server.py Fixed
@codecov

codecov Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.08%. Comparing base (31144a5) to head (c401524).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #250      +/-   ##
==========================================
+ Coverage   89.98%   90.08%   +0.10%     
==========================================
  Files          69       71       +2     
  Lines       10771    10835      +64     
==========================================
+ Hits         9692     9761      +69     
+ Misses       1079     1074       -5     
Files with missing lines Coverage Δ
odyssey/data/code_metadata.py 100.00% <100.00%> (ø)
odyssey/data/concepts.py 98.41% <100.00%> (+0.01%) ⬆️
odyssey/data/sequences.py 97.46% <100.00%> (+0.15%) ⬆️
odyssey/inference/case_study.py 83.42% <100.00%> (+0.97%) ⬆️
odyssey/inference/concept_edit_attribution.py 98.88% <100.00%> (+0.10%) ⬆️
odyssey/inference/counterfactual.py 83.33% <100.00%> (-0.14%) ⬇️
odyssey/inference/patient_stream.py 100.00% <100.00%> (ø)
odyssey/inference/steering.py 96.06% <100.00%> (+0.31%) ⬆️
odyssey/training/data.py 98.63% <100.00%> (+0.04%) ⬆️

Impacted file tree graph

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

amrit110 and others added 3 commits September 11, 2026 08:00
- server: static files are served from a map built once at startup
  (static_files); a request is a dictionary lookup, so no filesystem
  path is ever built from request data. Clears the three CodeQL
  "uncontrolled data used in path expression" alerts. Symlinks out of
  the root and non-allowlisted extensions are still excluded, now at
  map-build time; tests cover both.
- showcase: rename the `_thr` column and `thr` test argument, which the
  typos hook reads as "the".
- pre-commit: no hook rewrites scripts/gemini/out/ (pre-commit.ci's
  end-of-file fixer had edited 12 verbatim GEMINI exports on this PR),
  and mypy/typos skip the two cohort producers committed verbatim as
  they ran. pyproject's mypy exclude never applied to them because
  pre-commit passes files by name; this is why main's code check has
  failed since 0de877f.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012zuuVqnBfTsozfaj5F4y3r
This reverts commit 8348226, which
added a trailing newline to 12 files under scripts/gemini/out/evals/.
Those files are GEMINI's exported reports, kept byte-for-byte; the
previous commit stops the hooks from touching them again.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012zuuVqnBfTsozfaj5F4y3r
Front end only; the API and Python are unchanged.

- Light theme by default, dark via a footer toggle. One accent colour;
  event colours only on lines and dots.
- Replay: patient one-liner, sticky "now" bar (play, scrub, clock time),
  five risk tiles with a status word (Low / Watch / Alert on / Happened)
  and "n× the average patient", then the chart, what happened in the
  stay, what the model believes now, and the what-if / evidence tools.
- The chart fades everything after "now".
- Alert stories are composed client-side in clock time ("Began Day 1,
  22:53. The alert had been on since Day 1, 01:16: 22 h of warning").
- The 29-row concept strip, alert-line statistics and next-token list
  sit behind disclosures; model name and checkpoint move to the footer.
- Gallery cards carry one key line; the full admission list is a
  collapsed per-patient list with a note on how many were in training.
- Scorecard cells show one number per model, intervals on hover.
- Fix: class display rules overrode the browser's [hidden] rule, so the
  search box showed in open mode and "Alert on now" showed on every row.
- Runbook: static files can be copied over a running deployment.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QebToqtFMVakRkx3PhF3ZX
amrit110 added a commit that referenced this pull request Sep 14, 2026
- check-docstring-first read the module-level attribute docstrings in the
  new target module and probe script as second module docstrings; they
  are comments now.
- pre-commit.ci's autofix had rewritten 12 scripts/gemini/out/ JSONs
  (trailing newlines) on this branch, the same way it did on PR #250.
  This carries PR #250's hook config over: scripts/gemini/out/ is
  excluded from every hook, and mypy/typos skip the two cohort producers
  committed verbatim, which is also what has been failing main's code
  check. The 12 exports are byte-identical to main again.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01StsjDmueuDqVNNuVoCEF1P

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants