HealthHack ML

Lightweight, false-alarm-constrained four-state classifier for the HealthHack Sydney dataset. It predicts normal (0), warning (1), crisis (2), and dead (3) for every five-second observation.

Approach

The data audit found 4,186 non-overlapping encounters across the splits, each containing exactly 720 observations. The supplied labelled test_data.csv is therefore used as an encounter-level validation set; a random row split would leak observations from the same episode.

The pipeline uses two regularized LightGBM classifiers over the same 75 compact features:

• a four-class model estimates Normal, Warning, Crisis, and Dead severity;
• a dedicated binary gate learns Normal versus non-Normal independently.

The features combine:

• current vital signs and encounter progress;
• change from the first two-minute patient baseline;
• one- and five-minute trailing deviations, volatility, and changes;
• centred two- and five-minute context to distinguish sustained deterioration from brief abnormal readings;
• age, BMI, pain score, and low-cardinality patient context;
• summary statistics from each encounter's ECG.

This two-model cascade remains light enough for an Intel Ultra 5. The binary gate prevents Warning/Crisis discrimination from making the alarm/no-alarm boundary unnecessarily aggressive. Early stopping selects each tree count on the encounter-disjoint labelled validation split.

The prediction policy smooths probabilities within each encounter, combines the binary gate with the multiclass alarm probability, and requires the result to clear a validation-tuned threshold. Policy selection maximizes the documented reward subject to a hard 1% Normal false-alarm ceiling. It also requires each alternating encounter half to remain below 1.25%, reducing threshold-selection variance. Label 3 is handled separately without inventing an undocumented cost, using the observed fact that Dead is an absorbing state.

Reproduce

python -m venv .venv
.venv/bin/python -m pip install -r requirements.txt
.venv/bin/python -m healthhack_ml.train

Regenerate predictions from the saved model without retraining:

.venv/bin/python -m healthhack_ml.predict

Run the checks:

.venv/bin/python -m unittest discover -s tests -v

Validation result

Compared with the previous aggressive LightGBM policy:

Validation metric	Previous policy	Conservative policy
Normal false alarms	13,416 (4.10%)	3,248 (0.99%)
Normal recall	0.9590	0.9901
Accuracy	0.9311	0.9359
Macro-F1	0.9060	0.9112
Warning recall	0.8123	0.7327
Crisis recall	0.9304	0.8852
Documented 0–2 reward	152,457	122,945

False alarms fall 75.8% and accuracy improves slightly. This is deliberately a safety-first operating point: it gives up Warning/Crisis recall and reward to avoid nuisance escalation. On holdout, the new CSV emits 18,520 fewer Warning or Crisis rows than the existing submission (a 15.9% reduction).

Detailed metrics are in outputs/validation_metrics_conservative.json. Use --max-false-alarm-rate when retraining to choose a different operating point.

Generated artifacts

• models/healthhack_lgbm_conservative.joblib — final cascade fitted on all labelled encounters;
• outputs/submission_false_alarm_reduced.csv — 452,880 holdout predictions in sample format;
• outputs/validation_metrics_conservative.json — validation, policy, and timing audit.

Dataset files under data/ are read-only inputs and are never modified.