Skip to content

Fix time-dependent interventions avoided calculation (#412) - #413

Merged
uriahf merged 3 commits into
mainfrom
fix-time-interventions-avoided-412-3099309441451949724
Sep 6, 2026
Merged

Fix time-dependent interventions avoided calculation (#412)#413
uriahf merged 3 commits into
mainfrom
fix-time-interventions-avoided-412-3099309441451949724

Conversation

@uriahf

@uriahf uriahf commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Implements exact dcurves compatibility for time-dependent interventions avoided by deriving model interventions avoided as an exact transformation of conventional net benefit versus treat-all net benefit using cutoff-zero population event risk.

Fixes #412


PR created automatically by Jules for task 3099309441451949724 started by @uriahf

Derive time-dependent interventions avoided directly from model and
treat-all net benefit using population event risk from cutoff-zero.

Fixes #412

Co-authored-by: uriahf <11351434+uriahf@users.noreply.github.com>
@google-labs-jules

Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-06 11:01 UTC

Derive time-dependent interventions avoided directly from model and
treat-all net benefit using population event risk from cutoff-zero.

Fixes #412

Co-authored-by: uriahf <11351434+uriahf@users.noreply.github.com>
@uriahf

uriahf commented Sep 6, 2026

Copy link
Copy Markdown
Owner Author

PR #413 has the right high-level direction, but please do not merge yet. There is one statistical blocker plus the current lint failure.

The important statistical issue is that _recalculate_interventions_avoided_times() currently derives the supposed full-population event risk from:

chosen_cutoff == 0real_positives / n.

That is not always a pooled population KM/AJ estimate.

prepare_performance_data_times() uses the pooled_by_cutoff path, and that path still fits separate AJ/KM estimates in the predicted-positive and predicted-negative groups. At threshold 0 this collapses to the full population only if every prediction is strictly > 0.

Predictions exactly equal to 0 are valid. They remain in the predicted-negative subgroup (> cutoff versus <= cutoff), so real_positives = TP + FN becomes a weighted combination of subgroup KM/AJ estimates again. Under censoring this can differ from the actual pooled population KM/AJ risk — recreating the exact non-collapsibility problem from issue #412.

Please reproduce this explicitly using Fixture B:

  • change the censored subject's probability from 0.4 to 0.0;
  • keep outcome/time/horizon/threshold otherwise identical.

The true pooled population KM risk at t=10 must remain 0.4, since changing predictions cannot change population event risk.

The current cutoff-zero reconstruction should instead expose a different value (approximately 0.375), which would produce IA=25 rather than the dcurves-compatible IA=20 at threshold 0.5.

Please fix the architecture so NB_all uses a genuine pooled full-population KM/AJ event-risk estimate, independent of prediction values and threshold grouping.

Prefer reusing an existing true population-level AJ/KM calculation if one already exists. Do not introduce a second approximate reconstruction from threshold rows.

Then add focused regression coverage proving:

  1. population event risk is invariant to changing predictions, including exact 0 and 1 predictions;
  2. two models sharing the same reals/times population get exactly the same population event risk even if their predictions differ;
  3. the exact-zero Fixture B still gives the dcurves-compatible IA=20;
  4. the existing fixtures A-D and algebraic NB↔IA invariant remain green;
  5. multiple populations/horizons/heuristics remain isolated.

Also correct the existing test_multi_identity_isolation() description or fixture: it currently claims to test multiple models sharing one population, but keyed reals/times for m1/m2 actually represent different populations. Add a genuine shared-population test using shared reals and shared times.

Finally remove the unused pytest import currently failing Ruff.

Run:

  • ruff check
  • formatting
  • type checks
  • full tests
  • build/package checks

Push to the existing PR #413 branch and inspect exact-head CI. Stop before merge.

Do not broaden scope beyond issue #412.

…oled population risk (#412)

Compute genuine pooled full-population KM/AJ event risk estimate using
`predict_aj_estimates` on the full population dataset, independent of prediction
values and threshold grouping.

Fixes #412

Co-authored-by: uriahf <11351434+uriahf@users.noreply.github.com>
@uriahf
uriahf merged commit 93fe1e8 into main Sep 6, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

time-dependent: Model interventions avoided diverges from dcurves due to subgroup KM non-collapsibility under censoring

1 participant