From f4420c00ac126b80b26db7c90152ac743c0d7921 Mon Sep 17 00:00:00 2001 From: Uriah Finkel Date: Tue, 11 Aug 2026 17:50:20 +0300 Subject: [PATCH 01/17] document curve API compatibility --- user_guide/02-curve-api-compatibility.qmd | 115 ++++++++++++++++++++++ 1 file changed, 115 insertions(+) create mode 100644 user_guide/02-curve-api-compatibility.qmd diff --git a/user_guide/02-curve-api-compatibility.qmd b/user_guide/02-curve-api-compatibility.qmd new file mode 100644 index 00000000..818fbb40 --- /dev/null +++ b/user_guide/02-curve-api-compatibility.qmd @@ -0,0 +1,115 @@ +--- +title: "Curve API Compatibility" +guide-section: "Using rtichoke" +--- + +`rtichoke` intentionally exposes parallel function families for discrimination, calibration, and decision-curve analysis. Their interfaces are similar, but they are **not interchangeable in every edge case**. + +This page makes those differences explicit so that users — and coding agents reading `llms-full.txt` — can choose the right call without experimentally probing each function. + +::: {.callout-important} +## Calibration is the main exception + +The current calibration implementation has stricter multi-population behavior than ROC, precision-recall, and decision curves. In particular, do not assume that an input pattern accepted by a discrimination curve is automatically accepted by a calibration curve. +::: + +## Capability matrix + +| Function family | Multiple named populations | Unequal population sizes with `dict`/`dict` inputs | Time-dependent heuristic default | `fixed_time_horizons` | +|---|---|---|---|---| +| ROC | Supported | Supported | `adjusted` / `adjusted_as_negative` | Use floating-point values | +| Precision-recall | Supported | Supported | `adjusted` / `adjusted_as_negative` | Use floating-point values | +| Decision curve | Supported | Supported | `adjusted` / `adjusted_as_negative` | Use floating-point values | +| Calibration | Supported when inputs satisfy calibration alignment requirements | **Currently restricted**; differently sized named populations can raise a length-mismatch error | **No default on `create_calibration_curve_times()`**; pass explicitly | Use floating-point values | + +The matrix describes the current public behavior. It does **not** claim that every asymmetry is a permanent design decision. + +## Calibration with named populations + +This symmetric example is supported: + +```python +import numpy as np +import rtichoke as rk + +probs = { + "Train": np.array([0.10, 0.90, 0.20, 0.80, 0.30, 0.70]), + "Test": np.array([0.15, 0.85, 0.25, 0.75, 0.35, 0.65]), +} +reals = { + "Train": np.array([0, 1, 0, 1, 0, 1]), + "Test": np.array([0, 1, 0, 1, 0, 0]), +} + +fig = rk.create_calibration_curve(probs=probs, reals=reals) +``` + +At present, differently sized named populations are not a drop-in equivalent: + +```python +probs = { + "Train": np.array([0.10, 0.90, 0.20, 0.80, 0.30, 0.70]), + "Test": np.array([0.15, 0.85, 0.25, 0.75]), +} +reals = { + "Train": np.array([0, 1, 0, 1, 0, 1]), + "Test": np.array([0, 1, 0, 0]), +} + +# This input shape can raise a length-mismatch error for calibration. +rk.create_calibration_curve(probs=probs, reals=reals) +``` + +If you need to compare calibration across populations of different sizes, create each calibration curve from its own aligned population rather than assuming the ROC/PR multi-population behavior applies to calibration. See [Common Errors & Fixes](common-errors.html) for the failure pattern. + +::: {.callout-note} +The unequal-population restriction is documented as **current behavior**, not as a statistical recommendation. It should be investigated before adding an escape hatch such as `strict=False` or otherwise changing calibration semantics. +::: + +## Time-dependent calibration heuristics + +`create_calibration_curve_times()` differs from its ROC, precision-recall, and decision-curve siblings in two important ways: + +1. `heuristics_sets` is currently required rather than defaulted. +2. The sibling default `censoring_heuristic="adjusted"` is not a safe value to copy blindly into calibration. In the current calibration path, that choice can remove every requested horizon and end in `No data remaining after applying heuristics and time horizons.` + +Pass the calibration heuristic explicitly. For the currently working exclusion-based path: + +```python +heuristics_sets = [ + { + "censoring_heuristic": "excluded", + "competing_heuristic": "adjusted_as_negative", + } +] +``` + +Then call: + +```python +fig = rk.create_calibration_curve_times( + probs=probs, + reals=reals, + times=times, + fixed_time_horizons=[3.0, 6.0, 9.0], + heuristics_sets=heuristics_sets, +) +``` + +## Time horizons: prefer floats + +Use floating-point horizons such as `[3.0, 6.0, 9.0]`, not `[3, 6, 9]`. The current implementation can otherwise expose an internal Polars join-key datatype mismatch (`i64` versus `f64`) rather than a rtichoke-specific validation message. + +```python +# Prefer +fixed_time_horizons = [3.0, 6.0, 9.0] + +# Avoid for now +fixed_time_horizons = [3, 6, 9] +``` + +This is a usability limitation rather than a conceptual requirement; normalizing numeric horizons internally is a good candidate for a small future code fix. + +## Related functions + +When moving between curve families, compare the API reference for `create_calibration_curve()`, `create_calibration_curve_times()`, `create_roc_curve_times()`, `create_precision_recall_curve_times()`, and `create_decision_curve_times()` rather than assuming their defaults and accepted input shapes are identical. From 1c65c9475994ce67fa73079dbe7ba53978934e6d Mon Sep 17 00:00:00 2001 From: Uriah Finkel Date: Tue, 11 Aug 2026 17:50:34 +0300 Subject: [PATCH 02/17] add common errors guide --- user_guide/03-common-errors.qmd | 90 +++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 user_guide/03-common-errors.qmd diff --git a/user_guide/03-common-errors.qmd b/user_guide/03-common-errors.qmd new file mode 100644 index 00000000..db46dab2 --- /dev/null +++ b/user_guide/03-common-errors.qmd @@ -0,0 +1,90 @@ +--- +title: "Common Errors & Fixes" +guide-section: "Using rtichoke" +--- + +This page is deliberately keyed by **literal error text**. If an rtichoke call fails, search this page for a distinctive part of the exception before tracing into the implementation. + +## `probs['...'] length=... does not match sum of population sizes=...` + +### Where this commonly appears + +Calibration calls using dictionaries for both predictions and outcomes, especially when the named populations have different numbers of observations. + +### Why it happens + +The current calibration path has stricter alignment requirements than ROC, precision-recall, and decision curves. An input pattern that works for those sibling functions can therefore fail for `create_calibration_curve()` or `create_calibration_curve_times()`. + +### Fix + +First verify that every prediction vector corresponds to the intended outcome population. If you are deliberately comparing differently sized populations, do not assume calibration supports the same dict/dict overlay pattern as ROC or PR. Create the calibration result for each aligned population separately. + +See [Curve API Compatibility](curve-api-compatibility.html) for the family-by-family comparison. + +## `No data remaining after applying heuristics and time horizons.` + +### Where this commonly appears + +`create_calibration_curve_times()`. + +### Why it happens + +One current failure mode is passing the heuristic combination used as the default by ROC/PR/decision time-dependent functions. In calibration, `censoring_heuristic="adjusted"` can skip every horizon, leaving no data to plot. The final exception does not currently explain which heuristic caused the removal. + +### Fix + +Pass the calibration heuristic explicitly. For the currently working exclusion-based path: + +```python +heuristics_sets = [ + { + "censoring_heuristic": "excluded", + "competing_heuristic": "adjusted_as_negative", + } +] +``` + +Do not infer calibration defaults from `create_roc_curve_times()`, `create_precision_recall_curve_times()`, or `create_decision_curve_times()`. + +## Polars join-key datatype mismatch: `i64` versus `f64` + +### Where this commonly appears + +Time-dependent functions when integer values are supplied in `fixed_time_horizons`, for example: + +```python +fixed_time_horizons=[3, 6, 9] +``` + +### Why it happens + +The current internal time-horizon data can be floating point, while integer literals create integer-typed join keys. The resulting Polars error leaks an implementation detail instead of explaining the rtichoke input requirement. + +### Fix + +Use floating-point horizons: + +```python +fixed_time_horizons=[3.0, 6.0, 9.0] +``` + +Internal numeric normalization is a candidate for a future defensive code fix. + +## Why is `heuristics_sets` missing? + +If Python reports that `create_calibration_curve_times()` is missing the required `heuristics_sets` argument, that is currently expected API behavior. Unlike the ROC, precision-recall, and decision-curve `_times` functions, calibration does not currently provide a default. + +Pass it explicitly rather than copying a sibling default: + +```python +heuristics_sets = [ + { + "censoring_heuristic": "excluded", + "competing_heuristic": "adjusted_as_negative", + } +] +``` + +## Still stuck? + +Check [Curve API Compatibility](curve-api-compatibility.html) first. The most important debugging rule is that similarly named rtichoke curve functions can still differ in accepted population shapes, heuristic defaults, and time-horizon handling. From 5402e1207c35004bcc4fd8f1197de8c9f31d2b6f Mon Sep 17 00:00:00 2001 From: Uriah Finkel Date: Tue, 11 Aug 2026 17:50:49 +0300 Subject: [PATCH 03/17] surface calibration input constraints --- user_guide/00-getting-started.qmd | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/user_guide/00-getting-started.qmd b/user_guide/00-getting-started.qmd index b36ed56f..d0f6d76c 100644 --- a/user_guide/00-getting-started.qmd +++ b/user_guide/00-getting-started.qmd @@ -37,6 +37,10 @@ Most `rtichoke` plotting functions use two dictionaries: - `probs`: model predictions, keyed by model or population name. - `reals`: observed outcomes, keyed by population name. +::: {.callout-tip} +Similar curve families do not have identical edge-case behavior. Before combining differently sized populations or moving code between ROC/PR/decision and calibration, see [Curve API Compatibility](curve-api-compatibility.html). If a call fails, [Common Errors & Fixes](common-errors.html) is searchable by literal exception text. +::: + ## Single model ```python @@ -99,4 +103,8 @@ fig = rk.create_calibration_curve( fig.show() ``` -From here, use the API Reference for the full set of curve types, parameters, and time-to-event variants. The [Naming Conventions](user-guide/naming-conventions.html) guide explains how the exported function families fit together. +::: {.callout-warning} +The calibration example above deliberately uses equally sized Train and Test populations. The current calibration implementation can reject differently sized populations even when the analogous ROC, precision-recall, or decision-curve call works. See [Curve API Compatibility](curve-api-compatibility.html) for the documented constraint and an intentional failure example. +::: + +From here, use the API Reference for the full set of curve types, parameters, and time-to-event variants. The [Naming Conventions](naming-conventions.html) guide explains how the exported function families fit together, while [Curve API Compatibility](curve-api-compatibility.html) documents where those families intentionally or currently differ. From f87d82daa7676739ddedc1416acce8ce25ec22f3 Mon Sep 17 00:00:00 2001 From: Uriah Finkel Date: Tue, 11 Aug 2026 17:51:12 +0300 Subject: [PATCH 04/17] enable release-backed changelog --- great-docs.yml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/great-docs.yml b/great-docs.yml index 8ebc6ee2..ce8c7033 100644 --- a/great-docs.yml +++ b/great-docs.yml @@ -5,6 +5,10 @@ user_guide: user_guide homepage: user_guide site_url: https://uriahf.github.io/rtichoke_python/ +changelog: + enabled: true + max_releases: 50 + include_in_header: - text: |