Document curve API constraints and troubleshooting

great-docs.yml

@@ -5,6 +5,10 @@ user_guide: user_guide
 homepage: user_guide
 site_url: https://uriahf.github.io/rtichoke_python/
 
+changelog:
+  enabled: true
+  max_releases: 50
+
 include_in_header:
   - text: |
       <script>
@@ -63,7 +67,7 @@ reference:
       - plot_lift_curve
 
   - title: Calibration
-    desc: Calibration visualizations for classification and time-to-event models.
+    desc: Calibration visualizations for classification and time-to-event models. See Curve API Compatibility for time-dependent heuristic and horizon differences.
     contents:
       - create_calibration_curve
       - create_calibration_curve_times

skills/rtichoke/SKILL.md

@@ -0,0 +1,49 @@
+# rtichoke
+
+Use this skill when writing or debugging Python code that evaluates predictive-model performance with `rtichoke`.
+
+## Start here
+
+- Use the generated API reference for signatures and parameter details.
+- Use `llms-full.txt` for the complete API plus user-guide content.
+- Read **Curve API Compatibility** before assuming that all curve families accept identical time-dependent heuristics.
+- Search **Common Errors & Fixes** by literal exception text before source-diving.
+
+## Function families
+
+The main exported families include:
+
+- ROC: `create_roc_curve()`, `create_roc_curve_times()`
+- Precision-recall: `create_precision_recall_curve()`, `create_precision_recall_curve_times()`
+- Gains: `create_gains_curve()`, `create_gains_curve_times()`
+- Lift: `create_lift_curve()`, `create_lift_curve_times()`
+- Calibration: `create_calibration_curve()`, `create_calibration_curve_times()`
+- Decision curve: `create_decision_curve()`, `create_decision_curve_times()`
+
+Similar names do not guarantee identical edge-case behavior.
+
+## Shared input patterns
+
+- Named populations such as Train and Test can be represented by dictionaries. With dictionary-valued outcomes, keys are paired population-by-population and lengths must match within each population; populations themselves may have different sample sizes.
+- For time-dependent calls, dictionary-valued `times` follows the same population alignment.
+- A censoring heuristic affects estimates only when censored observations are present. A competing-event heuristic affects estimates only when competing events are present. Function-specific validation rules still apply independently of whether a heuristic would change the estimates.
+- `fixed_time_horizons` accepts numeric values. Integer horizons such as `[3, 6, 9]` are normalized to floats at the shared time-dependent processing boundary.
+
+## Calibration gotchas
+
+1. `create_calibration_curve_times()` currently requires `heuristics_sets`; it does not inherit the default used by ROC/PR/Gains/Lift/decision `_times` functions.
+2. Time-dependent calibration rejects `censoring_heuristic="adjusted"` and `competing_heuristic="adjusted_as_censored"` with an actionable `Unsupported calibration heuristics` error. A supported exclusion-based path uses `censoring_heuristic="excluded"` with `competing_heuristic="adjusted_as_negative"`.
+
+## Debugging rule
+
+When a call that works for another time-dependent curve family fails for calibration, first check whether the failure concerns:
+
+- mismatched population keys or within-population lengths,
+- `heuristics_sets`,
+- or an unsupported calibration heuristic.
+
+## Resources
+
+- Documentation site: https://uriahf.github.io/rtichoke_python/
+- Full machine-readable documentation: https://uriahf.github.io/rtichoke_python/llms-full.txt
+- Source repository: https://github.com/uriahf/rtichoke_python

user_guide/00-getting-started.qmd

@@ -37,6 +37,10 @@ Most `rtichoke` plotting functions use two dictionaries:
 - `probs`: model predictions, keyed by model or population name.
 - `reals`: observed outcomes, keyed by population name.
 
+::: {.callout-tip}
+Similar curve families can still differ in defaults and time-dependent handling. See [Curve API Compatibility](curve-api-compatibility.html), and if a call fails, search [Common Errors & Fixes](common-errors.html) by literal exception text.
+:::
+
 ## Single model
 
 ```python
@@ -79,16 +83,16 @@ fig.show()
 
 ## Compare populations
 
-To compare a model across populations, provide matching keys in `probs` and `reals`.
+To compare a model across populations, provide matching keys in `probs` and `reals`. Population sizes may differ; each probability vector only needs to match the outcome vector for the same key.
 
 ```python
 probs_populations = {
     "Train": np.array([0.1, 0.9, 0.2, 0.8, 0.3, 0.7]),
-    "Test": np.array([0.2, 0.8, 0.3, 0.7, 0.4, 0.6]),
+    "Test": np.array([0.2, 0.8, 0.3, 0.7]),
 }
 reals_populations = {
     "Train": np.array([0, 1, 0, 1, 0, 1]),
-    "Test": np.array([0, 1, 0, 1, 0, 0]),
+    "Test": np.array([0, 1, 0, 0]),
 }
 
 fig = rk.create_calibration_curve(
@@ -99,4 +103,6 @@ fig = rk.create_calibration_curve(
 fig.show()
 ```
 
-From here, use the API Reference for the full set of curve types, parameters, and time-to-event variants. The [Naming Conventions](user-guide/naming-conventions.html) guide explains how the exported function families fit together.
+Here, `Train` contains six observations and `Test` contains four. This matching-key contract is supported by calibration as well as the other curve families.
+
+From here, use the API Reference for the full set of curve types, parameters, and time-to-event variants. The [Naming Conventions](naming-conventions.html) guide explains how the exported function families fit together, while [Curve API Compatibility](curve-api-compatibility.html) documents where those families still differ.

user_guide/02-curve-api-compatibility.qmd

@@ -0,0 +1,85 @@
+---
+title: "Curve API Compatibility"
+guide-section: "Using rtichoke"
+---
+
+`rtichoke` exposes parallel function families for discrimination, calibration, and decision-curve analysis. Most of them share the same input conventions; the main differences are in time-dependent heuristic handling, especially calibration.
+
+This page focuses on the conventions that are shared across curve families and on the few places where users should expect different behavior.
+
+## Multiple named populations
+
+The curve families accept named probability arrays, so populations such as Train and Test can be evaluated together. When outcomes are also supplied as a dictionary, matching dictionary keys are paired population-by-population. Each probability vector must match the outcome vector for its own population, but different populations may have different sample sizes.
+
+```python
+import numpy as np
+import rtichoke as rk
+
+probs = {
+    "Train": np.array([0.10, 0.90, 0.20, 0.80, 0.30, 0.70]),
+    "Test": np.array([0.15, 0.85, 0.25, 0.75]),
+}
+reals = {
+    "Train": np.array([0, 1, 0, 1, 0, 1]),
+    "Test": np.array([0, 1, 0, 0]),
+}
+
+fig = rk.create_calibration_curve(probs=probs, reals=reals)
+```
+
+Here Train has six observations and Test has four. That is supported. What matters is the within-population alignment:
+
+```text
+len(probs["Train"]) == len(reals["Train"])
+len(probs["Test"])  == len(reals["Test"])
+```
+
+The same named-population pattern is used by ROC, precision-recall, Gains, Lift, decision, and calibration curve families. For time-dependent calls, `times` follows the same population alignment when supplied as a dictionary.
+
+## Censoring and competing-event heuristics
+
+Time-dependent functions distinguish censoring from competing events. The heuristic for an outcome type matters only when observations of that type are present:
+
+- If there are no competing events, changing `competing_heuristic` does not change the statistical estimates because there are no competing events for that rule to act on.
+- If there are no censored observations, changing `censoring_heuristic` does not change the statistical estimates because there are no censored observations for that rule to act on.
+- If neither censoring nor competing events are present, the heuristic choices do not alter the estimates.
+
+These statements describe the effect of the heuristics on the estimates. Function-specific input validation still applies: a function can reject an unsupported heuristic combination even when the corresponding outcome type is absent.
+
+## Time-dependent calibration heuristics
+
+`create_calibration_curve_times()` differs from its ROC, precision-recall, Gains, Lift, and decision-curve siblings in two important ways:
+
+1. `heuristics_sets` is currently required rather than defaulted.
+2. Calibration explicitly rejects unsupported heuristic combinations, including `censoring_heuristic="adjusted"` and `competing_heuristic="adjusted_as_censored"`, with an `Unsupported calibration heuristics` error instead of silently skipping every requested horizon.
+
+Pass the calibration heuristic explicitly. For the currently working exclusion-based path:
+
+```python
+heuristics_sets = [
+    {
+        "censoring_heuristic": "excluded",
+        "competing_heuristic": "adjusted_as_negative",
+    }
+]
+```
+
+Then call:
+
+```python
+fig = rk.create_calibration_curve_times(
+    probs=probs,
+    reals=reals,
+    times=times,
+    fixed_time_horizons=[3.0, 6.0, 9.0],
+    heuristics_sets=heuristics_sets,
+)
+```
+
+## Numeric time horizons
+
+`fixed_time_horizons` accepts integer or floating-point numeric values. Integer horizons are normalized to floats at the shared time-dependent processing boundary, so `[3, 6, 9]` and `[3.0, 6.0, 9.0]` are equivalent.
+
+## Related functions
+
+When moving between curve families, compare the API reference for the relevant `_times()` functions rather than assuming their defaults and accepted heuristics are identical. In particular, calibration has a narrower heuristic contract than the other time-dependent curve families.

user_guide/03-common-errors.qmd

@@ -0,0 +1,95 @@
+---
+title: "Common Errors & Fixes"
+guide-section: "Using rtichoke"
+---
+
+This page is deliberately keyed by **literal error text**. If an rtichoke call fails, search this page for a distinctive part of the exception before tracing into the implementation.
+
+## Population key or length mismatch
+
+For matching `probs` and `reals` dictionaries, rtichoke pairs values population-by-population. Different populations may have different sample sizes, but lengths must match within each key.
+
+```python
+probs = {
+    "Train": train_probs,
+    "Test": test_probs,
+}
+reals = {
+    "Train": train_outcomes,
+    "Test": test_outcomes,
+}
+```
+
+Check that the keys match and that each pair has equal length. Unequal Train/Test sample sizes are supported across the curve families that accept these inputs. For time-dependent calls, dictionary-valued `times` must follow the same population alignment.
+
+See [Curve API Compatibility](curve-api-compatibility.html) for the family-by-family comparison.
+
+## `Unsupported calibration heuristics`
+
+### Where this appears
+
+`create_calibration_curve_times()`.
+
+### Why it happens
+
+Calibration does not currently implement `censoring_heuristic="adjusted"` or
+`competing_heuristic="adjusted_as_censored"`. These inputs are rejected
+before curve construction rather than silently skipping every requested horizon.
+
+### Fix
+
+Pass a supported calibration heuristic explicitly. For the exclusion-based path:
+
+```python
+heuristics_sets = [
+    {
+        "censoring_heuristic": "excluded",
+        "competing_heuristic": "adjusted_as_negative",
+    }
+]
+```
+
+Do not infer calibration defaults from the other time-dependent curve families.
+
+A heuristic only changes estimates when the corresponding outcome type is present: a competing-event rule has no statistical effect when there are no competing events, and a censoring rule has no statistical effect when there is no censoring. Input validation is separate from this statistical point, so unsupported calibration combinations can still be rejected even when the relevant outcome type is absent.
+
+## `No data remaining after applying heuristics and time horizons.`
+
+Unsupported calibration heuristics now raise the targeted error above. If this
+message still appears, the supported heuristic and horizon combination removed
+all observations. Check the observed times, event values, requested horizons,
+and exclusion rules.
+
+## Integer and floating-point time horizons
+
+`fixed_time_horizons` accepts integer and floating-point numeric values.
+Integer horizons are normalized to floats internally:
+
+```python
+fixed_time_horizons=[3, 6, 9]
+```
+
+is equivalent to:
+
+```python
+fixed_time_horizons=[3.0, 6.0, 9.0]
+```
+
+## Why is `heuristics_sets` missing?
+
+If Python reports that `create_calibration_curve_times()` is missing the required `heuristics_sets` argument, that is currently expected API behavior. Unlike the ROC, precision-recall, Gains, Lift, and decision-curve `_times` functions, calibration does not currently provide a default.
+
+Pass it explicitly rather than copying a sibling default:
+
+```python
+heuristics_sets = [
+    {
+        "censoring_heuristic": "excluded",
+        "competing_heuristic": "adjusted_as_negative",
+    }
+]
+```
+
+## Still stuck?
+
+Check [Curve API Compatibility](curve-api-compatibility.html) first. The most important remaining difference is calibration's required and narrower heuristic selection, not unequal population sizes or integer horizons.