Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions blog-navbar.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
<script>
document.addEventListener('DOMContentLoaded', () => {
const marker = '/examples/';
const path = window.location.pathname;
const pos = path.indexOf(marker);
if (pos === -1) return;

document.documentElement.classList.add('rtichoke-blog-shell');

const examplesRoot = path.slice(0, pos) + '/examples/';
const nav = document.querySelector('nav.navbar');
if (!nav) return;

const brand = nav.querySelector('.navbar-brand');
if (brand) {
brand.textContent = 'rtichoke blog';
brand.setAttribute('href', 'https://rtichoke-blog.netlify.app/');
brand.setAttribute('aria-label', 'rtichoke blog home');
}

const navLists = [...nav.querySelectorAll('ul.navbar-nav')];
if (!navLists.length) return;

const mainList = navLists.find((el) => el.classList.contains('me-auto')) || navLists[0];
const rightList = navLists.find((el) => el.classList.contains('ms-auto')) || navLists[navLists.length - 1];

if (mainList !== rightList) mainList.replaceChildren();

const links = [
['R examples', 'https://rtichoke-blog.netlify.app/posts'],
['Python examples', examplesRoot],
['Talks', 'https://rtichoke-blog.netlify.app/talks.qmd'],
['About Me', 'https://rtichoke-blog.netlify.app/about_me']
];

const iconLinks = [
['github', 'https://github.com/uriahf'],
['twitter-x', 'https://twitter.com/finkeluriah'],
['facebook', 'https://www.facebook.com/groups/rforthemasses'],
['linkedin', 'https://www.linkedin.com/in/uriah-finkel'],
['telegram', 'https://t.me/as_if_uriah'],
['rss', 'https://rtichoke-blog.netlify.app/posts.xml']
];

rightList.replaceChildren();

for (const [label, href] of links) {
const li = document.createElement('li');
li.className = 'nav-item';
const a = document.createElement('a');
a.className = 'nav-link';
a.href = href;
a.textContent = label;
if (label === 'Python examples') a.classList.add('active');
li.appendChild(a);
rightList.appendChild(li);
}

for (const [icon, href] of iconLinks) {
const li = document.createElement('li');
li.className = 'nav-item compact';
const a = document.createElement('a');
a.className = 'nav-link';
a.href = href;
a.setAttribute('aria-label', icon);
const i = document.createElement('i');
i.className = `bi bi-${icon}`;
a.appendChild(i);
li.appendChild(a);
rightList.appendChild(li);
}
});
</script>
13 changes: 13 additions & 0 deletions examples/index.qmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
title: "Python Examples"
listing:
contents: posts
feed: true
sort: "date desc"
type: default
categories: true
sort-ui: false
filter-ui: false
page-layout: full
title-block-banner: true
---
103 changes: 103 additions & 0 deletions examples/posts/calibration_curve/index.qmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
---
title: "Calibration Curves for Multiple Models"
author: "Uriah Finkel"
date: "2024-05-15"
description: "An example of creating a calibration curve using rtichoke and scikit-learn."
categories: [calibration, scikit-learn]
draft: false
---

The following example is inspired by the [scikit-learn documentation displaying a calibration curve](https://scikit-learn.org/stable/auto_examples/calibration/plot_calibration_curve.html).

## Load data and fit models

```{python}
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.naive_bayes import GaussianNB
from sklearn.calibration import CalibratedClassifierCV
import numpy as np
from sklearn.svm import LinearSVC

X, y = make_classification(
n_samples=10_000, n_features=20, n_informative=2, n_redundant=10, random_state=42
)

X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.99, random_state=42
)

lr = LogisticRegression(C=1.0)
gnb = GaussianNB()
gnb_isotonic = CalibratedClassifierCV(gnb, cv=2, method="isotonic")
gnb_sigmoid = CalibratedClassifierCV(gnb, cv=2, method="sigmoid")

lr.fit(X_train, y_train)
gnb.fit(X_train, y_train)
gnb_isotonic.fit(X_train, y_train)
gnb_sigmoid.fit(X_train, y_train)

y_proba_lr = lr.predict_proba(X_test)[:, 1]
y_proba_gnb = gnb.predict_proba(X_test)[:, 1]
y_proba_gnb_isotonic = gnb_isotonic.predict_proba(X_test)[:, 1]
y_proba_gnb_sigmoid = gnb_sigmoid.predict_proba(X_test)[:, 1]

class NaivelyCalibratedLinearSVC(LinearSVC):
def fit(self, X, y):
super().fit(X, y)
df = self.decision_function(X)
self.df_min_ = df.min()
self.df_max_ = df.max()

def predict_proba(self, X):
df = self.decision_function(X)
calibrated_df = (df - self.df_min_) / (self.df_max_ - self.df_min_)
proba_pos_class = np.clip(calibrated_df, 0, 1)
proba_neg_class = 1 - proba_pos_class
return np.c_[proba_neg_class, proba_pos_class]

svc = NaivelyCalibratedLinearSVC(max_iter=10_000)
svc_isotonic = CalibratedClassifierCV(svc, cv=2, method="isotonic")
svc_sigmoid = CalibratedClassifierCV(svc, cv=2, method="sigmoid")

svc.fit(X_train, y_train)
svc_isotonic.fit(X_train, y_train)
svc_sigmoid.fit(X_train, y_train)

y_proba_svc = svc.predict_proba(X_test)[:, 1]
y_proba_svc_isotonic = svc_isotonic.predict_proba(X_test)[:, 1]
y_proba_svc_sigmoid = svc_sigmoid.predict_proba(X_test)[:, 1]
```

## Gaussian Naive Bayes

```{python}
from rtichoke import create_calibration_curve

create_calibration_curve(
probs={
"Logistic": y_proba_lr,
"Naive Bayes": y_proba_gnb,
"Naive Bayes + Isotonic": y_proba_gnb_isotonic,
"Naive Bayes + Sigmoid": y_proba_gnb_sigmoid,
},
reals=y_test
).show(config={"displayModeBar": False, "displaylogo": False})
```

## Linear SVC

```{python}
create_calibration_curve(
probs={
"Logistic": y_proba_lr,
"SVC": y_proba_svc,
"SVC + Isotonic": y_proba_svc_isotonic,
"SVC + Sigmoid": y_proba_svc_sigmoid,
},
reals=y_test
).show(config={"displayModeBar": False, "displaylogo": False})
```

This reproduces the core comparison from the scikit-learn example while keeping the rtichoke rendering as the Python example itself.
62 changes: 62 additions & 0 deletions examples/posts/dca_with_fixed_time_horizons/index.qmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
title: "Decision Curves with Fixed Time Horizons"
author: "Uriah Finkel"
date: "2025-12-15"
categories: [Decision, Time-Horizons]
---

`rtichoke` for Python introduces support for fixed time horizons, allowing flexible specification of the prediction horizon and automatically updating performance plots accordingly.

## The most under-discussed design choice in prediction models: The Time Horizon

Prediction models require a well-defined fixed time horizon: The end of follow-up over which the outcome probability is defined. A probability of dying within 1 week, 1 year, or 100 years represents fundamentally different clinical questions and very different implied decision contexts.

It is therefore important to explore the sensitivity of model performance to the choice of time horizon: Shorter horizons typically yield fewer observed events, resulting in smaller gaps between the two baseline strategies "treat all" (Everyone is considered Predicted Positive) and "treat none" (everyone is considered Predicted Negative).

In contrast, longer horizons may introduce ambiguity through censoring (loss to follow-up) or competing events (events that preclude the primary outcome).

## Pragmatic approach: Performance Sensitivity Analysis with rtichoke

You do not need to develop a new or more complex model to overcome these problems, first you need to ensure if there's a problem at all and for which time horizons:

You can reuse predictions trained for a specific horizon and evaluate their robustness across alternative fixed time horizons: This allows you to assess how performance changes as the effective follow-up window varies without retraining the model.

## Load data and fit a Cox Regression

```{python}
import pandas as pd
import lifelines

df_time_to_cancer_dx = pd.read_csv(
"https://raw.githubusercontent.com/ddsjoberg/dca-tutorial/main/data/df_time_to_cancer_dx.csv"
)

cph = lifelines.CoxPHFitter()
cph.fit(
df=df_time_to_cancer_dx,
duration_col="ttcancer",
event_col="cancer",
formula="age + famhistory + marker",
)

cph_pred_vals = cph.predict_survival_function(
df_time_to_cancer_dx[["age", "famhistory", "marker"]], times=[1.5]
)

df_time_to_cancer_dx["pr_failure18"] = [1 - val for val in cph_pred_vals.iloc[0, :]]
```

## Decision Curve with Multiple Fixed Time Horizons

The `fixed_time_horizons` argument allows you to explicitly define the set of follow-up horizons to evaluate.

```{python}
from rtichoke import create_decision_curve_times

create_decision_curve_times(
probs={"full": df_time_to_cancer_dx["pr_failure18"]},
reals=df_time_to_cancer_dx["cancer"],
times=df_time_to_cancer_dx["ttcancer"],
fixed_time_horizons=[0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0]
)
```
129 changes: 89 additions & 40 deletions great-docs.yml
Original file line number Diff line number Diff line change
@@ -1,49 +1,98 @@
display_name: rtichoke
parser: numpy
repo: https://github.com/uriahf/rtichoke_python
user_guide: user_guide
homepage: user_guide
display_name: rtichoke
parser: numpy
repo: https://github.com/uriahf/rtichoke_python
user_guide: user_guide
homepage: user_guide
site_url: https://uriahf.github.io/rtichoke_python/
include_in_header:
- text: |
<script>
document.addEventListener('DOMContentLoaded', () => {
const marker = '/examples/';
const path = window.location.pathname;
const pos = path.indexOf(marker);
if (pos === -1) return;

document.documentElement.classList.add('rtichoke-blog-shell');

const examplesRoot = path.slice(0, pos) + '/examples/';
const nav = document.querySelector('nav.navbar');
if (!nav) return;

const container = nav.querySelector('.navbar-container');
if (!container) return;

container.innerHTML = `
<div class="navbar-brand-container mx-auto">
<a class="navbar-brand" href="https://rtichoke-blog.netlify.app/" aria-label="rtichoke blog home">
<span class="navbar-title">rtichoke blog</span>
</a>
</div>
<button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarCollapse" aria-controls="navbarCollapse" aria-expanded="false" aria-label="Toggle navigation">
<span class="navbar-toggler-icon"></span>
</button>
<div class="collapse navbar-collapse" id="navbarCollapse">
<ul class="navbar-nav navbar-nav-scroll ms-auto">
<li class="nav-item"><a class="nav-link" href="https://rtichoke-blog.netlify.app/posts"><span class="menu-text">R examples</span></a></li>
<li class="nav-item"><a class="nav-link active" href="${examplesRoot}" aria-current="page"><span class="menu-text">Python examples</span></a></li>
<li class="nav-item"><a class="nav-link" href="https://rtichoke-blog.netlify.app/talks.qmd"><span class="menu-text">Talks</span></a></li>
<li class="nav-item"><a class="nav-link" href="https://rtichoke-blog.netlify.app/about_me"><span class="menu-text">About Me</span></a></li>
<li class="nav-item compact"><a class="nav-link" href="https://github.com/uriahf" aria-label="GitHub"><i class="bi bi-github"></i></a></li>
<li class="nav-item compact"><a class="nav-link" href="https://twitter.com/finkeluriah" aria-label="Twitter"><i class="bi bi-twitter-x"></i></a></li>
<li class="nav-item compact"><a class="nav-link" href="https://www.facebook.com/groups/rforthemasses" aria-label="Facebook"><i class="bi bi-facebook"></i></a></li>
<li class="nav-item compact"><a class="nav-link" href="https://www.linkedin.com/in/uriah-finkel" aria-label="LinkedIn"><i class="bi bi-linkedin"></i></a></li>
<li class="nav-item compact"><a class="nav-link" href="https://t.me/as_if_uriah" aria-label="Telegram"><i class="bi bi-telegram"></i></a></li>
<li class="nav-item compact"><a class="nav-link" href="https://rtichoke-blog.netlify.app/posts.xml" aria-label="RSS"><i class="bi bi-rss"></i></a></li>
</ul>
</div>`;
});
</script>

site:
css: site.css

sections:
- title: Examples
dir: examples
type: blog
navbar_after: User Guide

source:
enabled: true
branch: main
placement: usage
reference:
- title: Performance Data
desc: Prepare classification and time-to-event data for visualization.
contents:
- prepare_performance_data
- prepare_binned_classification_data
- prepare_performance_data_times
- prepare_binned_classification_data_times
- title: Discrimination
desc: ROC, precision-recall, gains, and lift visualizations.
contents:
- create_roc_curve
- create_roc_curve_times
- plot_roc_curve
- create_precision_recall_curve
- create_precision_recall_curve_times
- plot_precision_recall_curve
- create_gains_curve
- create_gains_curve_times
- plot_gains_curve
- create_lift_curve
- create_lift_curve_times
- plot_lift_curve
- title: Calibration
desc: Calibration visualizations for classification and time-to-event models.
contents:
- create_calibration_curve
- create_calibration_curve_times
enabled: true
branch: main
placement: usage

reference:
- title: Performance Data
desc: Prepare classification and time-to-event data for visualization.
contents:
- prepare_performance_data
- prepare_binned_classification_data
- prepare_performance_data_times
- prepare_binned_classification_data_times

- title: Discrimination
desc: ROC, precision-recall, gains, and lift visualizations.
contents:
- create_roc_curve
- create_roc_curve_times
- plot_roc_curve
- create_precision_recall_curve
- create_precision_recall_curve_times
- plot_precision_recall_curve
- create_gains_curve
- create_gains_curve_times
- plot_gains_curve
- create_lift_curve
- create_lift_curve_times
- plot_lift_curve

- title: Calibration
desc: Calibration visualizations for classification and time-to-event models.
contents:
- create_calibration_curve
- create_calibration_curve_times

- title: Utility
desc: Decision-curve analysis for classification and time-to-event models.
contents:
Expand Down
Loading
Loading