Fairness-Aware Classification on the COMPAS Recidivism Dataset

Authors: Timnah Weckner, Celina Kaliman, Anton Schmidt — Humboldt-Universität zu Berlin

Table of Content

• Summary
• Working with the repo
  • Dependencies
  • Setup
• Reproducing results
• Project structure

Summary

This project investigates algorithmic fairness in criminal justice risk assessment using the COMPAS recidivism dataset. It evaluates how far bias-mitigation techniques can reduce algorithmic discrimination without significantly sacrificing predictive accuracy.

A Logistic Regression classifier is trained to predict recidivism risk and evaluated across three sensitive attributes — race, sex, and the combination of both — using fairness metrics from the AIF360 toolkit (Demographic Parity Difference, Average Odds Difference, Predictive Parity Difference). Two bias-mitigation methods are then applied and compared against the unmitigated baseline:

• Pre-processing: Reweighing
• Post-processing: Equalized Odds

Results are compared both in tabular form and visually (bar charts, heatmaps, and an accuracy-vs-fairness trade-off plot).

Keywords: algorithmic fairness, criminal justice, COMPAS, bias mitigation, reweighing, equalized odds

Working with the repo

Dependencies

• All required libraries are listed in requirements.txt
• The dataset is loaded via aif360.datasets.CompasDataset(). AIF360 does not ship the raw COMPAS data for licensing reasons — before running the notebook, download compas-scores-two-years.csv from the ProPublica COMPAS analysis repo and place it in aif360/data/raw/compas/ inside your AIF360 installation, as described in the AIF360 documentation.

Setup

1. Clone this repository

git clone https://github.com/timnahw/compas-fairness-analysis
cd compas-fairness-analysis

2. Create a virtual environment and activate it

python -m venv venv
source venv/bin/activate

3. Install requirements

pip install --upgrade pip
pip install -r requirements.txt

Reproducing results

All results are produced by a single notebook, compas_fairness_experiment.ipynb:

1. Activate the environment set up above and launch Jupyter:

jupyter notebook compas_fairness_experiment.ipynb

2. Run the notebook top to bottom, in one session.
3. A fixed random seed (25) is set for reproducibility.

The notebook covers, for each of the three attribute settings (race, sex, race & sex combined): classification without fairness processors, classification with Reweighing (pre-processing), and classification with Equalized Odds (post-processing) — followed by a tabular and visual comparison of all results.

Project structure

├── README.md
├── requirements.txt
└── compas_fairness_experiment.ipynb