|
1 | 1 |
|
| 2 | +[](https://pat-alt.github.io/AlgorithmicRecourseDynamics.jl/stable) [](https://pat-alt.github.io/AlgorithmicRecourseDynamics.jl/dev) [](https://github.com/pat-alt/AlgorithmicRecourseDynamics.jl/actions/workflows/CI.yml?query=branch%3Amain) [](https://codecov.io/gh/pat-alt/AlgorithmicRecourseDynamics.jl) [](https://github.com/invenia/BlueStyle) [](https://github.com/SciML/ColPrac) [](https://twitter.com/paltmey) |
| 3 | + |
2 | 4 | # AlgorithmicRecourseDynamics |
3 | 5 |
|
4 | | -<!-- [](https://pat-alt.github.io/CounterfactualExplanations.jl/stable) --> |
5 | | -<!-- [](https://pat-alt.github.io/CounterfactualExplanations.jl/dev) --> |
| 6 | +`AlgorithmicRecourseDynamics.jl` is a small package for modeling Algorithmic Recourse Dynamics. It builds on `CounterfactualExplanations`, a package for generating counterfactual explanations. |
| 7 | + |
| 8 | +## Basic Usage |
| 9 | + |
| 10 | +### Data and Model |
| 11 | + |
| 12 | +``` julia |
| 13 | +N = 1000 |
| 14 | +xmax = 2 |
| 15 | +X, ys = make_blobs( |
| 16 | + N, 2; |
| 17 | + centers=2, as_table=false, center_box=(-xmax => xmax), cluster_std=0.1 |
| 18 | +) |
| 19 | +ys .= ys.==2 |
| 20 | +X = X' |
| 21 | +counterfactual_data = CounterfactualData(X,ys') |
| 22 | +``` |
6 | 23 |
|
7 | | -[](https://github.com/pat-alt/CounterfactualExplanations.jl/actions/workflows/CI.yml?query=branch%3Amain) <!-- [](https://codecov.io/gh/pat-alt/CounterfactualExplanations.jl) --> |
| 24 | +``` julia |
| 25 | +n_epochs = 100 |
| 26 | +model = Chain(Dense(2,1)) |
| 27 | +mod = FluxModel(model) |
8 | 28 |
|
9 | | -`AlgorithmicRecourseDynamics.jl` is a Julia package for modelling Algorithmic Recourse Dynamics. |
| 29 | +generator = GenericGenerator() |
| 30 | +``` |
10 | 31 |
|
11 | | -## Research Paper 📝 |
| 32 | +``` julia |
| 33 | +data_train, data_test = Data.train_test_split(counterfactual_data) |
| 34 | +Models.train(mod, data_train; n_epochs=n_epochs) |
| 35 | +plt_original = plot(mod, counterfactual_data; zoom=0, colorbar=false) |
| 36 | +display(plt_original) |
| 37 | +``` |
12 | 38 |
|
13 | | -**Note** ⚠: You are on the `#original-paper` branch of `AlgorithmicRecourseDynamics.jl`. This branch is a static artifact corresponding to the state of the package at the time the paper was first published. It can be used to replicate the original findings of the paper. For an up-to-date version of the package, please switch to the [`#main`](https://github.com/pat-alt/AlgorithmicRecourseDynamics.jl) branch. |
| 39 | + |
14 | 40 |
|
15 | | -## At a Glance |
| 41 | +### Simulation |
16 | 42 |
|
17 | | -The paper titles **Endogenous Macrodynamics in Algorithmic Recourse** is currently under review and not yet published. You can find a preprint along with other resources right here on this branch of the repository: |
| 43 | +``` julia |
| 44 | +models = Dict(:mymodel => mod) |
| 45 | +generators = Dict(:wachter => generator) |
| 46 | +experiment = set_up_experiment(data_train, data_test, models, generators) |
| 47 | +``` |
18 | 48 |
|
19 | | -- [Paper](paper/paper.pdf) |
20 | | -- [Notebooks](dev/notebooks/) |
21 | | -- [Supplementary Appendix](build/dev/notebooks/appendix.html) (download the HTML and view in browser) |
22 | | -- [Artifacts](https://github.com/pat-alt/AlgorithmicRecourseDynamics.jl/releases/tag/artifacts) (including data and experimental results) |
| 49 | +``` julia |
| 50 | +run!(experiment) |
| 51 | +``` |
23 | 52 |
|
24 | | -In this work we investigate what happens if Algorithmic Recourse is actually implemented by a large number of individuals. The chart below illustrates what we mean by Endogenous Macrodynamics in Algorithmic Recourse: (a) we have a simple linear classifier trained for binary classification where samples from the negative class ($y=0$) are marked in blue and samples of the positive class ($y=1$) are marked in orange; (b) the implementation of AR for a random subset of individuals leads to a noticable domain shift; (c) as the classifier is retrained we observe a corresponding model shift; (d) as this process is repeated, the decision boundary moves away from the target class. |
| 53 | +``` julia |
| 54 | +new_data = experiment.recourse_systems[1][1].data |
| 55 | +new_model = experiment.recourse_systems[1][1].model |
| 56 | +plt_original = plot(new_model, new_data; zoom=0, colorbar=false) |
| 57 | +``` |
25 | 58 |
|
26 | | - |
| 59 | + |
27 | 60 |
|
28 | | -## Paper Abstract |
| 61 | +## Related Research Paper 📝 |
29 | 62 |
|
30 | | -Existing work on Counterfactual Explanations (CE) and Algorithmic Recourse (AR) has largely been limited to the static setting and focused on single individuals: given some estimated model the goal is to find valid counterfactuals for individual instance that fulfill various desiderata. The ability of such counterfactuals to handle dynamics like data and model drift remains a largely unexplored research challenge at this point. There has also been surprisingly little work on the related question of how the actual implementation of recourse by one individual may affect other individuals. Through this work we aim to close that gap by systematizing and extending existing knowledge. We first show that many of the existing methodologies can be collectively described by a generalized framework. We then argue that the existing framework fails to account for a hidden external cost of recourse, that only reveals itself when studying the endogenous dynamics of recourse at the group level. Through simulation experiments involving various state-of-the-art counterfactual generators and several benchmark datasets, we generate large numbers of counterfactuals and study the resulting domain and model shifts. We find that the induced shifts are substantial enough to likely impede the applicability Algorithmic Recourse in situations that involve competition for scarce resources. Fortunately, we find various potential mitigation strategies that can be used in combination with existing approaches. Our simulation framework for studying recourse dynamics is fast and open-sourced. |
| 63 | +The package was developed for a research project that investigates the dynamics of various counterfactual generators. |
0 commit comments