minor changes to paper draft

Author: pat-alt

paper/paper.qmd

@@ -45,6 +45,7 @@ format:
 csl: https://raw.githubusercontent.com/citation-style-language/styles/master/ieee.csl
 number-sections: true
 license: "CC BY"
+link-citations: true
 ---
 
 {{< include sections/abstract.qmd >}}

paper/sections/limitations.qmd

@@ -1,5 +1,4 @@
 ## Limitations and Future Work {#sec-limit}
 
-The experimental setup proposed here is designed to mimic a real-world recourse process in a simple fashion. In practice, models are in fact updated on a regular basis [@upadhyay2021towards]. We also find it plausible to assume that the implementation of recourse happens periodically for different individuals, rather that all at once at time $t=0$. That being said, our experimental design is a vast over-simplification of potential real-world scenarios. In practice, any endogenous shifts that may occur can be expected to be entangled with exogenous shifts of the nature investigated in @upadhyay2021towards. We also make implicit assumptions about the utility functions of the involved agents that may well be too simple: individuals seeking recourse are assumed to always implement the proposed counterfactual explanations; conversely, the agent in charge of the model $M$ is assumed to always treat individuals that have implemented valid recourse as if they were truly now in the target class. Relating this back to the consumer credit example, we assume that the would-be borrowers are always willing and able to implement recourse and the bank is always willing to provide credit as would-be borrowers move across the decision boundary. In practice it is doubtful that agents behave according to such simple rules. 
+The experimental setup proposed here is designed to mimic a real-world recourse process in a simple fashion. In practice, models are in fact updated on a regular basis [@upadhyay2021towards]. We also find it plausible to assume that the implementation of recourse happens periodically for different individuals, rather that all at once at time $t=0$. That being said, our experimental design is a vast over-simplification of potential real-world scenarios. In practice, any endogenous shifts that may occur can be expected to be entangled with exogenous shifts of the nature investigated in @upadhyay2021towards. We also make implicit assumptions about the utility functions of the involved agents that may well be too simple: individuals seeking recourse are assumed to always implement the proposed counterfactual explanations; conversely, the agent in charge of the model $M$ is assumed to always treat individuals that have implemented valid recourse as if they were truly now in the target class. Relating this back to the consumer credit example, we assume that the would-be borrowers are always willing and able to implement recourse and the bank is always willing to provide credit as would-be borrowers move across the decision boundary. In practice it is doubtful that agents behave according to such simple rules. Nonetheless, we think that our simple framework offers a starting point for future work on recourse dynamics (both endogenous and exogenous dynamics). 
 
-Nonetheless, we think that our simple framework offers a starting point for future work on recourse dynamics (both endogenous and exogenous dynamics). 

paper/sections/methodology.qmd

@@ -17,19 +17,19 @@ Our motivation for including these different generators in our analysis, is that
 
 ## Experimental Setup {#sec-method-experiment}
 
-The dynamics illustrated in @fig-poc in @sec-intro were generated through a simple experiment that aims to simulate the process of algorithmic recourse in practice. We begin in the static setting at time time $t=0$: given some pre-trained classifier $M$ we generate recourse for a random batch of $B$ individuals in the non-target class. Note that we focus our attention on classification problems, since classification poses the most common practical use-case for algorithmic recourse. In order to simulate the dynamical process we suppose that the model $M$ is retrained following the actual implementation of recourse in time $t=0$. Following the update to the model, we assume that at time $t=1$ recourse is generated for yet another random subset of individuals in the non-target class. This process is repeated for a number of time periods $T$. To get a clean read on endogenous dynamics we keep the total population of samples closed: we allow existing samples to move from factual to counterfactual states, but do not allow any entirely new samples to enter the population. The experimental setup is summarized in Algorithm \ref{euclid}
+The dynamics illustrated in @fig-poc in @sec-intro were generated through a simple experiment that aims to simulate the process of algorithmic recourse in practice. We begin in the static setting at time time $t=0$: given some pre-trained classifier $M$ we generate recourse for a random batch of $B$ individuals in the non-target class. Note that we focus our attention on classification problems, since classification poses the most common practical use-case for algorithmic recourse. In order to simulate the dynamical process we suppose that the model $M$ is retrained following the actual implementation of recourse in time $t=0$. Following the update to the model, we assume that at time $t=1$ recourse is generated for yet another random subset of individuals in the non-target class. This process is repeated for a number of time periods $T$. To get a clean read on endogenous dynamics we keep the total population of samples closed: we allow existing samples to move from factual to counterfactual states, but do not allow any entirely new samples to enter the population. The experimental setup is summarized in Algorithm \ref{algo-experiment}
 
 \begin{algorithm}
-\caption{PLACEHOLDER}\label{euclid}
+\caption{Experiment}\label{algo-experiment}
 \begin{algorithmic}[1]
-\Procedure{Euclid}{$a,b$}\Comment{The g.c.d. of a and b}
-\State $r\gets a\bmod b$
-\While{$r\not=0$}\Comment{We have the answer if r is 0}
-\State $a\gets b$
-\State $b\gets r$
-\State $r\gets a\bmod b$
-\EndWhile\label{euclidendwhile}
-\State \textbf{return} $b$\Comment{The gcd is b}
+\Procedure{Experiment}{$M,D,G$}
+\State $t\gets 0$
+\While{$t<T$}
+\State $D_B \subset D$
+\State $D_B\gets G(D_B)$ \Comment{Generate counterfactuals.}
+\State $M\gets M(D)$ \Comment{Retrain model.}
+\EndWhile
+\State \textbf{return} $M,D$
 \EndProcedure
 \end{algorithmic}
 \end{algorithm}
@@ -45,7 +45,7 @@ We have chosen to work with both synthetic and real-world datasets. Using synthe
 
 We use 6 synthetic binary classification datasets consisting of 200-400 samples grouped in normally-distributed clusters.^[To see how the data is generated see here: [https://github.com/pat-alt/algorithmic_recourse_dynamics/blob/main/notebooks/synthetic_datasets.ipynb](https://github.com/pat-alt/algorithmic_recourse_dynamics/blob/main/notebooks/synthetic_datasets.ipynb)] The datasets are presented in @fig-synthetic-data (see also Appendix A for a formal description). Samples from the negative class are marked in blue while samples of the positive class are marked in orange.
 
-![A visualization of the synthetic classification datasets used in our experiments.](www/synthetic_data.png){#fig-synthetic-data}
+![PLACEHOLDER: A visualization of the synthetic classification datasets used in our experiments.](www/synthetic_data.png){#fig-synthetic-data}
 
 Ex-ante we expect to see that Wachter will create a new cluster of counterfactual instances in the proximity of the initial decision boundary. Thus, the choice of a black-box model may have an impact on the paths of the recourse. For generators that use latent space search (@joshi2019towards, @antoran2020getting) or rely on (and have access to) probabilistic models (@antoran2020getting, @schut2021generating) we expect that counterfactuals will end up in regions of the target domain that are densely populated by training samples. Finally, we expect to see the counterfactuals generated by DiCE to be uniformly spread around the feature space inside the target class. 
 
@@ -79,6 +79,8 @@ Finally, **feature mean and feature standard deviation** are also calculated to
 
 ### Model Shifts
 
+As our baseline for quantifying model shifts we measure perturbations to the model parameters at each point in time $t$ following @upadhyay2021towards. We define $\Delta=||\theta_{t+1}-\theta_{t}||^2$, that is the euclidean distance between the vectors of parameters before and after retraining the model $M$. We shall refer to this baseline metric simply as **Perturbations**.
+
 We extend the metric in @eq-mmd for the purpose of quantifying model shifts. Specifically, we introduce **Predicted Probability MMD (PP MMD)**: instead of applying @eq-mmd to features directly, we apply it to the predicted probabilities assigned to a set of samples by the model $M$. If the model shifts, the probabilities assigned to each sample will change; again, this metric will equal 0 only if the two classifiers are the same. It is worth noting that while we apply the technique to samples drawn uniformly from the dataset, it can also be employed on arbitrary points in the entire feature space (or a subspace). The latter approach is theoretically more robust. Unfortunately, in practice this approachs suffers from the curse of dimensionality, since it becomes increasingly difficult to select enough points to overcome noise as the dimension $D$ grows.
 
 As an alternative to PP MMD we use a pseudo-distance for the **Disagreement Coefficient** (Disagreement). This metric was introduced in @hanneke2007bound and estimates $p(M(x) \neq M^\prime(x))$, that is the probability that two classifiers do not agree on the predicted outcome for a randomly chosen sample. Thus, it is not relevant whether the classification is correct according to the ground truth, but only whether the sample lies on the same side of the two respective decision boundaries. In our context, this metric quantifies the overlap between the initial model (trained before the application of recourse) and the updated model. A Disagreement Coefficient unequal to zero is indicative of a model shift. The opposite is not true: even if the Disagreement Coefficient is equal to zero a model shift may still have occured. This is one reason for why PP MMD is our our preferred metric.