toned down and added more examples

Author: pat-alt

paper/paper.pdf

@@ -389,6 +389,23 @@
 
 \hyphenation{op-tical net-works semi-conduc-tor}
 
+\usepackage{amsthm}
+\newtheorem{theorem}{Theorem}[section]
+\newtheorem{lemma}{Lemma}[section]
+\newtheorem{corollary}{Corollary}[section]
+\newtheorem{proposition}{Proposition}[section]
+\newtheorem{conjecture}{Conjecture}[section]
+\theoremstyle{definition}
+\newtheorem{definition}{Definition}[section]
+\theoremstyle{definition}
+\newtheorem{example}{Example}[section]
+\theoremstyle{definition}
+\newtheorem{exercise}{Exercise}[section]
+\theoremstyle{definition}
+\newtheorem{hypothesis}{Hypothesis}[section]
+\theoremstyle{remark}
+\newtheorem*{remark}{Remark}
+\newtheorem*{solution}{Solution}
 \begin{document}
 %
 % paper title
@@ -540,7 +557,7 @@ \section{Introduction}\label{intro}}
 
 Research on Algorithmic Recourse has also so far typically addressed the issue from the perspective of one single individual and has indeed been referred to as \textbf{individual recourse} in some places. Arguably though, most real-world applications that warrant Algorithmic Recourse involve potentially large groups of individuals typically competing for scarce resources. Our work demonstrates that in such scenarios, choices made by or for one single individual are likely to affect the broader collective of individuals in ways that current approaches to AR fail to account for. More specifically, we argue that a strict focus on minimizing the private costs faced by individuals may be too narrow an objective.
 
-Figure \ref{fig:poc} illustrates this idea for a binary problem involving a probabilistic classifier and the counterfactual generator proposed by \protect\hyperlink{ref-wachter2017counterfactual}{{[}4{]}}: the implementation of AR for a subset of individuals leads to a domain shift (b), which in turn triggers a model shift (c). As this game of implementing AR and updating the classifier is repeated, the decision boundary moves away from training samples that were originally in the target class (d). We refer to these types of dynamics as \textbf{endogenous} because they are induced by the implementation of recourse itself. The term \textbf{macrodynamics} is borrowed from the economics literature and used to describe processes involving whole groups or societies.
+Figure \ref{fig:poc} illustrates this idea for a binary problem involving a probabilistic classifier and the counterfactual generator proposed by \protect\hyperlink{ref-wachter2017counterfactual}{{[}4{]}}: the implementation of AR for a subset of individuals immediately leads to a visible domain shift in the (orange) target class (b), which in turn triggers a model shift (c). As this game of implementing AR and updating the classifier is repeated, the decision boundary moves away from training samples that were originally in the target class (d). We refer to these types of dynamics as \textbf{endogenous} because they are induced by the implementation of recourse itself. The term \textbf{macrodynamics} is borrowed from the economics literature and used to describe processes involving whole groups or societies.
 
 \begin{figure}
 
@@ -551,7 +568,17 @@ \section{Introduction}\label{intro}}
 \caption{Dynamics in Algorithmic Recourse: we have a simple linear classifier trained for binary classification (a); the implementation of AR for a random subset of individuals leads to a noticable domain shift (b); as the classifier is retrained we observe a corresponding model shift (c); as this process is repeated, the decision boundary moves away from the target class (d).}\label{fig:poc}
 \end{figure}
 
-We think that these types of endogenous dynamics may be problematic and warrant our attention. Firstly, model shifts may inadvertently change classification outcomes for individuals who never received and implemented recourse. Secondly and relatedly, we observe in Figure \ref{fig:poc} that as the decision boundary moves in the direction of the non-target class, counterfactual paths become shorter: in the consumer credit example, individuals that previously would have been denied credit based on their input features are suddenly considered as creditworthy. Average default risk across all borrowers can therefore be expected to increase. Conversely, lenders that anticipate such dynamics may choose to refrain from offering recourse (and hence credit) to more than just a tiny share of individuals. In that latter and perhaps more likely scenario, the probability of being offered recourse decreases with every individual that implements recourse: in other words, the actions of first-movers exert a negative externality on future would-be borrowers.
+We think that these types of endogenous dynamics may be problematic and warrant our attention. From a purely technical we note the following: firstly, model shifts may inadvertently change classification outcomes for individuals who never received and implemented recourse. Secondly, we observe in Figure \ref{fig:poc} that as the decision boundary moves in the direction of the non-target class, counterfactual paths become shorter. We would argue that in some practical applications this can be expected to generate costs for the involved stakeholders. To follow our argument, consider the following two examples:
+
+\begin{example}[Consumer Credit]
+\protect\hypertarget{exm:consumer}{}\label{exm:consumer}Suppose Figure \ref{fig:poc} relates to an automated decision-making system used by a retail bank to evaluate credit applicants with respect to their credit worthiness. Assume that the two features are actually meaningful in the sense that creditworthiness increases in the south-east direction. Then we can think of the outcome in panel (d) as representing a situation where the bank supplies credit to more borrowers (orange), but these borrowers are on average less creditworthy and more of them can be expected to default on their loan. This represents a cost to the retail bank.
+\end{example}
+
+\begin{example}[Student Admission]
+\protect\hypertarget{exm:student}{}\label{exm:student}Suppose Figure \ref{fig:poc} relates to an automated decision-making system used by a university in their student admission process. Assume that the two features are actually meaningful in the sense that the likelihood of students successfully completing their degree increases in the south-east direction. Then we can think of the outcome in panel (b) as representing a situation where more students are admitted to university (orange), but they are more liekly to fail their degree than students that were admitted in previous years. The university admission committee catches on to this and suspends its efforts to offer Algorithmic Recourse. This represents an opportunity cost to future student applicants, that may have derived utility from being offered recourse.
+\end{example}
+
+Both of these examples are exaggerated simplifications of potential real-world scenarios, but they serve to illustrate the point that recourse for one single individual may exert negative externalities on other individuals.
 
 To the best of our knowledge this is the first work investigating endogenous macrodynamics in AR. Our contributions to the state of knowledge are as follows: firstly, we posit a compelling argument that calls for a novel perspective on Algorithmic Recourse extending our focus from single individuals to groups. Secondly, we introduce an experimental framework extending previous work by \protect\hyperlink{ref-altmeyer2022CounterfactualExplanations}{{[}5{]}}, which enables us to study macrodynamics of Algorithmic Recourse through simulations that can be fully parallelized. Thirdly, we use this framework to provide a first in-depth analysis of endogenous recourse dynamics induced by various popular counterfactual generators including \protect\hyperlink{ref-wachter2017counterfactual}{{[}4{]}}, \protect\hyperlink{ref-schut2021generating}{{[}6{]}}, \protect\hyperlink{ref-joshi2019towards}{{[}7{]}}, \protect\hyperlink{ref-mothilal2020explaining}{{[}8{]}} and \protect\hyperlink{ref-antoran2020getting}{{[}9{]}}. To this end we propose a number of novel evaluation metrics that can be used to quantify and benchmark the macrodynamics introduced by the different generators. Finally, we also discuss what drives endogenous dynamics and propose strategies to mitigate them.
 
@@ -850,8 +877,6 @@ \subsection{Endogenous Macrodynamics}\label{endogenous-macrodynamics}}
 
 Out-of-sample model performance also deteriorates across the board and substantially so: the smallest average reduction in F-Scores of around 15-20 percentage points is observed for the California Housing dataset. For this dataset we achieved the highest initial model performance of just under 90 percent, indicating once again that weaker classifiers may be more exposed to endogenous dynamics. As with the synthetic data, the estimates for logistic regression are qualitatively in line with the above, but quantitatively even more pronounced.
 
-Zooming in on individual features we \ldots{}
-
 \begin{figure}
 
 {\centering \includegraphics[width=0.9\linewidth]{www/real_world_results} 
@@ -944,7 +969,7 @@ \subsection{Gravitational Counterfactual Explanations}\label{gravitational-count
 \hypertarget{discussion}{%
 \section{Discussion}\label{discussion}}
 
-Our results in Section \ref{empirical-2} indicate that state-of-the-art approaches to Algorithmic Recourse induce substantial domain and model shift, if implemented at scale in practice. These induced shifts can and should be considered as an (expected) external cost of individual recourse. While they do not affect the individual directly as long as we look at the individual in isolation, they can been seen to affect the broader group of stakeholders in automated data-driven decision-making. We have seen, for example, that out-of-sample model performance generally deteriorates in our simulation experiments. In practice, this can be seen as a cost to model owners, that is the group of stakeholders using the model as decision-making tool. As we have set out in the introduction, these model owners will generally be unwilling to carry that cost, and hence can be expected to stop offering recourse to individuals altogether. This in turn is costly to those individuals that would otherwise derive utility from being offered recourse.
+Our results in Section \ref{empirical-2} indicate that state-of-the-art approaches to Algorithmic Recourse induce substantial domain and model shift, if implemented at scale in practice. These induced shifts can and should be considered as an (expected) external cost of individual recourse. While they do not affect the individual directly as long as we look at the individual in isolation, they can been seen to affect the broader group of stakeholders in automated data-driven decision-making. We have seen, for example, that out-of-sample model performance generally deteriorates in our simulation experiments. In practice, this can be seen as a cost to model owners, that is the group of stakeholders using the model as decision-making tool. As we have set out in Example \ref{exm:student} of our introduction, these model owners may be unwilling to carry that cost, and hence can be expected to stop offering recourse to individuals altogether. This in turn is costly to those individuals that would otherwise derive utility from being offered recourse.
 
 So, where does this leave us? We would argue that the expected external costs of individual recourse should be shared by all stakeholders. The most straight-forward way to achieve this is to introduce a penalty for external costs in the counterfactual search objective function, as we have set out in Equation \eqref{eq:collective}. This will on average lead to more costly counterfactual outcomes. But it may help to avoid extreme scenarios, in which minimal-cost recourse is reserved to a tiny minority of individuals. We have shown various types of shift-mitigating strategies that can be used to this end. Since all of these strategies can be seen simply as specific adaption of Equation \eqref{eq:collective}, they can be applied to any of the various counterfactual generators studied here.
 
@@ -976,7 +1001,7 @@ \subsection{Classifiers}\label{classifiers}}
 \hypertarget{conclusion}{%
 \section{Concluding Remarks}\label{conclusion}}
 
-This work has revisited and extended some of the most general and defining concepts underlying the literature on Counterfactual Explanations and, in particular, Algorithmic Recourse. We demonstrate that long-held beliefs as to what defines optimality in AR, may not be applicable in situations that involve decisions over scarce resources. Specifically, we run experiments that simulate the application of recourse in practice using various state-of-the-art counterfactual generators and find that all of them induce substantial domain and model shifts. We argue that these shifts should be considered as an expected external cost of individual recourse and call for a paradigm shift from individual to collective recourse. By proposing an adapted counterfactual search objective that incorporates this cost, we make that paradigm shift explicit. We show that this modified objective lends itself to mitigation strategies that can be used to effectively decrease the magnitude of induced domain and model shifts. Through our work we hope to inspire future research on this important topic. To this end we have open-sourced all of our code along with a Julia package - \href{https://anonymous.4open.science/r/AlgorithmicRecourseDynamics/README.md}{\texttt{AlgorithmicRecourseDynamics.jl}}. The package is built on top of \texttt{CounterfactualExplanations.jl} and inherits its extensibility \protect\hyperlink{ref-altmeyer2022CounterfactualExplanations}{{[}5{]}}. That is to say that future researchers should find it relatively easy to replicate, modify and extend the simulation experiments presented here and apply to their own custom counterfactual generators.
+This work has revisited and extended some of the most general and defining concepts underlying the literature on Counterfactual Explanations and, in particular, Algorithmic Recourse. We demonstrate that long-held beliefs as to what defines optimality in AR, may not be suitable in contexts that involves large groups of individuals facing adverse outcomes. Specifically, we run experiments that simulate the application of recourse in practice using various state-of-the-art counterfactual generators and find that all of them induce substantial domain and model shifts. We argue that these shifts should be considered as an expected external cost of individual recourse and call for a paradigm shift from individual to collective recourse in these types of situations. By proposing an adapted counterfactual search objective that incorporates this cost, we make that paradigm shift explicit. We show that this modified objective lends itself to mitigation strategies that can be used to effectively decrease the magnitude of induced domain and model shifts. Through our work we hope to inspire future research on this important topic. To this end we have open-sourced all of our code along with a Julia package - \href{https://anonymous.4open.science/r/AlgorithmicRecourseDynamics/README.md}{\texttt{AlgorithmicRecourseDynamics.jl}}. The package is built on top of \texttt{CounterfactualExplanations.jl} and inherits its extensibility \protect\hyperlink{ref-altmeyer2022CounterfactualExplanations}{{[}5{]}}. That is to say that future researchers should find it relatively easy to replicate, modify and extend the simulation experiments presented here and apply to their own custom counterfactual generators.
 
 \hypertarget{references}{%
 \section*{References}\label{references}}

paper/paper.tex

@@ -1,3 +1,3 @@
 # Concluding Remarks {#conclusion}
 
-This work has revisited and extended some of the most general and defining concepts underlying the literature on Counterfactual Explanations and, in particular, Algorithmic Recourse. We demonstrate that long-held beliefs as to what defines optimality in AR, may not be applicable in situations that involve decisions over scarce resources. Specifically, we run experiments that simulate the application of recourse in practice using various state-of-the-art counterfactual generators and find that all of them induce substantial domain and model shifts. We argue that these shifts should be considered as an expected external cost of individual recourse and call for a paradigm shift from individual to collective recourse. By proposing an adapted counterfactual search objective that incorporates this cost, we make that paradigm shift explicit. We show that this modified objective lends itself to mitigation strategies that can be used to effectively decrease the magnitude of induced domain and model shifts. Through our work we hope to inspire future research on this important topic. To this end we have open-sourced all of our code along with a Julia package - [`AlgorithmicRecourseDynamics.jl`](https://anonymous.4open.science/r/AlgorithmicRecourseDynamics/README.md). The package is built on top of `CounterfactualExplanations.jl` and inherits its extensibility [@altmeyer2022CounterfactualExplanations]. That is to say that future researchers should find it relatively easy to replicate, modify and extend the simulation experiments presented here and apply to their own custom counterfactual generators. 
+This work has revisited and extended some of the most general and defining concepts underlying the literature on Counterfactual Explanations and, in particular, Algorithmic Recourse. We demonstrate that long-held beliefs as to what defines optimality in AR, may not be suitable in contexts that involves large groups of individuals facing adverse outcomes. Specifically, we run experiments that simulate the application of recourse in practice using various state-of-the-art counterfactual generators and find that all of them induce substantial domain and model shifts. We argue that these shifts should be considered as an expected external cost of individual recourse and call for a paradigm shift from individual to collective recourse in these types of situations. By proposing an adapted counterfactual search objective that incorporates this cost, we make that paradigm shift explicit. We show that this modified objective lends itself to mitigation strategies that can be used to effectively decrease the magnitude of induced domain and model shifts. Through our work we hope to inspire future research on this important topic. To this end we have open-sourced all of our code along with a Julia package - [`AlgorithmicRecourseDynamics.jl`](https://anonymous.4open.science/r/AlgorithmicRecourseDynamics/README.md). The package is built on top of `CounterfactualExplanations.jl` and inherits its extensibility [@altmeyer2022CounterfactualExplanations]. That is to say that future researchers should find it relatively easy to replicate, modify and extend the simulation experiments presented here and apply to their own custom counterfactual generators.