Arie's final round of comments

Author: pat-alt

paper/_bookdown.yml

@@ -0,0 +1,3 @@
+language:
+  label:
+    prp: 'Research Question '

paper/paper.pdf

@@ -1,17 +1,10 @@
 # Experiments {#empirical-2}
 
-Using the framework introduced in the previous section we are interested in answering the following research questions in this section:
-
-- Does the repeated implementation of recourse provided by state-of-the-art generators lead to shifts in the domain and model?
-- If so, are any of these dynamics substantial enough to be considered costly to any of the stakeholders involved in real-world automated decision-making processes?
-- Do different counterfactual generators yield significantly different outcomes in this context? Furthermore, is there any heterogeneity with respect to the chosen classifier and dataset?
-- What seem to be drivers of endogenous dynamics in Algorithmic Recourse?
-
 Below we first present our main experimental findings regarding these questions. We conclude this section with a brief recap providing answers to all of these questions.
 
 ## Endogenous Macrodynamics
 
-We start this section off with a few high-level observations. Across all datasets (synthetic and real), classifiers and counterfactual generators we observe either most or all of the following dynamics at varying degrees:
+We start this section off with the key high-level observations. Across all datasets (synthetic and real), classifiers and counterfactual generators we observe either most or all of the following dynamics at varying degrees:
 
 - Statistically significant domain and model shift as measured by MMD. 
 - A deterioration in out-of-sample model performance as measured by the F-Score evaluated on a test sample. In many cases this drop in performance is substantial.
@@ -52,5 +45,5 @@ Out-of-sample model performance also deteriorates across the board and substanti
 knitr::include_graphics("www/real_world_results.png")
 ```
 
-To recap, we can answer our research questions as follows: firstly, endogenous dynamics do emerge in our experiments and we find them substantial enough to be considered costly; secondly, the choice of the counterfactual generator does matter, with Latent Space search generally having a dampening effect. The observed dynamics therefore seem to be driven by a discrepancy between counterfactual outcomes that minimize costs to the individual and outcomes that comply with the data generating process. 
+To recap, we can answer our research questions as follows: firstly, endogenous dynamics do emerge in our experiments (RQ \@ref(prp:shifts)) and we find them substantial enough to be considered costly (RQ \@ref(prp:costs)); secondly, the choice of the counterfactual generator does matter, with Latent Space search generally having a dampening effect (RQ \@ref(prp:het)). The observed dynamics therefore seem to be driven by a discrepancy between counterfactual outcomes that minimize costs to the individual and outcomes that comply with the data generating process (RQ \@ref(prp:drive)). 
 

paper/paper.tex

@@ -26,7 +26,5 @@ Suppose Figure \@ref(fig:poc) relates to an automated decision-making system use
 
 Both of these examples are exaggerated simplifications of potential real-world scenarios, but they serve to illustrate the point that recourse for one single individual may exert negative externalities on other individuals.
 
-To the best of our knowledge this is the first work investigating endogenous macrodynamics in AR. Our contributions to the state of knowledge are as follows: firstly, we posit a compelling argument that calls for a novel perspective on Algorithmic Recourse extending our focus from single individuals to groups. Secondly, we introduce an experimental framework extending previous work by @altmeyer2022CounterfactualExplanations, which enables us to study macrodynamics of Algorithmic Recourse through simulations that can be fully parallelized. Thirdly, we use this framework to provide a first in-depth analysis of endogenous recourse dynamics induced by various popular counterfactual generators including @wachter2017counterfactual, @schut2021generating, @joshi2019towards, @mothilal2020explaining and @antoran2020getting. To this end we propose a number of novel evaluation metrics that can be used to quantify and benchmark the macrodynamics introduced by the different generators. Finally, we also discuss what drives endogenous dynamics and propose strategies to mitigate them.
-
-The remainder of the paper is structured as follows: Section \@ref(related) places our work in the broader context of related literature. Section \@ref(method) posits a generalized framework for gradient-based Algorithmic Recourse and introduces the notion of hidden external costs. Section \@ref(method-2) sets out our experimental framework for modeling endogenous macrodynamics in AR. Section \@ref(empirical) presents our experimental setup, including the data, classifiers and counterfactual generators that we have employed. Section \@ref(empirical-2)  presents the results for synthetic and real-world datasets along with evidence for the effectiveness of various proposed mitigation strategies. Our findings are then discussed in the broader context of the literature in Section \@ref(discussion), before pointing to some of the limitations of our work as well as avenues for future research in Section \@ref(limit). Finally, Section \@ref(conclusion) concludes.
+To the best of our knowledge this is the first work investigating endogenous macrodynamics in AR. Our contributions to the state of knowledge are as follows: firstly, we posit a compelling argument that calls for a novel perspective on Algorithmic Recourse extending our focus from single individuals to groups (Section \@ref(related) and \@ref(method)). Secondly, we introduce an experimental framework extending previous work by @altmeyer2022CounterfactualExplanations, which enables us to study macrodynamics of Algorithmic Recourse through simulations that can be fully parallelized (\@ref(method-2)). Thirdly, we use this framework to provide a first in-depth analysis of endogenous recourse dynamics induced by various popular counterfactual generators including @wachter2017counterfactual, @schut2021generating, @joshi2019towards, @mothilal2020explaining and @antoran2020getting (\@ref(empirical) and \@ref(empirical-2)). Fourthly, given that we find substantial impact of recourse, we propose key mitigation strategies and measure their impact experimentally (Section \@ref(mitigate)). Finally, we discuss our findings in the broader context of the literature in Section \@ref(discussion), before pointing to some of the limitations of our work as well as avenues for future research in Section \@ref(limit). Section \@ref(conclusion) concludes.
 

paper/sections/empirical_2.rmd

@@ -1,6 +1,24 @@
 # Modeling Endogenous Macrodynamics in Algorithmic Recourse {#method-2}
 
-In the following we describe the framework we propose for modeling and analysing endogenous macrodynamics in Algorithmic Recourse. We first describe the basic simulations that were generated to produce the findings in this work and also constitute the core of [`AlgorithmicRecourseDynamics.jl`](https://anonymous.4open.science/r/AlgorithmicRecourseDynamics/README.md) - the Julia package we introduced earlier. The remainder of this section then introduces various evaluation metrics that can be used to benchmark different counterfactual generators with respect to how they perform in the dynamic setting. 
+In the following we describe the framework we propose for modeling and analysing endogenous macrodynamics in Algorithmic Recourse. We introduce this framework with the ambition to shed light on the following research questions:
+
+::: {.proposition #shifts name="Endogenous Shifts"}
+Does the repeated implementation of recourse provided by state-of-the-art generators lead to shifts in the domain and model?
+:::
+
+::: {.proposition #costs name="Costs"}
+If so, are any of these dynamics substantial enough to be considered costly to any of the stakeholders involved in real-world automated decision-making processes?
+:::
+
+::: {.proposition #het name="Heterogeneity"}
+Do different counterfactual generators yield significantly different outcomes in this context? Furthermore, is there any heterogeneity with respect to the chosen classifier and dataset?
+:::
+
+::: {.proposition #drive name="Drivers"}
+What are drivers of endogenous dynamics in Algorithmic Recourse?
+:::
+
+Below we first describe the basic simulations that were generated to produce the findings in this work and also constitute the core of [`AlgorithmicRecourseDynamics.jl`](https://anonymous.4open.science/r/AlgorithmicRecourseDynamics/README.md) - the Julia package we introduced earlier. The remainder of this section then introduces various evaluation metrics that can be used to benchmark different counterfactual generators with respect to how they perform in the dynamic setting. 
 
 ## Simulations {#method-2-experiment}
 

paper/sections/introduction.rmd

@@ -1,14 +1,20 @@
-# Potential Mitigation Strategies {#mitigate}
+# Mitigation Strategies and Experiments {#mitigate}
 
-Having established in the previous section that endogenous macrodynamics in AR are substantial enough to warrant our attention, in this section we introduce and test a number of potential mitigation strategies. All of them essentially boil down to one simple principle: to avoid substantial domain and model shifts, the generated counterfactuals should comply as much as possible with the true data generating process. This principle is really at the core of Latent Space generators, and hence it is not surprising that we have found these types of generators to perform comparably well in the previous section. But as we have mentioned earlier, generators that rely on separate generative models carry an additional computational burden and - perhaps more importantly - their performance hinges on the performance of said generative models. Fortunately, it turns out that we can use a number of other, much simpler strategies, which we will discuss now. 
+Having established in the previous section that endogenous macrodynamics in AR are substantial enough to warrant our attention, in this section we ask ourselves:
+
+::: {.proposition #mitigate name="Mitigation Strategies"}
+What are potential mitigation strategies with respect to endogenous macrodynamics in AR?
+:::
+
+We proposed and test a number of simple mitigation strategies. All of them essentially boil down to one simple principle: to avoid substantial domain and model shifts, the generated counterfactuals should comply as much as possible with the true data generating process. This principle is really at the core of Latent Space generators, and hence it is not surprising that we have found these types of generators to perform comparably well in the previous section. But as we have mentioned earlier, generators that rely on separate generative models carry an additional computational burden and - perhaps more importantly - their performance hinges on the performance of said generative models. Fortunately, it turns out that we can use a number of other, much simpler strategies, which we will discuss now. 
 
 ## More Conservative Decision Thresholds
 
-The most obvious and trivial mitigation strategy is to simply choose a higher decision threshold $\gamma$. This threshold determines when a proposed counterfactual should be considered as valid. Under $\gamma=0.5$, counterfactuals will end up near the decision boundary by construction. Since this is the region of maximal aleotoric uncertainty, the classifier is bound to be thrown off. By simply setting a more conservative decision threshold, we can avoid this issue to some extent. A potential drawback of this approach is that a classifier with high decisiveness may classify samples with high confidence even far away from the training data. 
+The most obvious and trivial mitigation strategy is to simply choose a higher decision threshold $\gamma$. This threshold determines when a proposed counterfactual should be considered as valid. Under $\gamma=0.5$, counterfactuals will end up near the decision boundary by construction. Since this is the region of maximal aleotoric uncertainty, the classifier is bound to be thrown off. By simply setting a more conservative decision threshold, we can avoid this issue to some extent. A drawback of this approach is that a classifier with high decisiveness may classify samples with high confidence even far away from the training data. 
 
 ## Classifier Preserving ROAR
 
-Another potential strategy draws inspiration from ROAR @upadhyay2021towards: to preserve the classifier, we propose to simply explicitly penalize the loss it incurs when evaluated on the counterfactual $x^\prime$ at given parameter values. Recall that $\text{extcost}(\cdot)$ denotes what we had defined as the external cost in Equation \@ref(eq:collective). Then formally we let
+Another strategy draws inspiration from ROAR @upadhyay2021towards: to preserve the classifier, we propose to simply explicitly penalize the loss it incurs when evaluated on the counterfactual $x^\prime$ at given parameter values. Recall that $\text{extcost}(\cdot)$ denotes what we had defined as the external cost in Equation \@ref(eq:collective). Then formally we let
 
 \begin{equation}
 \begin{aligned}
@@ -28,7 +34,7 @@ Yet another strategy simply extends Wachter as follows: instead of only penalizi
 \end{aligned}
 \end{equation}
 
-Once again we can putting this in the context of Equation \@ref(eq:collective), the former penalty can be thought of here as the private cost incurred by the individual, while the latter reflects the external cost incurred by other individuals. Higher choices of $\lambda_2$ relative to $\lambda_1$ will lead counterfactuals to gravitate towards the specified point $\bar{x}$ in the target domain. In the remainder of this paper we will therefore refer to this approach as **Gravitational** generator, when we investigate its potential usefulness for mitigating endongenous macrodynamics^[Note that despite the naming convention our goal here is not to provide yet another counterfactual generator, but merely investigate the most simple penalty we can think of with respect to its effectiveness.]. 
+Once again we can putting this in the context of Equation \@ref(eq:collective), the former penalty can be thought of here as the private cost incurred by the individual, while the latter reflects the external cost incurred by other individuals. Higher choices of $\lambda_2$ relative to $\lambda_1$ will lead counterfactuals to gravitate towards the specified point $\bar{x}$ in the target domain. In the remainder of this paper we will therefore refer to this approach as **Gravitational** generator, when we investigate its usefulness for mitigating endongenous macrodynamics^[Note that despite the naming convention our goal here is not to provide yet another counterfactual generator, but merely investigate the most simple penalty we can think of with respect to its effectiveness.]. 
 
 Figure \@ref(fig:mitigation) shows an illustrative example that demonstrates the differences in counterfactual outcomes when using the various mitigation strategies compared to the baseline approach, that is Wachter with $\gamma=0.5$: choosing a higher decision threshold pushes the counterfactual a little further into the target domain; this effect is even stronger for ClapROAR; finally, using the Gravitational generator the counterfactual ends up all the way inside the target domain in the neighbourhood of $\bar{x}$^[In order for the Gravitational generator and ClapROAR to work as expected, one needs to ensure that counterfactual search continues, independent of the threshold probability $\gamma$.].
 

paper/sections/methodology_2.rmd

@@ -22,7 +22,8 @@ Many approaches for the generation of Algorithmic Recourse have been described i
 
 Efforts so far have largely been directed at improving the quality of Counterfactual Explanations within a static context: given some pre-trained classifier $M: \mathcal{X} \mapsto \mathcal{Y}$, we are interested in generating one or multiple meaningful Counterfactual Explanations for some individual characterized by $x$. The ability of Counterfactual Explanations to handle dynamics like data and model shifts remains a largely unexplored research challenge at this point [@verma2020counterfactual]. We have been able to identify only one recent work that considers the implications of **exogenous** domain and model shifts in the context of AR [@upadhyay2021towards]. Exogenous shifts are strictly of external origin. For example, they might stem from data correction, temporal shifts or geospatial changes [@upadhyay2021towards]. The authors of [@upadhyay2021towards] propose ROAR - a framework for Algorithmic Recourse that evidently improves robustness to such exogenous shifts.
 
-As mentioned earlier, research has so far also generally focused on generating counterfactuals for single individuals or instances. We have been able to identify only one existing work that investigates black-box model behavior towards a group of individuals [@carrizosa2021generating]. The authors propose an optimization framework that generates collective counterfactuals. We provide a motivation for doing so from the perspective of endogenous macrodynamics of Algorithmic Recourse. 
+As mentioned earlier, research has so far also generally focused on generating counterfactuals for single individuals or instances. We have been able to identify only one existing work that investigates black-box model behavior towards a group of individuals [@carrizosa2021generating]. The authors propose an optimization framework that generates collective counterfactuals. We provide a motivation for doing so from the perspective of endogenous macrodynamics of Algorithmic Recourse.
+
 
 ## Domain and Model Shifts {#related-shifts}