further progress on extended abstract

Author: pat-alt

dev/submissions/aies/extended_abstract/extended_abstract.pdf

@@ -1,5 +1,6 @@
 ---
 title: Dynamics in Algorithmic Recourse
+subtitle: Trustworthy Artificial Intelligence for Finance and Economics
 format:
     pdf:
         documentclass: acmconf
@@ -8,24 +9,40 @@ format:
         keep-tex: true
 ---
 
-Recent advances in artificial intelligence (AI) have propelled its adoption in domains outside of computer science including health care, bioinformatics and genetics. In finance, economics and other social sciences, applications of AI are still relatively limited. Decision-making in these fields has traditionally been guided by Generalized Linear Models (GLM), which are theoretically founded, interpretable and often sufficient to model relationships between variables. Model interpretability is crucial in the social sciences context, because inference is typically at least as important as predictive performance. Decision-makers in the social sciences are also typically required to explain their decisions to human stakeholders: central bankers, for example, are held accountable by the public for the policies they decide on. It is therefore not surprising that practitioners and academics in these fields are reluctant to adopt AI technologies that ultimately cannot be trusted. Deep learning models, for example, are generally considered as black boxes and therefore difficult to apply in a context that demands explanations. 
+## Introduction
 
-In my research I explore and develop methodologies that improve the trustworthiness of AI. I would like to understand how we can unlock the enormous potential of AI without sacrificing the human aspect of decision-making in finance and economics. My work so far has focused primarily on counterfactual explanations, algorithmic recourse and probabilistic machine learning. Counterfactual explanations are intuitive, largely model-agnostic and straight-forward to implement. They are also intrinsically linked to the potential outcome framework for causal inference and therefore should be somewhat familiar to social scientists. Counterfactual explanations that involve realistic and actionable changes can be used for the purpose of algorithmic recourse to help individuals facing adverse decisions. Probabilistic machine learning can be leveraged in this context and more generally facilitates inference and interpretability. It is also closely related to Bayesian statistics, which has played an important role in both finance and economics for many years. 
+Recent advances in artificial intelligence (AI) have propelled its adoption in domains outside of computer science including health care, bioinformatics and genetics. In finance, economics and other social sciences, applications of AI are still relatively limited. Decision-making in these fields has traditionally been guided by Generalized Linear Models (GLM), which are theoretically founded, interpretable and often sufficient to model relationships between variables. Model interpretability is crucial in the social sciences context, because inference is typically at least as important as predictive performance. Decision-makers in the social sciences are also typically required to explain their decisions to human stakeholders: central bankers, for example, are held accountable by the public for the policies they decide on. It is therefore not surprising that practitioners and academics in these fields are reluctant to adopt AI technologies that ultimately cannot be trusted. Deep learning models, for example, are generally considered as black boxes and therefore difficult to apply in a context that demands explanations. This PhD project is focused on exploring and developing methodologies that improve the trustworthiness of AI and thereby enable its application in Finance and Economics. 
 
-In the following (@sec-main), I will first briefly present one particular research question I have explored during the first months of my PhD: how do counterfactual explanations handle dynamics? I will also briefly present related projects I have worked on (@sec-related) and ideas for future projects (@sec-future).
+The remainder of this extended abstract is structured as follows: @sec-main presents one of the research questions I have investigated during the first months of my PhD: how do counterfactual explanations handle dynamics? @sec-related places this work in the broader context of Trustworthy AI for Finance and Economics.
 
 ## Dynamics in Algorithmic Recourse {#sec-main}
 
-Existing work on counterfactual explanations and algorithmic recourse has largely been limited to the following static setting: given some classifier $M: \mathcal{X} \mapsto \mathcal{Y}$ we are interested in finding close [@wachter2017counterfactual], actionable [@ustun2019actionable], plausible [@joshi2019towards, @antoran2020getting, @schut2021generating], sparse [@schut2021generating], diverse [@mothilal2020explaining] and ideally causally founded counterfactual explanations [@karimi2021algorithmic] for some individual $x$. The ability of counterfactual explanations to handle dynamics like data and model shifts remains a largely unexplored research challenge at this point [@verma2020counterfactual]. Only one recent work considers the implications of **exogenous** domain shifts on the validity of recourse [@upadhyay2021towards]. The authors propose a simple minimax objective, that minimizes the counterfactual loss function for a maximal domain and model shift. They show that their approach yields more robust counterfactuals than existing approaches. In my project I investigate **endogenous** domain and model shifts, i.e. shifts that occur as algorithmic recourse is actually impemented by a proportion of individuals. Preliminary findings indicate that individuals who receive and implement algorithmic recourse end up forming a distinct subgroup inside the target class, which may leave them vulnerable to discrimination (@fig-dynamics). This is a work-in-progress that I would like to present and discuss at AIES. 
+**Counterfactual explanations** (CE) explain how inputs into a model need to change for it to produce different outputs. They are intuitive, simple and intrinsically linked to the potential outcome framework for causal inference, which social scientists are familiar with. Counterfactual explanations that involve realistic and actionable changes can be used for the purpose of **Algorithmic Recourse** (AR) to help individuals facing adverse decisions. An example relevant to the Finance and Econonomics domain is consumer credit: in this context AR can be used to guide individuals to improve their credit worthiness, should they have previously been denied access to credit based on a black-box decision-making system . 
+
+Existing work on CE and AR has largely been limited to the static setting: given some classifier $M: \mathcal{X} \mapsto \mathcal{Y}$ we are interested in finding close [@wachter2017counterfactual], actionable [@ustun2019actionable], plausible [@joshi2019towards, @antoran2020getting, @schut2021generating], sparse [@schut2021generating], diverse [@mothilal2020explaining] and ideally causally founded counterfactual explanations [@karimi2021algorithmic] for some individual $x$. The ability of counterfactual explanations to handle dynamics like data and model shifts remains a largely unexplored research challenge at this point [@verma2020counterfactual]. Only one recent work considers the implications of **exogenous** domain shifts on the validity of recourse [@upadhyay2021towards]. The authors propose a simple minimax objective, that minimizes the counterfactual loss function for a maximal domain and model shift. They show that their approach yields more robust counterfactuals than existing approaches. 
+
+This project investigates **endogenous** domain and model shifts, that is shifts that occur when AR is actually impemented by a proportion of individuals and the classifier is updated in response. @fig-dynamics illustrates this idea for a binary problem involving a probabilistic classifier and a greedy counterfactual generator proposed by @schut2021generating: AR leads to a domain shift, which in turn causes a drastic model shift. As this game of implementing AR and updating the classifier is repeated, individuals who receive and implement algorithmic recourse end up forming a distinct subgroup inside the target class, which may leave them vulnerable to discrimination. Through future experiments we want to investigate if this phenomenon is robust across different benchmark datasets and counterfactual generators. 
 
 ![PLACEHOLDER: The dynamics of algorithmic recourse.](www/dynamics.png){#fig-dynamics fig.pos="h" width=250px}
 
-## Related Projects {#sec-related}
+## Related and Future Work {#sec-related}
+
+### Benchmarking CE in Julia
 
 Alongside my research I have developed open-source implementations related to explainable AI. [CounterfactualExplanations.jl](https://www.paltmeyer.com/CounterfactualExplanations.jl/stable/) is a Julia package that can be used to generate counterfactual explanations for models developed and trained not only in Julia, but also in other popular programming languages like Python and R. I have recently submitted the package along with a companion paper as a proposal for a main talk at [JuliaCon](https://juliacon.org/2022/). [BayesLaplace.jl](https://www.paltmeyer.com/BayesLaplace.jl/dev/) is a small Julia package that can be used to recover Bayesian representations of deep neural networks through Laplace approximation in a post-hoc manner. It is inspired by a recent paper [@daxberger2021laplace] and has also been submitted to JuliaCon. Finally, [deepvars](https://github.com/pat-alt/deepvars) is an R package that implements an approach towards vector autoregression that leverages deep learning. This was originally my master's thesis and later presented at the NeurIPS 2021 MLECON workshop. I have also published several blog posts on explainable AI and probabilisitic ML in an effort to make my research accessible to a broad audience. 
 
-## Future Projects {#sec-future}
+### Probabilistic methods for realistic counterfactual explanations
+
+Probabilistic machine learning can be leveraged in this context and more generally facilitates inference and interpretability. It is also closely related to Bayesian statistics, which has played an important role in both finance and economics for many years. 
+
+To ensure that the generated explanations are realistic it is important to understand which input-output pairs are likely and which are not. To quantify their joint likelihood, previous work has either relied on generative models or restricted the analysis to probabilistic models that incorporate uncertainty in their predictions. While the former approach is more versatile since it is applicable to both deterministic and probabilistic models, the latter is computationally much more efficient. In my work I want to explore how recent advances in post-hoc uncertainty quantification can be leveraged to generate realistic and unambiguous counterfactual explanations for any model.
+
+### CE for time series
+
+Data sets in finance and economics typically involve time series data. Therefore, I am naturally interested in the application of explainable AI to sequential data, an area which has so far not been explored extensively. In the future, I want to work on counterfactual explanations for time series models. I am also interested in seeing if and how Laplace approximation can be used for Bayesian deep learning with time series data. I hope that the findings from both of these projects can ultimately be used to build complex but interpretable time series models for classification and forecasting in finance and economics. 
+
+### Explainable black-box models for time series
 
-Data sets in finance and economics typically involve time series data. Therefore, I am naturally interested in the application of explainable AI to sequential data, an area which has so far not been explored extensively. In the future, I want to work on counterfactual explanations for time series models. I am also interested in seeing if and how Laplace approximation can be used for Bayesian deep learning with time series data. I hope that the findings from both of these projects can ultimately be used to build complex but interpretable time series models for classification and forecasting in finance and economics. For example, I would like to leverage effortless Bayesian deep learning to make our proposed Deep Vector Autoregression model explainable. 
+For example, I would like to leverage effortless Bayesian deep learning to make our proposed Deep Vector Autoregression model explainable. 
 
 ## References