work on online appendix. Just need to finish mitigation strats and sort out tables.

Author: pat-alt

README.qmd

@@ -16,6 +16,7 @@ repository:
 
 - [Paper](paper/paper.pdf)
 - [Notebooks](dev/notebooks/)
+- [Supplementary Appendix](build/dev/notebooks/appendix.html) (download the HTML and view in browser)
 - [Artifacts](https://github.com/pat-alt/AlgorithmicRecourseDynamics.jl/releases/tag/artifacts) (including data and experimental results)
 
 In this work we investigate what happens if Algorithmic Recourse is actually implemented by a large number of individuals. The chart below illustrates what we mean by Endogenous Macrodynamics in Algorithmic Recourse: (a) we have a simple linear classifier trained for binary classification where samples from the negative class ($y=0$) are marked in blue and samples of the positive class ($y=1$) are marked in orange; (b) the implementation of AR for a random subset of individuals leads to a noticable domain shift; (c) as the classifier is retrained we observe a corresponding model shift; (d) as this process is repeated, the decision boundary moves away from the target class.

_freeze/dev/notebooks/appendix/execute-results/html.json

@@ -0,0 +1,13 @@
+project:
+  title: "AlgorithmicRecourseDynamics.jl"
+  output-dir: build
+  execute-dir: project
+filters:
+  - lua/abstract-to-meta.lua
+  - quarto
+bibliography: https://raw.githubusercontent.com/pat-alt/bib/main/bib.bib
+execute:
+  freeze: auto  # re-render only when source changes
+
+
+

_quarto.yml

@@ -15,14 +15,14 @@ This is a supplementary appendix to the research paper **Endogenous Macrodynamic
 
 # Experimental Results {#sec-results}
 
-{{< include ../dev/notebooks/experiments/synthetic.qmd >}}
+{{< include experiments/synthetic.qmd >}}
 
-{{< include ../dev/notebooks/experiments/real_world.qmd >}}
+{{< include experiments/real_world.qmd >}}
 
-{{< include ../dev/notebooks/experiments/mitigation_strategies.qmd >}}
+{{< include experiments/mitigation_strategies.qmd >}}
 
 # Generators {#sec-generators}
 
-{{< include ../dev/notebooks/generators/gravitational_generator.qmd >}}
+{{< include generators/gravitational_generator.qmd >}}
 
-{{< include ../dev/notebooks/generators/clap_roar_generator.qmd >}}
+{{< include generators/clap_roar_generator.qmd >}}

build/dev/notebooks/appendix.html

@@ -1,10 +1,12 @@
 ## Mitigation Strategies
 
 ```{julia}
+#| eval: true
 using Pkg; Pkg.activate("dev")
 ```
 
 ```{julia}
+#| eval: true
 include("dev/utils.jl")
 using AlgorithmicRecourseDynamics
 using CounterfactualExplanations, Flux, Plots, PlotThemes, Random, LaplaceRedux, LinearAlgebra
@@ -28,7 +30,7 @@ generators = Dict(
 )
 ```
 
-## Synthetic
+### Synthetic
 
 ```{julia}
 max_obs = 1000

dev/.gitignore

@@ -1,16 +1,18 @@
 ## Real-World Data
 
 ```{julia}
+#| eval: true
 using Pkg; Pkg.activate("dev")
 ```
 
 ```{julia}
+#| eval: true
 include("dev/utils.jl")
 using AlgorithmicRecourseDynamics
 using CounterfactualExplanations, Flux, Plots, PlotThemes, Random, LaplaceRedux, LinearAlgebra
 theme(:wong)
 output_path = output_dir("real_world")
-www_path = www_dir("real_world")
+www_path = www_dir("real_world");
 ```
 
 ```{julia}
@@ -58,7 +60,7 @@ experiments = set_up_experiments(
 )
 ```
 
-## Running Experiment
+### Experiment
 
 ```{julia}
 n_evals = 5
@@ -78,6 +80,8 @@ results = run_experiments(
 Serialization.serialize(joinpath(output_path,"results.jls"),results)
 ```
 
+### Plots
+
 ```{julia}
 using Serialization
 results = Serialization.deserialize(joinpath(output_path,"results.jls"))
@@ -97,8 +101,50 @@ for (data_name, res) in results
 end
 ```
 
+#### Line Charts
+
+@fig-real-line shows the evolution of the evaluation metrics over the course of the experiment.
+
+```{julia}
+#| eval: true
+#| fig-cap: "Line Charts"
+#| fig-subcap: 
+#|      - "California Housing"
+#|      - "Credit Default"
+#|      - "GMSC"
+#| layout-ncol: 1
+#| label: fig-real-line
+img_files = readdir(www_path)[contains.(readdir(www_path),"line_chart")]
+img_files = joinpath.(www_path,img_files)
+for img in img_files
+    display(load(img))
+end
+```
+
+#### Error Bar Charts
+
+@fig-real-error shows the evaluation metrics at the end of the experiments.
+
+```{julia}
+#| eval: true
+#| fig-cap: "Error Bar Charts"
+#| fig-subcap: 
+#|      - "California Housing"
+#|      - "Credit Default"
+#|      - "GMSC"
+#| layout-ncol: 1
+#| label: fig-real-error
+img_files = readdir(www_path)[contains.(readdir(www_path),"errorbar_chart")]
+img_files = joinpath.(www_path,img_files)
+for img in img_files
+    display(load(img))
+end
+```
+
 ### Chart in paper
 
+@fig-real-paper shows the chart that went into the paper.
+
 ```{julia}
 using DataFrames, Statistics
 model_ = :FluxEnsemble
@@ -156,6 +202,13 @@ img = Images.load(rcopy(R"temp_path"))
 Images.save(joinpath(www_path,"paper_real_world_results.png"), img)
 ```
 
+```{julia}
+#| label: fig-real-paper
+#| fig-cap: "Chart in paper"
+#| eval: true
+Images.load(joinpath(www_path,"paper_real_world_results.png"))
+```
+