Small experiments on when an agent should gather information before acting.
The repository starts with a two-door clue task, then moves through evidence valuation, model misspecification, finite-horizon Bayesian planning, and a controlled Active Inference comparison. Each stage keeps the assumptions visible instead of introducing the full framework at once.
A hidden reward is behind one of two doors. The agent can act immediately or inspect a noisy clue first.
For belief
The simple epistemic policy trades task reward against clue cost and information gain. This is active-inference-inspired, not a full implementation of variational or expected free energy.
Run:
python -m epistemic_action.experiment
python -m epistemic_action.plotThe second experiment asks:
Is the most informative observation also the most useful observation?
A synthetic transaction has a hidden state: legitimate or suspicious. Before approving or rejecting it, a policy may acquire at most one evidence source. Sources differ in sensitivity, specificity, and acquisition cost.
The benchmark compares:
greedy: act immediatelyrandom: acquire one random sourceinformation_gain: maximize expected entropy reductionvalue_of_information: acquire evidence only when expected decision improvement exceeds cost
For source
Run:
python -m epistemic_action.evidence_experimentThe source parameters are synthetic and are not estimates for real fraud or finance systems.
The one-step benchmark assumes evidence is correctly specified. The stress test deliberately violates that assumption.
Two evidence channels have the same marginal accuracy but share a signal draw with probability
The benchmark compares:
-
single_source: ignore the second observation -
naive_independent: use both observations but assume$rho=0$ -
correlation_aware: use the configured dependence strength -
oracle: use the true dependence and true source accuracy
Run:
python -m epistemic_action.stress_experimentIt separately varies true source accuracy and assumed source accuracy so dependence misspecification and calibration error can be studied independently.
The next experiment asks:
Can an observation be worth acquiring only because it changes which future observation becomes useful?
The agent may repeatedly choose between acting, querying another unused source, or escalating at a fixed cost.
The benchmark compares:
greedy: no evidence acquisitioninformation_gain: maximize expected entropy reduction, ignoring costmyopic_voi: one-step decision valuelookahead: exact finite-horizon Bayesian planning over future signals
The corrected default option-value condition is:
P(suspicious) = 0.05
false approve loss = 5.0
false reject loss = 1.0
escalation cost = 0.40
screen accuracy = 0.70
screen cost = 0.05
review accuracy = 0.95
review cost = 0.20
Under this condition neither source has positive one-step decision value. Acting immediately has expected loss 0.250. A two-step Bayesian planner still selects screen, because a clear result lets it stop while a flagged result can make review worth acquiring. Its exact expected loss is 0.212.
This is the intended non-myopic option-value case.
Run:
python -m epistemic_action.sequential_experimentThis is still an exact Bayesian planner, not Active Inference.
The fifth experiment keeps the same sequential task and adds a transparent discrete Active Inference formulation.
active_inference.py contains:
- a factorized A/B/C/D-style generative model
- expected hidden-state information gain in nats
- an open-loop
standard_efeplanner - an observation-contingent
sophisticated_efeplanner
The hidden transaction state is static. A controlled context factor records whether the current action is screen, review, approve, reject, or escalate. Evidence outcomes, terminal outcomes, and query costs are represented as separate observation modalities. Source availability is fully observed task memory and is enforced by the planner rather than represented as a hidden state.
Task losses are mapped into log-preference units with a positive preference_precision parameter. The scaling is swept rather than tuned to one favorable result.
The comparison includes:
bayes_lookahead: exact finite-horizon expected-utility planningstandard_efe: open-loop negative-EFE policy scoringsophisticated_efe: recursive observation-contingent negative-EFE planning
Run:
python -m epistemic_action.active_inference_experimentThe distinction between the two EFE planners is deliberate. The standard form propagates predicted beliefs without conditioning later policy choices on anticipated observations. The sophisticated form branches over possible observations, updates beliefs, and then chooses the next action.
At the corrected default condition and preference precision 5, the exact Bayesian and sophisticated-EFE planners both select screen, while standard EFE stops. At low preference precision such as 0.5, the epistemic term can dominate enough that sophisticated EFE selects the expensive review first. That is treated as a failure mode to measure, not a desired result.
This module is a benchmark-specific, inspectable implementation. It is not a reimplementation of pymdp.
A single diagnostic condition is not enough to support a research claim. disagreement_experiment.py therefore sweeps the task and computes exact expected trajectory loss rather than relying on Monte Carlo estimates.
It varies:
- prior suspicious probability
- screen accuracy and cost
- review accuracy and cost
- Active Inference preference precision
For each condition it records:
- first action
- exact expected total loss
- exact expected query count
- exact expected evidence cost
- regret relative to finite-horizon Bayesian planning
The compared planners are:
myopic_voibayes_lookaheadstandard_efesophisticated_efe
Run the default campaign with:
python -m epistemic_action.disagreement_experimentThe default grid contains 1,350 unique conditions and writes 5,400 planner rows to results/disagreement.csv.
The campaign is designed to answer two separate questions:
- where does non-myopic planning matter?
- where does the epistemic term create useful exploration versus unnecessary exploration?
The Bayesian planner is the utility-optimal reference only for the correctly specified synthetic model used by this campaign. Later misspecification experiments should not treat it as an oracle for the real world.
Python 3.12+ is required.
python -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[dev,plot]'
pytestsrc/epistemic_action/
├── agents.py # original clue policies and information gain
├── environment.py # two-door clue environment
├── experiment.py # clue reliability/cost sweep
├── plot.py # clue experiment plots
├── evidence.py # binary evidence model, Bayes updates, VoI
├── policies.py # one-step evidence-selection baselines
├── evidence_experiment.py # reproducible evidence benchmark
├── stress.py # correlated evidence model
├── stress_experiment.py # correlation/calibration stress sweep
├── sequential.py # exact finite-horizon Bayesian planner
├── sequential_experiment.py # sequential acquisition benchmark
├── active_inference.py # A/B/C/D model and EFE planners
├── active_inference_experiment.py # Bayesian vs EFE comparison
└── disagreement_experiment.py # exact disagreement/regret campaign
tests/
The repository is now set up to evaluate, rather than merely discuss, Active Inference.
The next questions are:
- At what preference precision does epistemic value improve or hurt task utility?
- When does standard open-loop EFE fail because useful future actions depend on future observations?
- When does sophisticated EFE recover the same behavior as exact Bayesian planning, and when does it deliberately differ?
- How do those conclusions change under source correlation, calibration error, and prior shift?
- Does the same behavior reproduce in
pymdprather than only in this transparent reference implementation?
The goal is not to make Active Inference win. The goal is to identify which assumptions create useful epistemic behavior and what utility or computational cost comes with it.
MIT