[MaxGhenis]

Fixes #579. Stacked on #581 (base fix-set-input-helper-order), whose helper it reuses. The diff below is this PR's change only.

What this fixes

set_input for one of a variable's own periods (a month of a monthly variable, a year of a yearly one) replaced the stored value. It left what calculate had derived from the old value at periods of another size: the sum calculate_add stores for a monthly flow over a year, and the twelfth calculate_divide stores for a yearly flow at a month. A later calculate returned the old result, so it depended on whether that period was read before the input was set. The audit's witness: annual flow read as 0, then January set to 1, and the annual read stays 0.

#581 already drops such calculated values when a set_input helper spreads a longer period (_drop_calculated_overlapping). #579 is the remaining path, an input that needs no helper. This PR applies the same rule there:

• Holder.set_input, after storing an own-period input of a non-ETERNITY variable, calls _drop_calculated_over_own_period. That drops what the simulation calculated for the variable at overlapping periods, on the input's branch and the branches it reads. Inputs are kept.
• It also evicts calculate's fast-cache entries for the variable at overlapping periods, the input's own period included. The fast cache never holds the sums and twelfths themselves (I checked; calculate_add and calculate_divide store only in the holder). The eviction matters for inputs set on the holder directly, where Simulation.set_input's exact-key pop never runs.
• It does not touch Simulation.set_input, whose fast-cache block draft Drop every contained period from the fast cache when deleting or setting a period #566 rewrites.

#581's property test_inputs_do_not_depend_on_what_was_calculated_before excluded other-size reads for variables given an own-period input, because of #579. This PR removes that exclusion.

Invariants

• For any sequence of inputs (own-period and helper), any calculate calls in between on the simulation and up to two nested branches, and memory or disk storage: every plain read, at the variable's own periods and at periods of another size, equals the same read on a simulation that calculated nothing first. This is Count only inputs as already set when a set_input helper splits a longer period #581's property with the set_input for one period leaves the variable's cached sum or twelfth over an overlapping period in place #579 exclusion removed.
• An own-period input never drops another input.

Tests

• New test_own_period_input_overlapping_cache.py, 7 examples:
  • issue set_input for one period leaves the variable's cached sum or twelfth over an overlapping period in place #579's table, in memory and on disk;
  • the audit witness, and a sum over two years;
  • an input for another month kept;
  • a branch input updating the branch's sum while the parent keeps its own;
  • an input set through the holder.
• test_set_input_helper_order_property.py: the set_input for one period leaves the variable's cached sum or twelfth over an overlapping period in place #579 exclusion removed.

Before, on #581's head 917f0c23:

test_own_period_input_overlapping_cache.py:  7 failed (e.g. annual [0, 0] after setting January to [1, 1])
test_set_input_helper_order_property.py:     1 failed, 2 passed
  shrunk: requests_by_level=[[], [], [('flow_m', '2013')]], depth=2, one own-period input:
  AssertionError: (('flow_m', '2013'), array([0., 0.]), array([0., 1.]))

After:

test_own_period_input_overlapping_cache.py:  7 passed
test_set_input_helper_order_property.py:     3 passed

Ruff format and ruff check pass. I ran these files one at a time, because the shared host is saturated. This repository's PR workflow runs only for PRs into master, so CI runs once this PR is retargeted, after #581 lands or after this PR is merged into #581's branch.

Impact

Outputs can change wherever an own-period input is set after the simulation has calculated a sum or twelfth of that variable. policyengine-us marginal_tax_rate, for example, sets inputs on branches after calculating. Dataset loading sets inputs before any calculation, so there is nothing to drop there.

PE-US A/B (core 917f0c23 vs this head, same policyengine-us checkout and data, 2026; household_net_income, income_tax, state_income_tax, snap, marginal_tax_rate): pending. It is queued behind the shared microsim lock, and I will add results here when it finishes.

Known limits (from the independent review, which approved at 1fa8b034)

• Inherited, not introduced here: the rule trusts the simulation's record of inputs, _user_input_keys, and Simulation.clone (so get_branch) still shares that set with the parent. An input the parent sets after branching can therefore make the branch keep a stale calculated value. Executed witness:

  • read flow_m for 2013 (0);
  • branch;
  • the parent sets twelve months as month:2013-01:12;
  • the branch sets May to 50.

  The branch's annual then reads [0, 0] instead of [50, 0]. Giving each clone its own copy of the record fixes it. That belongs with the record's own fixes (Keep the record of set_input values in step with storage #561), not this PR.

• Cost: each own-period input scans the variable's stored keys, so N successive inputs to one variable check N(N+1)/2 keys. The reviewer measured 0.012 s for 100 inputs, 0.040 s for 200 and 0.152 s for 400. Dataset loading sets few periods per variable and starts with an empty fast cache, so that is negligible; long panels would merit a cheaper index.

Landing

Either merge this into #581's branch, so #581 (decision d899) lands both, or land it after #581. GitHub retargets it to master when #581's branch is deleted. #581 currently conflicts with master in holders/helpers.py and tools/simulation_dumper.py. This PR adds no conflict of its own: git merge-tree against master reports the same two files.

Other audit findings, handled elsewhere

The 2026-10-06 invariants audit of master 757147c7 also confirmed defects that this PR leaves alone, because other work owns them:

• Stock ADD changing later plain reads: draft Cache ADD and DIVIDE results only where a plain read returns them #571.
• Year deletion leaving monthly fast-cache entries: draft Drop every contained period from the fast cache when deleting or setting a period #566.
• Disk deletion of contained periods, and disk branch-name parsing and prefix collisions: draft Delete the periods within a deleted period from disk storage #565 (it inherits Keep branch period reads, dumps, and disk deletion consistent #552).
• Clone aliases bound to the parent: fixed by Bind a clone's calc and df to the clone, and give it its own system's entities #586.
• Serialization and subsample input provenance: Restore inputs as inputs, evict the fast cache on holder writes, reject non-numeric uprating and defined_for #576 and Rebuild a subsampled simulation from its inputs, not its calculated values #573.
• Restored entity counts: Fix restore_simulation entity counts #548.

Branch staleness after set_input and raw-file restore are documented behaviour, not defects.

Merge with master

Merged master 7399c4ed8b9401febd546512cc55ba1c3e896ab0 with merge commit 10574bb63e7f6329f0a7072d6865fc3efe6b2515; validated/pushed head adc18425ee153799911678a379fc31c3fd65c9ce includes the separate compatibility fix below. This section supersedes the historical stacked-base, pending-CI, and old dump-format notes above. Master already contains #581 and independent clone/input provenance handling.

Conflict resolutions:

• policyengine_core/holders/helpers.py: retained master's tier-aware supplied-input checks, normalized/early-year storage keys, unconditional successful helper cleanup, and dropped provenance/sequence metadata cleanup; restored Drop sums and twelfths an own-period input makes stale #593's own-period cleanup.
• policyengine_core/holders/holder.py: retained master's pre-store dependency invalidation and custom-handler calculation/failure cleanup; run Drop sums and twelfths an own-period input makes stale #593's cleanup after a successful direct non-ETERNITY input store.
• policyengine_core/tools/simulation_dumper.py: retained master unchanged. Its branch-visible derived-value format, format marker, and input-first restore with unknown-source sequence history supersede the older stacked inputs.txt implementation.
• tests/core/test_set_input_helper_order.py: retained master's expanded input-tier and early-year regression coverage.
• tests/core/test_set_input_helper_order_property.py: retained master's disk/branch restrictions and removed the own-period aggregate exclusion exactly as Drop sums and twelfths an own-period input makes stale #593 originally did.
• tests/fixtures/set_input_helper_order.py: retained master's fixture and conservative disk-period domain documentation.
• Nontextual release resolution: removed the inherited Count only inputs as already set when a set_input helper splits a longer period #581 set-input-helper-order.fixed.md fragment, already consumed by master release commit a934dc3465cd3284b6d21fb6fd7a0506b3e6581b; kept Drop sums and twelfths an own-period input makes stale #593's fragment.

Combined behavior: master still invalidates dependencies before an input and after calculations inside a custom handler. #593 additionally drops this variable's overlapping calculated sums/twelfths and fast-cache entries on readable branches after an own-period input, preserving supplied inputs in either storage tier and leaving parent simulations intact. A same-value branch input still skips master's broad dependency invalidation, while #593 may clear same-variable overlapping calculated caches. Restored derived values follow the same cleanup; restored inputs remain inputs. ETERNITY skips #593's additional cleanup.

Compatibility fix adc18425ee153799911678a379fc31c3fd65c9ce:

The first merged checks exposed two interactions with newer master. A separate compatibility commit keeps ordinary overlap comparisons unchanged and reuses master's Gregorian end ordering only when period.stop overflows, preserving supported periods ending after year 9999. Two warmed-default cases cover stored values and fast-cache-only values. Provenance examples and the generated tier property now explicitly confirm old disk-cache eviction, then recreate a later shadow disk cache so their deletion/export/reform witnesses remain meaningful. Branch documentation distinguishes dependency invalidation from overlap cleanup. There is no change to #593's stale-sum/twelfth rule.

Validation: 883 passed, 2 expected xfails across 38 files, with no unexpected failures. Each file ran separately in the foreground using uv run pytest -q <file>. All runs used OMP_NUM_THREADS=4 OPENBLAS_NUM_THREADS=4 MKL_NUM_THREADS=4 VECLIB_MAXIMUM_THREADS=4 NUMEXPR_NUM_THREADS=4 OMP_WAIT_POLICY=PASSIVE.

File (under tests/core/)	Passed	Expected xfails
test_branch_helper_invalidation.py	8	0
test_branch_input_fixed_point.py	23	0
test_branch_input_fixed_point_property.py	2	0
test_branch_input_invalidation.py	98	0
test_branch_input_invalidation_property.py	1	0
test_branch_scoped_delete_arrays.py	6	0
test_branch_shared_arrays.py	47	0
test_branch_shared_arrays_differential.py	1	0
test_carry_over_order.py	66	0
test_carry_over_order_property.py	4	0
test_data_storage_dir.py	40	2
test_data_storage_dir_property.py	1	0
test_dataset_default_role.py	41	0
test_disk_storage_differential.py	1	0
test_disk_storage_ownership.py	33	0
test_dump_restore.py	1	0
test_fast_cache_contained_periods.py	7	0
test_fast_cache_contained_periods_property.py	2	0
test_holder_branch_fallback.py	5	0
test_holder_write_fast_cache.py	16	0
test_holder_write_fast_cache_property.py	2	0
test_holders.py	22	0
test_input_tier_invalidation.py	5	0
test_on_disk_storage_clones.py	33	0
test_option_result_cache.py	24	0
test_own_period_input_overlapping_cache.py	7	0
test_registered_input_deletion.py	54	0
test_restore_input_provenance.py	2	0
test_restore_input_registry.py	14	0
test_restore_input_registry_property.py	1	0
test_set_input_helper_order.py	60	0
test_set_input_helper_order_property.py	3	0
test_subsample_restore_inputs.py	8	0
test_uprating_order.py	141	0
test_uprating_order_property.py	5	0
test_user_input_keys.py	29	0
test_user_input_keys_property.py	3	0
test_user_input_keys_soundness.py	67	0

Ruff on all 9 changed/resolved Python files: ruff check → All checks passed!; ruff format --check → 9 files already formatted. git diff --check passed.

The hub will rerun PE-US A/B and impact at merged head adc18425ee153799911678a379fc31c3fd65c9ce, then queue shell review and Max's core-semantics ruling. No microsim was run in this workspace. This push starts the PR's first CI run; CI outcome remains pending.

axiom: n/a: engine cache fix, no policy change

🤖 Generated with Claude Code

[MaxGhenis]

Merged by the US + core hub at adc18425ee153799911678a379fc31c3fd65c9ce.

• Ruling: Max approved merging without a full-sample marginal-tax-rate A/B first (decision d1690, 2026-10-19).
• Review: GPT-6.1 Sol shell-backed review APPROVE at this exact head (CORE_SEMANTICS yes: setting an own-period input now drops stale overlapping calculated sums, twelfths and defaults).
• CI: 19/19 green at this head. PE-US core gate: 494 tests passed.
• Admin merge: the only block was a required approving review on a PR from Max's account. Squash, --match-head-commit.