[MaxGhenis]

Summary

Fixes #1426. The order of columns in PolicyEngineRunner output depended on what else was in the batch. The same two records put frate in column 8 or column 40, depending only on which tax year carried idtl=0.

The runner simulates one tax year at a time. Each chunk's frame carries only the columns its rows' idtl levels ask for, and the chunks are joined with pd.concat, which keeps columns in order of first appearance. A chunk that asks only for the standard columns puts frate, srate ahead of the full-detail columns a later chunk adds.

Change

run() now puts the joined frame in one canonical order: the three identifiers, the implemented columns in the order of the policyengine_to_taxsim mapping, then the marginal rates. The order is derived from the mapping, not listed by hand.

That is the order a batch already had when all its chunks carried the same columns, so batches with a single idtl level normally keep their layout. Which columns appear still depends on what the batch asks for; their relative order no longer does. Per decision d1312 this keeps the runner's current layout and does not move to TAXSIM's header order.

What changes for users: only batches whose chunks carry different column sets. In practice that is batches that mix idtl levels across tax years, which now get the same layout as a full-detail batch, and the rare batch where one year's taxsimid values are all blank, which used to put taxsimid last and now puts it first. Values are unchanged.

The ordering is applied where run() returns, not at the pd.concat lines, to stay clear of #1406 and #1423, which edit those lines.

Invariants

1. Batch independence. The output columns are the canonical order restricted to the columns the batch carries. Which rows accompany a record never moves a column relative to another.
2. Batches with uniform chunks unchanged. When every chunk carries the same columns, the ordered frame has the same column order as the frame before ordering. That covers single-idtl batches unless one year's IDs are all blank.
3. Nothing added, dropped or moved between names. The same set of columns comes back and every value stays under its own column name.
4. Unknown columns survive. A column the mapping does not know is kept, after the known ones, in the order it arrived.

Invariants 1, 3 and 4 have Hypothesis property tests on the ordering function. Invariants 1, 2 and 3 are also checked on real runs.

Verification

Run on this branch (base main at 72fdc04), macOS, with the 11 tests in tests/test_output_column_layout.py.

Check	Result
New tests on this branch	12 passed (after merging main at be55c81)
Same tests with the ordering turned into a no-op	4 fail: both property tests, the run() wiring test and the mixed-batch layout test. The single-idtl and value tests pass, as they should
ruff format --check .	clean

The real-run tests take each batch's frame from _run_once, before ordering, and compare it with the ordered frame. So "unchanged" is measured against what the runner produced before this change, for idtl 0, 2 and 5. The two mixed batches are the ones from #1426. The no-op run used a copy of the package on PYTHONPATH; the branch itself was not edited.

Review

An independent review (GPT-6.1 Sol) approved with no blocking findings. It ran the tests, confirmed the no-op version fails them, and found one case the "unchanged" claim missed: a year whose taxsimid values are all blank. The third commit qualifies the claim, rewords "position" as "relative order", and tests that case. It found no positional use of runner columns in policyengine_taxsim/, tests/, r-package/ or dashboard/.

Not covered

• Batches routed to TAXSIM. StitchedRunner sends pre-2021 rows to the TAXSIM binary. A batch with only such rows comes back in TAXSIM's own layout; a batch that also has later years uses PolicyEngine's layout with the TAXSIM-only columns after it. That difference stays, since d1312 keeps each current layout.
• RemoteTaxsimRunner over 2,000 rows. It joins 2,000-row batches the same way (remote_taxsim_runner.py line 216). I read that code but did not run it against the service; tracked separately in RemoteTaxsimRunner: column positions may depend on batch order for inputs over 2,000 rows #1439.

axiom: n/a: emulator output layout, no policy change

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
policyengine-taxsim	Ready	Preview	Oct 12, 2026 12:44pm UTC