Skip to content

Certify the Microcosm UK national from microcosm#1428 on policyengine-uk 2.164.2 (0a4c8e2c; local area 6528c865 or c9df4b86) - #731

Draft
juaristi22 wants to merge 2 commits into
mainfrom
uk-certify-microcosm-national-1428
Draft

juaristi22 wants to merge 2 commits into
mainfrom
uk-certify-microcosm-national-1428

Conversation

@juaristi22

@juaristi22 juaristi22 commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

@MaxGhenis, for the UK hub: this certifies the national from PolicyEngine/microcosm#1428 on policyengine-uk 2.164.2 and supersedes #728. It needs the same decision from you as #728: which local-area cut it keeps, (a) or (b) below. That call carries over from #728, and this PR stays a draft until you make it.

The national

microcosm-uk-2024-25-national-20261031T181401Z-0a4c8e2c (microcosm_uk_2024_25):

The local area: your call

Constituency and local-authority regions route to the local-area cut (regional_release_kind: local_area). Neither cut below carries the fixes in PolicyEngine/microcosm#1419, PolicyEngine/microcosm#1420 or PolicyEngine/microcosm#1428, so area runs don't see them. The next rebuilt pair will.

(a) #725's microcosm-uk-2024-25-dense-20261027T154955Z-6528c865. This draft's default.

(b) main's microcosm-uk-2024-25-dense-20261026T105309Z-c9df4b86, the local-area data the dashboard reads today.

  • Built on policyengine-uk 2.151.0.
  • It needs its own input-surface audit against 2.164.2 before certify-data accepts it, with María as auditor (--audited-by juaristi22) and a limitation saying the cut was built on 2.151.0. Its H5 is on hand and matches the manifest pin, and the audit takes about a minute.
  • It doesn't store 25 inputs that 6528c865 stores and 2.164.2 reads: 24 alcohol, tobacco and gambling quantities, and is_apprentice. Area runs on it would read them as zero, along with the five inputs (a) defaults, so they would carry no alcohol, tobacco or gambling duty.

If you choose (b), I'll run the audit, re-certify and push it here.

Before the dashboard moves to this bundle

  • The Pension Tax Allowance row ("State Pension income tax protection") still can't run: see note 1 under the table.
  • Distributional results. policyengine-uk 2.164.2 predates Count other investment income in market and HBAI income policyengine-uk#2736, which counts other_investment_income in market and HBAI income. On this pin, household net income carries the tax on that income without the income itself. Costings read tax and benefits, so the table below is unaffected.

Budget measures on this national

Scored with #728's method: the PolicyEngine/uk-autumn-budget-dashboard-2026 measures (57095fd) and their October 2026 forecast. Each figure is the weighted change in gov_balance, in £bn. They were scored on policyengine-uk 2.162.14, as #728's table was; the bundle pins 2.164.2, and a rescore there can follow.

Columns:

Measure Year This cut #728 (4374e0bf) Published OBR line Table 6.1
Household Energy Payment 2026-27 −1.969 −1.969 −1.969 — −2.625
Zero VAT on domestic electricity 2026-27 −0.788 −0.841 −0.840 — −0.755
2027-28 −0.332 −0.353 −0.353 — −0.305
Fuel duty rises deferred 2026-27 −0.290 −0.290 −0.301 — −0.575
2027-28 −0.501 −0.501 −0.521 — −0.780
State Pension income tax protection 2027-28 to 2029-30 not scored (1) not scored (1) −0.030 / −0.083 / −0.144 — −0.045 / −0.090 / −0.150
Trading allowance raised to £3,000 2028-29 −0.148 −0.112 −0.118 −0.221 −0.225
2029-30 −0.149 −0.116 −0.119 −0.165 −0.165
2030-31 −0.148 −0.116 −0.120 −0.171 −0.175
2031-32 −0.147 −0.119 −0.130 −0.178 −0.180
Property income rates +2pp 2029-30 +0.402 +0.416 +0.429 +0.394 +0.435
2030-31 +0.417 +0.432 +0.446 +0.403 +0.445
2031-32 +0.429 +0.445 +0.459 +0.412 +0.455
Savings income rates +2pp 2029-30 +0.582 +0.603 held back +0.443 +0.520
2030-31 +0.611 +0.636 held back +0.475 +0.555
2031-32 +0.636 +0.666 held back +0.510 +0.590
Carer's Allowance earnings taper 2030-31 −0.037 −0.017 −0.024 — −0.040
2031-32 −0.073 −0.032 −0.048 — −0.080

(1) The dashboard's State Pension template wraps income_tax, which on policyengine-uk 2.162.x adds the Winter Fuel Payment charge, so the template refuses. The port is the dashboard's to do; the data play no part (#728, note 1).

What moved against #728's national:

Savings against the OBR's income tax line

Edited after Max's review, which compares the savings rates with the OBR's income tax line rather than the Table 6.1 total. That total adds £78m of onshore corporation tax in 2029-30, which the model doesn't cover. The earlier reading here is withdrawn: a difference of base level, with ISA shifting small.

  • The gap is 31%, 29% and 25% above the income tax line in 2029-30 to 2031-32. That compares this cut's static figure with a line after behaviour.
  • What sets this cut's figure.
    • The national pins the 2025 tax on savings to HMRC's Income Tax Liabilities Table 2.6 projection, by rate: 0.98, 1.00 and 1.00.
    • On that base, +2 points gives £0.48bn in 2025 and £0.58bn in 2029-30.
    • No data change moves it without departing from HMRC's projected tax. SPI donor savings rebased to SPI's 2023-24 bands (savings follow-up to #1418) microcosm#1428 tried binding those rows at £7.51bn, the 2023-24 outturn grown on the OBR's savings path. The calibration couldn't meet them beside SPI Table 3.7's savings income, and the change was reverted.
  • What the OBR's figures imply.
    • EFO paragraph 3.17 gives the measure about £0.6bn a year after behaviour and puts the savers' response at around £0.2bn a year, mainly extra use of ISAs.
    • That puts the static yield at roughly £0.75bn across all taxes.
    • This cut's static income tax averages about £0.61bn over 2029-30 to 2031-32: below that, not above it.
  • Not verified:
    • the size of the response;
    • its split between income tax and corporation tax;
    • whether the costing's base grows like HMRC's projection or in proportion to income, which would make it up to 13% lower.
  • The OBR estimates that savers cut the yield by around £0.2bn a year across taxes. It publishes no split by tax or year, so we can't yet say how much of the gap that response explains.
  • Behaviour belongs in the microsimulation, not the dataset. A savings response would be an opt-in policyengine-uk behaviour. The national already carries what it would read: ISA subscriptions, and so unused allowance, and taxable interest.
  • Dashboard: it shows static figures, and savings stays held back there until the comparison is settled.
  • Payment timing: the costing's +£125m and +£555m in 2027-28 and 2028-29 carry the Self Assessment catch-up.

Impact

  • The UK model pin moves from 2.151.0 to 2.164.2. The data moves from the 2.151.0 pair (a6299504 and c9df4b86) to 0a4c8e2c, with (a) or (b) for the local area. UK results change with both.

Commits

  • 59662673: the national and the local area, certified by one certify-data run. It also has the lock, the release lock, the TROs, the certified test constants and the changelog fragment.
  • 6b2d0b9f: the household calculator snapshots, regenerated on 2.164.2 (PE_UPDATE_SNAPSHOTS=1). The UK households and the model surface move with the model; the US snapshots don't change.

How it was made

uv sync --frozen --extra uk --extra dev --python 3.13
uv pip install policyengine-uk==2.164.2
python scripts/audit_regional_input_surface.py \
  --regional-manifest-uri "hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-dense-20261027T154955Z-6528c865/releases/microcosm-uk-2024-25-dense/release_manifest.json" \
  --artifact microcosm_uk_2024_25_dense=<the cut's H5> --model-version 2.164.2 \
  --audited-by juaristi22 --limitation "<the limitations in the regional record>" \
  --output regional_input_surface.json
python scripts/bundle.py certify-data --country uk --data-producer populace \
  --manifest-uri "hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-national-20261031T181401Z-0a4c8e2c/releases/microcosm-uk-2024-25-national/release_manifest.json" \
  --regional-manifest-uri "hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-dense-20261027T154955Z-6528c865/releases/microcosm-uk-2024-25-dense/release_manifest.json" \
  --model-version 2.164.2 \
  --accept-regional-model-mismatch "<the reason in the regional record>" \
  --regional-input-surface-evidence regional_input_surface.json
uvx --from uv==0.13.4 uv lock --no-config --no-sources --default-index https://pypi.org/simple --python 3.13
uv sync --frozen --extra uk --extra dev --python 3.13
python scripts/release_lock.py
python scripts/bundle.py generate --include-tros
PE_UPDATE_SNAPSHOTS=1 python -m pytest tests/test_household_calculator_snapshot.py -q
python scripts/bundle.py check
python scripts/bundle.py check --include-tros --strict-tros
python scripts/check_uk_uc_floor.py --dataset microcosm_uk_2024_25 --year 2025 --output uc_floor.json

uk_qualification.rewrite_certified_constants rewrote the certified constants in tests/test_models.py, tests/test_release_manifests.py and tests/test_uk_regions.py.

Evidence run on this head

  • The UC floor passed (check_uk_uc_floor.py, Great Britain, 2025, within 10%). The receipt holds weighted national totals only:

    • benefit units at 0.921 of DWP's figure;
    • spend at 0.951 of the OBR's.

    It took 119 s and peaked at 8.2 GB.

  • Both bundle.py check runs pass, the second with --include-tros --strict-tros.

  • Locally, 435 tests pass on the release manifests, UK regions, models, household snapshots, UC floor, bundle, TRO, certification, regional-evidence, pin and release-lock paths. The full suite runs in CI.

Disclosure

Aggregates only: weighted national totals over many records, and published HMRC, ONS, DWP, OBR and Treasury figures.

🤖 Generated with Claude Code

juaristi22 and others added 2 commits November 1, 2026 11:18
…e2c national, 6528c865 local area)

One certify-data run: the national microcosm-uk-2024-25-national-20261031T181401Z-0a4c8e2c
certifies on its own build (policyengine-uk 2.164.2); the local-area cut
microcosm-uk-2024-25-dense-20261027T154955Z-6528c865 routes on the accepted
model mismatch, with its input surface re-audited for 2.164.2 (no dropped
column; five inputs added after 2.162.12 take their defaults). Also the
relock, the release lock, the TROs, the certified test constants and the
changelog fragment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PE_UPDATE_SNAPSHOTS=1. The UK households and the model surface move with
the model; the US snapshots are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

juaristi22 commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor Author

Property income rates +2pp: what moved, and how it compares with the OBR's income tax line

Edited 1 Nov after Max's review: compared with the OBR's income tax line rather than the Table 6.1 total, and the partnership rescale is withdrawn.

The table splits the measure's 2029-30 yield into its two parts:

  • The rate part: 2 points more tax on property profit outside Scotland.
  • The relief part: 2 points more finance-cost relief. Relief follows the property basic rate UK-wide, so it covers Scottish landlords too.

Measures and setup are as in the body's table: the dashboard's measure, policyengine-uk 2.162.14, change in income tax in £bn.

National Yield Rate part Relief part Landlords' finance costs, 2024-25
Published (a6299504) +0.429 +0.659 −0.231 £9.65bn
#728 (4374e0bf) +0.416 +0.653 −0.237 £10.07bn
This cut (0a4c8e2c) +0.402 +0.653 −0.251 £10.59bn
Official +0.435

What moved: the relief part.

Against the OBR's income tax line.

  • The OBR's income tax line for the measure is +£394m, +£403m and +£412m in 2029-30 to 2031-32. The Table 6.1 total (+£435m in 2029-30) adds onshore corporation tax and two Scottish lines, which the model doesn't cover.
  • This cut is 2–4% above the income tax line, so there's no shortfall to explain. EFO paragraph 3.17 puts landlords' response (incorporation and non-compliance) at about £0.1bn a year.
  • Withdrawn: "the model reproduces the costing once partnerships are in". The rescale to PRIS's base for individuals and partnerships lands about 10% above the income tax line and quantifies no omission.

Partnership property income: a lead, not a quantified gap.

  • The costing's base (policy costings, p.74) "consists of all individuals and partnerships that receive property income".
  • HMRC's documentation doesn't say whether SPI's property income includes partners' shares. The totals bound it:
    • The SPI public use tape documentation defines it as net income from UK and overseas property. PRIS's individuals report UK property only.
    • This cut carries £32.8bn of property profit before finance costs and 2.82m landlords. PRIS 2024-25 gives individuals about £30.9bn and 2.85m landlords; partnerships add about £6.2bn of profit and £1.7bn of finance costs.
    • So this cut is £1.9bn above PRIS's individuals and £4.3bn below individuals and partnerships. At most £1.9bn of the partnerships' £6.2bn could be in it, and less to the extent overseas property fills that excess.
    • Partners are taxed on their shares as property income: the SA800 partnership statement allocates UK property income to each partner. So the engine would tax them if the data carried them.
  • PRIS doesn't split finance costs by entity, so any individuals' share of them is pro rata.

Bringing it in.

  • The only published source is PRIS by tax entity: Tables 1, 2 and 6 give partnerships' landlord count, receipts and expenses for 2020-21 to 2024-25. It is already in the pinned Chronicle feed.
  • No public microdata names partners. The SPI tape has one property variable, so carrying partnership property income in the national means assigning it to landlords by an imputation, controlled to the PRIS totals.
  • Finance costs together with it. Nothing binds landlords' finance costs today, and they moved the yield between builds. Binding them belongs in the same change.
  • That is a candidate follow-up for PolicyEngine/microcosm.

🤖 Generated with Claude Code

@MaxGhenis MaxGhenis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks María. You've added gross receipts to the national data, and we now estimate a £148m trading allowance cost in 2028-29.

Could we compare the savings and property estimates with the OBR's income tax lines before publishing them? Since 28 October, we've compared each estimate with the OBR line that covers what we model. We keep Table 6.1 totals as context, as we do in Tables 6 and 7 of the briefing. You use Table 6.1 totals in the Official column.

Measure Year This cut OBR line This cut against that line Table 6.1 total
Savings income rates 2029-30 +582 +443 exceeds by 31% +520
2030-31 +611 +475 exceeds by 29% +555
2031-32 +636 +510 exceeds by 25% +590
Property income rates 2029-30 +402 +394 exceeds by 2% +435
2030-31 +417 +403 exceeds by 3% +445
2031-32 +429 +412 exceeds by 4% +455
Trading allowance 2028-29 −148 −221 trails by 33% −225
2029-30 −149 −165 trails by 10% −165
2030-31 −148 −171 trails by 14% −175
2031-32 −147 −178 trails by 17% −180

Figures in £ million. We compare savings and property with income tax (measures 62 and 61), and the trading allowance with income tax plus NICs (measure 15).

For savings in 2029-30, the OBR reports £443m of income tax and £78m of onshore corporation tax, £521m in all, and the Treasury rounds that to £520m. You report a scaled £521m, which matches that total and exceeds the income tax line by £78m, or 18%.

In EFO paragraph 3.17, the OBR estimates that savers reduce the yield by around £0.2bn a year, mainly by using tax-advantaged assets such as ISAs. Could you drop "leaves little room for the costing's ISA shifting" and "ISA shifting then looks small"? If we add all £0.2bn to 2029-30 income tax, our £582m would trail the roughly £640m static estimate by about 10%. But the OBR gives no split of the reduction by tax or year, so we can't derive a static income tax line.

We need a verified reason for the 2029-30 and 2030-31 savings gaps before publishing, since both exceed our 25% bar. Our 2031-32 estimate exceeds the income tax line by 24.7%. Could you revise "Why savings sits above the costing" and #1428's savings lines against the income tax lines?

For property in 2029-30, the OBR reports £394m of income tax and £39m across onshore corporation tax and two Scottish lines. The Treasury rounds the £433m total to £435m. You report a rescaled £434m, which exceeds the income tax line by 10%; our published £429m exceeds it by 9%. The OBR estimates that landlords reduce the yield by £0.1bn a year through incorporation and non-compliance. We can use your PRIS bounds to investigate missing partnership income, but we can't quantify any omission from the rescale.

@juaristi22

Copy link
Copy Markdown
Contributor Author

Thanks @MaxGhenis, agreed on all four points. The model covers income tax, and income tax plus NICs for the trading allowance, so those are the right lines to compare with.

Could you verify the savings reading? I'd like your check on whether the savers' response really drives most of the gap, before we rely on it. The evidence is below. What I can't verify is how the OBR's response splits by tax.

Savings. Against the income tax line we sit 31%, 29% and 25% above. That compares our static figure with a line after behaviour.

  • What sets our figure.
  • What the OBR's own figures imply.
    • EFO paragraph 3.17 gives the measure about £0.6bn a year after behaviour and puts the savers' response at around £0.2bn a year, mainly ISAs.
    • That puts the static yield at roughly £0.75bn across all taxes.
    • Our static income tax averages about £0.61bn over 2029-30 to 2031-32: below that, not above it.
  • What isn't verified:
    • the size of the response;
    • its split between income tax and the £78m of corporation tax;
    • whether the costing's base grows like HMRC's projection or in proportion to income, which would make it up to 13% lower.
  • So the gap reads as at least half behaviour, probably most of it. That reading is what I'm asking you to verify: from the OBR's static costing by tax if you can get it, or from your own reading of the costing.
  • Withdrawn: both ISA phrases and my base-level explanation. My scaled £521m had matched the Table 6.1 total, which includes the corporation tax we don't model.
  • Behaviour belongs in the microsimulation, not the dataset, and the dashboard shows static figures. If the reading holds, should the bar compare our static figure with the OBR's implied static range? Until then savings stays held back from the dashboard.

Property.

Trading allowance.

Revised:

🤖 Generated with Claude Code

@MaxGhenis MaxGhenis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks María. The OBR and the government publish no static costing by tax in the policy table, the EFO or the costing note, so we can't settle "at least half" from published figures. The OBR estimates that savers cut the yield by around £0.2bn a year across taxes, while we hold savers' choices fixed.

The government leaves ISA returns out of the costing's tax base (costing, p.75), so savers moving into ISAs cut the income tax line. The OBR also scores keeping the cash ISA limit at £20,000 on income tax alone, about −£35m in 2029-30 (policy table, measure 11), and the government gives that measure "no static impact" (costing, p.19). Neither of them sizes the income tax loss from ISA shifting under the savings rise, or splits its effects across taxes. Neither the OBR nor the government explains the £78m of corporation tax, which starts at +£13m in 2026-27, before the rates rise.

Your £0.75bn includes that corporation tax, so we can compare our estimates with the OBR's income tax line plus an assumed response on income tax:

Assumed response on income tax, each year Implied static income tax, 2029-30 to 2031-32 Our estimates
£75m £518m, £550m, £585m exceed it by 12%, 11% and 9%
£150m £593m, £625m, £660m trail it by 2%, 2% and 4%
£200m £643m, £675m, £710m trail it by 9%, 9% and 10%
£250m £693m, £725m, £760m trail it by 16%, 16% and 16%

About £70m a year on income tax would cover half of the gap, and about £140m all of it. At least £23m a year would bring our excess under 25% in all three years. Your review of PolicyEngine/microcosm#1428 also finds that our model includes annuity and trust income in the savings measure's base (finding 1). Our estimates and these amounts can move with the rebuild.

The government grows the 2023-24 SPI base "using the OBR's forecast for growth in savings income" (costing, p.75), but it doesn't say whether that growth applies before the frozen allowances. If the static income tax line trailed ours by 13%, the income tax response would come to about £63m, £57m and £43m, under a third of the OBR's £0.2bn. Neither the OBR nor the government publishes enough detail to rule out those amounts.

The OBR counts Self Assessment receipts in its line, and we count liabilities. Taxpayers who make payments on account normally base them on an earlier bill and settle the balance later, so growing savings liabilities can widen the gap from 2029-30 too.

Could you replace "So the gap reads as at least half behaviour, and probably most of it" in the PR with "The OBR estimates that savers cut the yield by around £0.2bn a year across taxes. It publishes no split by tax or year, so we can't yet say how much of the gap that response explains"? Could you also drop "matters only before 2029-30"?

For the trading allowance, could you replace the sentence beginning "The rest is" with "The government includes traders switching to the allowance in its costing but doesn't quantify that response. This scoring engine also doesn't relieve miscellaneous income (PolicyEngine/policyengine-uk#2726)"?

I'll come back on the bar and on whether we can get the split from the OBR, and we'll keep savings off the dashboard until then.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants