[MaxGhenis]

Adds CenTax's published costing of removing the £100,000 childcare limit as an external reform score, with PolicyEngine's static counterpart on the certified UK pair. Also adds the Centre for British Progress estimate that CenTax reviews (lane uk-childcare-100k).

Status: approved in round 8

Seven independent reviews (GPT-6.1 Sol) asked for changes, and the eighth approved:

• Round 1, at 661c14d: 8 findings, answered in 233c609.

• Round 2, at 12b94b0: all 8 round-1 findings confirmed fixed, plus 5 new findings. These are answered in 371f1cf.

• Round 3, at 371f1cf: round-2 items 2 to 4 and the three body points confirmed fixed, and item 1 fixed for normal ingestion. It found 2 minor issues, 1 nit and 2 body nits, answered in e5736b1.

• Round 4, at e5736b1: every round-3 item confirmed fixed. Ordering was fixed only in part, and so was the metadata scan, which still let 8 mutants through. It found 4 minor issues and 1 nit, all future-artifact or edge cases; the committed numbers are unaffected. Answered in 39d0027.

• Round 5, at 39d0027: every round-4 item confirmed fixed for the reported cases, and both direct-call limits closed. It found 3 minor issues and 1 nit, all about future artifacts or future edits. The committed numbers are unaffected: two fresh builds matched all 174 committed feed files. Answered in 491bb2e.

• Round 6, at 491bb2e: every round-5 item confirmed fixed (the reserved ids agree with SQLite in 32,160 comparisons). It found 1 minor issue and 1 nit, both in the metadata contract's tests; the committed numbers are unaffected: two fresh builds matched all 174 committed feed files apart from today's date stamps. Answered in 0311c08.

• Round 7, at 0311c08: the round-6 nit and no-recipient note confirmed fixed, and the round-6 callback fixed for the reported example (all 34 mutants and the 51-case matrix independently re-run). It found 1 minor issue, a neighbouring callback route in the metadata contract's tests, and noted a nonblocking producer boundary; the committed numbers are unaffected: two fresh builds hash e5497fd5… and reproduce all 174 feed files apart from the date stamps. Answered in 49e782b.

• Round 8, at 49e782b: APPROVE, no new findings. Both round-7 items confirmed fixed; all 38 mutant outcomes, the 51 earlier cases and the 20 mutation counts reproduced; two fresh builds hash e5497fd5… and reproduce all 174 feed files apart from the date stamps. It noted one boundary outside this PR's cases: a population whose total weight is zero would still give NaN take-up shares; no supported pair reaches it, and the guard goes with Run the £100,000 childcare lane on policyengine.py's certified Microcosm pair (6cba5d71, policyengine-uk 2.132.6) #228, which reruns the compute script.

Each response is listed under "Review response".

While answering round 2, I found that the published dashboard figure is no longer on policyengine.org:

• policyengine.org served it from 12 October 2026. On 21 October it took the page down while the dashboard's costing is revised (policyengine-app-v2 de2b07cf), and the path now redirects to /uk/research.
• The row stays as history and now states that window.
• policyengine.py#685, merged on 21 October, certified a Microcosm pair with the fixed hours draw. This lane has not run that pair yet; the rerun is #228.

The comparison (£bn, negative = cost to the Exchequer)

	2027-28	2029-30
CenTax static cost, central (Table 4.2)	−0.80	−0.98
CenTax net fiscal cost, central	−0.52	−0.64
PolicyEngine, certified pair (static)	−0.690	−0.838
PolicyEngine, published dashboard (history row; on policyengine.org 12–21 October 2026)	−0.434	−0.505
PolicyEngine dashboard since #8 merged (1b83dd46, 24 October; its own Vercel deployment, not policyengine.org; not attached)	−0.902	−1.037

The certified pair is the default replay variant: populace-uk-2023-dd68c73-4aa4b14-20260619T023711Z with policyengine-uk 2.89.2 (data/uk/certified_bundle.json). Against CenTax's central static cost, PolicyEngine is 14% below in both years (ratios 0.863 and 0.855).

The certified pair has the pre-fix hours draw. Families receiving the extended hours use a measured weighted mean of 15.3 of the 30 weekly hours. microcosm#1126 puts DfE's mean at about 28.5. The pair was built on 19 June 2026, before the fix merged in microcosm on 16 October (microcosm#1299). A diagnostic run on the same pair with every family at the full 30 hours gives £1.40bn and £1.72bn. It is not attached to any claim.

policyengine.py#685 certified national cut microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71 with policyengine-uk 2.132.6. It is the first cut with the fixed draw, and policyengine.py 6.11.0 (25 October) still pins it. This repository records it only as an experiment population, not as a replay variant. Until #228 runs it, the latest result is the 19 June pair's, and the row shows that pair's miss.

The dashboard's current figure runs on Microcosm's 23 October release (a133f37a, built from microcosm 829c1bc26, which contains #1299) with policyengine-uk 2.147.2. That release is in no certified pair. The published row records the figure and does not attach it. When policyengine.org serves the page again, its figure becomes a new published row.

What changed

• sources/centax-childcare-100k/: the report PDF and an extract of the twelve pages it is cited on. The adapter parses Table 4.2 (3 scenarios × 2 tax years × 6 columns) and cross-checks it three ways:

  • static + both margins = net within the £20m that £10m rounding allows;
  • Table 4.3's abolition rows match;
  • the prose under the table restates the £340m, £210m and £130m figures.

  annotations.json gives each difference with its page or engine-file basis.

  pe/published_dashboard.json holds the published figure, pinned to uk-childcare-100k-threshold@f25f0db and its results file's SHA-256. Two fields record what has happened since:

  • live: when policyengine.org served the page, with the deployment and takedown commits.
  • after: what the dashboard has served since Scorecard database: reform-keyed claims, populace-targets shape #8 merged, pinned to 1b83dd46 and its results file's SHA-256.

  The README notes the calibration follow-up.

• sources/cbp-rates-and-wrongs/: CBP's own technical appendix, which carries the numbers. Tables 2 and 3 have 72 cells; the 12 "Abolish Cliff" cells become claims. CenTax cites the main report at a URL that returns "Article Not Found" (checked 2026-10-21), and source.json says so.

• pipeline/compute_uk_childcare_100k.py: the checks run in this order.

  1. Before any engine import, the installed wheels are checked against the variant record: check_variant_environment validates the record and runs verify_installed.
  2. compute_uk_event.preflight then:
     • sets offline mode and isolates the certification environment;
     • checks the managed release against certified_bundle.json and the record's release bundle;
     • checks import origins.
  3. The script checks the managed release's revision and SHA-256 against the record.
  4. Each world hashes the exact file the managed simulation reports as runtime_dataset_source, before construction and after its reads.
  5. Import origins are checked again after the last world.

  A variant materialised with --variant goes through variant_preflight instead, which runs the same checks.

  The script runs three worlds: current law, the 30-hours limit removed, and both limits removed. Each runs years 2026 to 2030 (2026-27 to 2030-31), and two hours diagnostics run years 2027 and 2029. Every head is summed UK-wide and for people in English households. The artifact records the take-up and usage inputs the dataset supplied, and is written to results/uk/childcare_100k/<variant>.json. The default run id is childcare-100k-<variant>-<UTC start second>.

• scorecard_db/ingest_uk_childcare_100k.py (build step after uk_ab2025_lanes):

  • 36 claims, all revenue_change and held out;
  • 22 certified-pair results plus 4 published-history results;
  • 2 pe_gap diagnoses. Each claim's diagnosis comes from the newest run that computed that claim's year, so a newer run that skips a year leaves that year's diagnosis in place.

  Drops are tallied: CenTax's 12 bunching counts and CBP's 60 taper cells, both still in data/externals.

  Each result's status is the one status-setting annotation that matches it; anything else raises.

  Every certified artifact in results/uk/childcare_100k/ is checked against its own variant record and keeps its run in the history. Each claim's latest result is the newest run that computed its year. Counterparts are England-only.

  A note whose basis is the engine or a run names the pairs it was checked on (variants) and attaches only to their results. Each pair's own statements are in verified_pairs in results/uk/childcare_100k/construction_annotations.json: why its run is static, its hours draw, and its pe_gap diagnosis. Ingest refuses an artifact on a pair with no entry.

• models.STANDARD_CONDITIONS gains scenario. relationships.uk_relationship gains CBP: none of the certified build's 149 targets is a childcare or CBP quantity. The app gains the CBP source label. export_populations shows a claim's publication.pe_missing as conditions.pe_missing. Docs are updated, and the regenerated feeds are included.

Statuses

• CenTax static against PE static: comparable (all three scenarios).

• CenTax net, and every CBP row (net of behaviour, includes Tax-Free Childcare): concept_mismatch.

• The published dashboard figure: constructed. The dashboard applied six local corrections to policyengine-uk 2.102.3. Its computed_at (2026-10-09) is older than the certified run, so the comparisons view and the app's latest show the certified run, and the published miss stays in the history beside it.

• Some claims have no PE value:

  • CenTax's behavioural margins, because this lane's counterparts are static;
  • CBP's 2031-32, which is outside every run's data years.

  They are published without a result. publication.pe_missing says why, read off the years each run computed; the feed shows it as conditions.pe_missing.

Annotations (each cites its basis)

The engine and run notes name the pairs they were checked on (all the default pair):

• static vs static;
• net includes bunching and labour-supply responses;
• free hours only, no Tax-Free Childcare: the run measured TFC unchanged in every year;
• the year convention: CenTax's 2028 is 2027-28, and pe-uk 2.89.2 runs year Y on its 30 April Y parameters (utils/parameters.py);
• scenario frictions;
• the engine's universal-hours switch-off (policyengine-uk#1930, changed after the pin by policyengine-uk#2177);
• no hours for under-1s: ages on the pair are whole years, and 0 under-1s receive hours in every world and year (both measured).

Never a calibration target

Both sources are held out. never_calibrate() is keyed on metric and covers only the poverty family, so it does not by itself cover reform estimates. A new built-DB test enforces the rule for every source: any claim scoring a non-baseline world is held_out. It holds on a fresh build at 371f1cf: all 6,812 consumed_as_target claims are baseline-world.

Invariants (load-bearing logic) and their tests

• Source fidelity:

  • each page of the extract re-extracts from the PDF byte for byte;
  • every published cell equals a hard-coded copy read from the PDF.

• Accounting identity: static + intensive + extensive = net within the rounding bound. The bound is derived, and a Hypothesis property checks it for any three real components.

• Round trip: a published money cell parses back to its value (Hypothesis).

• Conservation: in every reform world and year, gov_spending moves by the four childcare heads to within £29 (measured; the test and ingest allow £1m), and gov_tax moves by £0. Removing the 30-hours limit moves Tax-Free Childcare by £0.

• Geography: no free-hours head moves outside England (ingest raises otherwise; a mutation test checks the raise). England's Tax-Free Childcare is between 0 and the UK's.

• Provenance: each world's dataset digest before and after equals the certified one, read from the file the simulation reported. An artifact is refused when any of these differ from its variant record: package versions, RECORD hashes, Python, packaged manifest, release bundle, dataset or data years. A parametrised test covers 14 mutations.

• Pair scoping: a note on the engine or a run reaches only results on the pairs it names. An artifact on a pair with no verified_pairs entry is refused at stage, results, diagnoses and the expected tally, whether or not it computed a claim year. A two-pair test checks that a note not verified on the second pair, the default pair's hours draw and its diagnosis never reach the second pair's 22 results. _spec refuses an engine or run note without variants, and a variants list on a note that is neither.

• Coverage: the expected tally and every pe_missing follow the years each run computed. Tested on three histories: one artifact, two artifacts, and two artifacts where the newer one also computes 2031. Certified results are 22, 44 and 46; published 4; claim ids are identical across all three.

• Diagnoses follow the claim's own latest run: a test adds a newer verified run that computes only 2031. Both diagnoses stay, with the default pair's text, and the tally expects 2.

• Identity: a claim id ignores the value, its printed form and pe_missing (Hypothesis).

• Run ids and order: the default id differs across variants and across start seconds. One history check runs on every public path (load, results, certified_result, diagnoses, the expected tally); the private helpers _covering and _pe_missing are reached only through them. It refuses:

  • two artifacts with one run id;
  • a run id that the comparisons view's LIKE 'platform-grid-%' matches, in any ASCII case (the view prefers those runs within a data bundle). The compute CLI refuses the same ids, and a Hypothesis differential checks both tests against SQLite evaluating the view's own LIKE;
  • two artifacts with one computed_at;
  • a computed_at that is not a UTC time in the form datetime.isoformat() writes. In that form text order is time order, and a Hypothesis property checks it for any two UTC times.

  The published row's time is checked the same way.

• Differential, database against ingest: a real ingest into an isolated database runs on three runs: two pairs share a data bundle, and the newest run computes only 2027. For every claim, the comparisons view's latest result is the run _covering picks, and its diagnosis is that run's pair's.

• History: every certified run stays on each claim it computed; the published figure must be older than every certified run. The after figure is stated on the published row and never stored as a value.

• Differential: the free-hours heads agree between the 30-hours-only world and the both-limits world.

• Bounds and monotonicity:

  • removing the limit costs money every year;
  • extended hours rise and universal hours fall;
  • full hours use costs more than the drawn usage.

• Metadata (vetted scan and stand-in run): each calculate() argument must resolve to names the artifact's metadata lists. Two layers check this. The scan alone refuses all 38 mutants kept as regressions. With the scan set aside, the stand-in run alone sees an unlisted name in 30 of them. Of the other eight, two (tuple_target, module_constant_bound_twice) calculate only names their own METADATA lists, one (match_mapping_rest) crashes before it writes an artifact, and in five a stand-in refuses the object or object array the mutant passes it. A test pins that outcome for each mutant.

  • The scan reads the compute script's own code. A name resolves only if it is bound exactly once in its scope chain, as one of:

    • a plain for or comprehension target over a tuple display or a module tuple;
    • a single module-level assignment.

    Any other binding raises. So does any form that could bind a name or reach calculate out of the scan's sight:

    • assignment expressions, global, nonlocal and match;
    • dynamic name access, and getattr without a literal name;
    • calculate used other than as a direct call, or named by a string;
    • compute_uk_ab2025._resolve_parameter, which is getattr in a loop, used other than as a direct call, named by a string, or on a path that does not resolve (as a calculate name does) to gov. parameter paths with no calculate segment;
    • a class, by a class statement or by type() with more than one argument. The script defines none. A method on one (resolve, __fspath__, __float__, …) is the script's code that library or engine code can call implicitly, inside a stub or an array conversion, where the stand-in run does not follow;
    • attribute writes to module constants.

    Then every import, builtin name and attribute name the script uses must be on a vetted list: 16 imports, 19 names and 81 attributes, which a test keeps equal to what the script uses. I read each for a route to calculate other than a direct call. The one I found is _resolve_parameter, which walks any dotted path; that is why the scan holds its paths. A new name is refused until someone reads it and adds it.

  • The stand-in run runs the script's main() end to end on stand-in simulations. That covers all three worlds in every default year and both full-use worlds, with the hours input drawn below 30 in the ordinary worlds and set to 30 in the full-use ones. Only code outside the script that needs the engine, the data or the variant registry is stubbed: the variant record and preflight, the dataset hash, the bundle checks, the import-origin check and build_sim. The path helper runs as it is. Each stub, and each stand-in engine method, refuses any argument whose type is not exactly a builtin value, a path or an array, and any array of an object dtype, structured ones included. So no object can carry a callback that a stub would drop or that only the engine's own conversion would make. A stand-in offers only what the script reads, so any other engine call raises. The test loader registers each module as an import does. The names it records must equal the names the scan resolves (a differential test), and the artifact it writes must carry metadata for exactly METADATA.

  • Limits. The scan covers the compute script, not the modules it imports. An AST check of the thirteen pipeline functions the script calls finds getattr on a string the caller passes only in _resolve_parameter; it does not follow those functions' own callees. The stand-in run sees a call only on a path it takes, and its values are fixed, so a branch that turns on other data values may not run (the review's probe that tests an age below 120 is one). The scan reads every branch, so it covers those.

  The 38 mutants are round 3's two, round 4's eight survivors and its mapping capture, four more, round 5's thirteen survivors, round 6's callback and its five neighbours, and round 7's object-array carrier and its three neighbours. Each of round 5's thirteen is caught by either layer alone, and so is round 7's carrier. The round-5 review's whole 51-case matrix, re-applied to this head, is refused by the scan.

• Evidence for full use: a note or diagnosis says every family is at the full 30 hours only when both full-use worlds show it. Each must record:

  • the override: the variable, the value 30 and the year set, with every holder that reads the input still empty before it. Those are the four holders set_full_use checks; a differential test reads them off a stand-in run of it;
  • the read-back at 30, in the override record and in the world's means. The receiving mean may be absent only when the weighted count of benefit units receiving the offer is zero, as year_block writes it;
  • the main world's reform: the same limits, each removed on 30 April of the year under the repo's NO_LIMIT sentinel. Each start.stop period is read as Reform.from_dict applies it, with both days in force, and overlapping periods must agree;
  • the main world's executed limits, read back from the engine, each at NO_LIMIT exactly when the reform removes it.

  A parametrised test breaks each of 23 conditions. Another keeps four true record shapes: per-year windows from reform_dict, a one-day window on 30 April, overlapping periods that agree, and no recipients.

• Determinism: two builds hash the same, as in CI.

• Exhaustive status decision: every attached facet has exactly one stated status.

Checks run

• At 49e782b:
  • ruff format --check be nz scorecard_db tests pipeline clean; ruff check on the three lane files clean;
  • database rebuilt from scratch: content hash e5497fd5…, unchanged since 371f1cf, so the feeds are unchanged; the re-export differs only in the date stamps, which are not committed;
  • full suite under Python 3.12 (pytest tests/ -n 4): 2,806 passed, 18 skipped; the lane's tests: 238 passed;
  • the review's object-array diff applied to this head: refused by the scan ('_resolve_parameter' named by a string), refused by the stand-in run alone (an object array), refused by the contract. Each scan rule alone refuses it too: the class rule with the getter rules off, and the string rule with the class rule off; with both off the scan accepts it. The round-6 callback diff: refused by the scan and the contract, seen by the stand-in run alone. The round-5 51-case matrix: all 51 refused by the scan, and the stand-in run alone sees an unlisted name in 45;
  • the 38 kept metadata mutants: the scan alone refuses all 38; the stand-in run alone: 30 unlisted, 2 listed only, 1 crash, 5 refused;
  • mutation (each must make at least one test fail; all 20 did): no getter rule (4 tests fail); getter paths not checked for shape (1); getters aliased (1); getters named by a string allowed (1); classes allowed (2); object arrays accepted (4); the path helper stubbed with str (1); stubs taking any argument (2); get_holder taking any argument (1); the loader not registering modules (1); vetting off (12); the stand-in run for 2027 only (11); hours never drawn (1); year_block dividing by zero receiving weight (2); the receiving mean not checked (12); checked against the record count instead of the weight (6); non-finite numbers accepted (2); bools accepted (2); the note formatting a missing mean (2); the diagnosis formatting one (1).
• At 0311c08:
  • ruff format --check be nz scorecard_db tests pipeline clean; ruff check on the three lane files clean;
  • database rebuilt from scratch: content hash e5497fd5…, unchanged since 371f1cf, so the feeds are unchanged; the re-export differs only in the date stamps, which are not committed;
  • full suite under Python 3.12 (pytest tests/ -n 4): 2,784 passed, 18 skipped; the lane's tests: 216 passed;
  • the review's callback diff applied to this head: refused by the scan (an unresolved getter path), seen by the stand-in run alone, refused by the contract; the round-5 51-case matrix: all 51 refused;
  • the 34 kept metadata mutants: the scan alone refuses all 34, each for its own reason; the stand-in run alone: 29 unlisted, 2 listed only, 1 crash, 2 refused;
  • mutation (each must make at least one test fail; all did): no getter rule (6 tests fail); getter paths not checked for shape (1); getters aliased (1); the path helper stubbed with str (1); stubs taking any argument (2); get_holder taking any argument (1); the loader not registering modules (1); vetting off (13); the stand-in run for 2027 only (11); hours never drawn (1); the receiving mean not checked against the records (3); the note or the diagnosis formatting a missing mean (1 each).
• At 491bb2e:
  • ruff format --check be nz scorecard_db tests pipeline clean; ruff check on the three changed Python files clean;
  • database rebuilt from scratch: content hash e5497fd5…, the same as every build since 371f1cf, so the feeds are unchanged;
  • feeds re-exported and copied to app/public/data; the only diff is the export's date stamp (built, updated: today's date), which is not committed;
  • full suite under Python 3.12 (pytest tests/ -n 4): 2,746 passed, 18 skipped; the lane's tests: 178 passed;
  • the reviewer's round-5 probes: each of its 26 full-use records is refused for the right condition, both of its true records are accepted, and its 51 metadata cases, re-applied as diffs onto this head's script, are all refused by the scan;
  • the 28 kept metadata mutants: the scan alone refuses all 28; the stand-in run alone sees 25 (two crash the run, and tuple_target calculates only listed names);
  • mutation (each must make at least one test fail; all did): reserved_run_id case-sensitive in ingest (5 tests fail) and in the compute script (3); _in_force reading year prefixes only (2); _same_reform not checking values (7); no executed-limit check (2); any dependents dict accepted (2); no zero-recipient allowance (1); no sentinel check (1); _refuse_unvetted switched off (13); the stand-in run for 2027 only (11); hours never drawn (1).
• At 39d0027:
  • ruff format --check be nz scorecard_db tests pipeline and ruff check clean;
  • database rebuilt from scratch: content hash e5497fd5…, the same as the e5736b1 and 371f1cf builds, so the feeds are unchanged;
  • feeds re-exported and copied to app/public/data; git diff --exit-code clean;
  • full suite under Python 3.12 (pytest tests/ -n 4): 2,667 passed, 18 skipped; the lane's tests: 99 passed;
  • mutation:
    • _covering reverted to the newest run overall: 2 tests fail, one of them the database differential;
    • _utc without the canonical-form check: 3 tests fail;
    • full_use_cost ignoring the read-back: 1 test fails;
    • every metadata mutant is refused for its own reason; with the static scan switched off, the recording simulation alone catches the 8 round-4 survivors.
• At e5736b1:
  • ruff format --check be nz scorecard_db tests pipeline and ruff check clean;
  • database rebuilt from scratch: content hash e5497fd5…, the same as the 371f1cf build, so the feeds are unchanged;
  • full suite under Python 3.12 (pytest tests/ -n 4): 2,638 passed, 18 skipped; the lane's tests: 70 passed;
  • mutation: with _covering reverted to the newest run overall, the new diagnosis test fails (0 diagnoses);
  • the tree is clean afterwards.
• At 371f1cf:
  • ruff format --check be nz scorecard_db tests pipeline (CI's scope) and ruff check clean;
  • database rebuilt from scratch, and two builds' content hashes agree (e5497fd5…);
  • feeds re-exported and copied to app/public/data;
  • full suite under Python 3.12 with CI's dependencies (pytest tests/ -n 4): 2,630 passed, 18 skipped; the lane's tests: 62 passed;
  • app: bun test src (65 pass), bun run lint (warnings only, in a file this PR does not touch) and bun run build;
  • the tree is clean afterwards.
• At 12b94b0: the rerun on the certified pair (artifact schema 2, run childcare-100k-20261025, 528 s, 26 GB max RSS).
  • Every free-hours head in every world and year equals the 661c14d run to the pound.
  • England's sums equal the UK's: all three offers test country == ENGLAND.
  • CBP counterparts lose the Tax-Free Childcare change outside England, £4m to £10m a year.

Review response, round 7 (GPT-6.1 Sol at 0311c08)

1. An object array carried a callback past both layers (minor). The review's carrier was a class defined in the script. Its __float__ reached calculate through a literal getattr(ab, "_resolve_parameter"), and the script handed it to set_input in an object array. policyengine-core 3.27.1 converts such an array with astype(float) (holder.py:278), which calls __float__; a stand-in stores it without converting. Earlier answers closed one delivery route at a time, and that had not converged, so this answer refuses the carrier. Three guards:

   • The scan refuses classes: no class statement, and no type() with more than one argument. The script defines none.
   • The scan refuses a string naming calculate or _resolve_parameter, so a literal getattr cannot reach the getter past its path check.
   • The stand-in refuses arrays of any object dtype, structured ones included.

   Each guard alone refuses the review's carrier, and with both scan rules off the scan accepts it, which reproduces the hole. The carrier is kept as object_array_callback, with three neighbours: the getter named by a string with no class, a class whose __float__ calculates, and the same class built with type(). "any" leaves the vetted attributes (now 81), because item 2's guard no longer calls receiving.any().

2. Zero receiving weight wrote NaN (the review's nonblocking producer boundary). year_block tested receiving.any() over records. Records that receive the offer but weigh nothing in total therefore gave 0/0, which the note printed as nan. year_block now writes no mean when the weighted count is zero. Ingest keys the mean on that weighted count, not on the record count: a finite mean with positive weight, or no mean with zero weight. It refuses anything else, including NaN, infinity and bools. The new test builds the block with year_block on a stand-in whose receiving units weigh nothing, and the note then says no family receives the hours. On the committed run every receiving weight is positive, so its numbers and the build hash are unchanged.

Review response, round 6 (GPT-6.1 Sol at 491bb2e)

1. A stub hid a callback in the compute script (minor). The vetted _resolve_parameter reaches any attribute, so a class defined in the script could call calculate from its resolve() without a direct call. Because the stand-in run stubbed relative_to_root with str, that resolve() never ran. My claim that the vetted names leave only direct calls was wrong. Three changes:
   • The scan holds _resolve_parameter to direct calls on paths it resolves to gov. parameter paths, and refuses it aliased.
   • The stand-in run keeps the real path helper. Every stub and stand-in engine method refuses an argument of a type the script could define.
   • Regressions: the review's mutant, plus five neighbours: the getter on a built, a literal and an aliased path, and the same callback handed to an engine method and to a stub. The review's exact diff, applied to this head, is refused by the scan. The stand-in run alone sees its employment_income, and the contract refuses it.
2. The stand-in breakdown was wrong (nit). The corrected counts are in the invariant, and test_what_the_stand_in_run_alone_makes_of_each_mutant now pins each mutant's outcome. Writing that test showed that the test loader did not register modules as an import does. So module_attribute_write crashed with a KeyError in the harness, where a real run rebinds the name. The loader now registers them.
3. No recipients under current law (the review's nonblocking robustness note). The note and the diagnosis now say that no family receives the extended hours, instead of formatting a mean the run did not record. Ingest refuses a mean that disagrees with the receiving record count. The test builds the no-recipient block with year_block itself, on a stand-in.

Unchanged, as the stated scope says: _in_force refuses overlapping windows that disagree, and the private published_result is reached only through the guarded _published.

Review response, round 5 (GPT-6.1 Sol at 39d0027)

1. The reserved prefix was case-sensitive (minor). The comparisons view's LIKE ignores ASCII case, and startswith did not. Ingest (the history and the published row) and the compute CLI now refuse any id the view's LIKE matches. A Hypothesis differential checks both tests against SQLite evaluating the LIKE read from db.py, including cased prefixes and non-ASCII look-alikes. The history, stage, published-row and CLI tests carry mixed-case ids.

2. Full-use evidence accepted records that do not show it (minor). full_use_cost now reads the reform in force on 30 April of the year. It parses the dates and treats start and stop as both in force, as policyengine-core 3.27.1's Reform.from_dict and Parameter.update apply them. It also checks the sentinel, the executed limits against the main world's, and the exact holders set_full_use checks.

   • The review's ten accepted counterexamples are regression cases, with three more. Each is refused on the condition it breaks.
   • The converse is fixed too: per-year windows from the shared reform_dict, and a world with no recipients, are now kept.

3. The metadata recorder skipped a helper and branches (minor). Two changes:

   • The scan now refuses any import, builtin or attribute name that is not vetted. So every route to calculate in the script is a direct call, which the scan resolves.
   • The stand-in run replaces the recording run. It runs main() end to end, through every world, year and hours branch, and only code outside the script is stubbed.

   Each layer alone refuses all thirteen survivors: with either layer switched off, the other still catches them. The review's 51-case matrix, re-applied to this head, is all refused. The invariant above now states what the two layers cover and their limits.

4. "Exactly" overstated conservation (nit). The body now says within £29 (measured), against the £1m tolerance, and gov_tax at £0.

The review also noted that "every path" needed scope: the invariant now names the public paths and says the private helpers are reached only through them.

Review response, round 4 (GPT-6.1 Sol at e5736b1)

1. The diagnosis selector could differ from the database's latest result (minor). This happened when an older run took the platform-grid- prefix, which the comparisons view prefers within a bundle. Ingest now refuses that prefix on every path, and so does the compute CLI. A real-ingest differential test checks that the view and _covering pick the same run for every claim on a same-bundle history.

2. UTC validation did not fix the order (minor). computed_at must now be exactly what datetime.isoformat() writes for a UTC time, so Z, .000000 and other spellings are refused. Duplicates are checked on that one form, and a Hypothesis property checks that text order is time order.

   The checks live in one check_history, called by load, results, certified_result, diagnoses and expected. This also closes the direct-call tie and duplicate-run-id bypasses noted under "Checks completed and limits". A test calls each path.

3. Metadata enforcement incomplete (minor). The scan now refuses each form the eight survivors used, plus match (the mapping capture), dynamic name access and attribute writes to module constants. A recording simulation of the functions that calculate must see exactly the names the scan resolves. All 15 mutants are regression cases, each refused for its own reason.

4. Diagnostic prose could contradict the recorded override (minor). full_use_cost is shared by the notes and the diagnoses. It refuses unless both full-use worlds record the override and read it back at 30, on the main world's reform in force that year. Ten broken records are regression cases.

5. Global-newest wording (nit). The lane README, docs/ARCHITECTURE.md and this body now say that each claim's latest is the newest run that computed its year. The round-2 response no longer says "three-run test".

Review response, round 3 (GPT-6.1 Sol at 371f1cf)

1. A partial newer run dropped diagnoses (minor). Diagnoses and the expected count now come from each claim's newest run that computed its year (_covering), not from the newest run overall. New test: a newer verified run that computes only 2031 leaves both 2027 and 2029 diagnoses and the tally of 2.
2. The metadata scan accepted rebinding (minor). The resolver is now strict. A name must be bound exactly once in its scope chain, by a plain loop or comprehension target over a tuple or a module tuple, or by one module assignment; anything else raises. Both round-3 mutants (a second generator rebinding v; v = inside the HEADS loop) are now regression cases, with three more.
3. Run-id uniqueness overstated (nit). The README and the docstring now say the id is unique per variant to the start second, and that ingest refuses two artifacts with one run id.

From "Completed checks and limits":

• Uneven refusal: certified_result checks the pair before the year check, and results and expected check every artifact. An unverified pair is now refused even when it computed no claim year; the unverified-pair test checks results, diagnoses and the tally, including an artifact with no years.
• Generated text: measured_notes reads whether the dataset supplies an input in the run's own year (dataset_inputs periods), not whether it supplies it in any year. A missing input is now described as the engine's default, and no longer as 30 hours.
• One newest run per claim: with diagnoses now per claim, "newest" must be a strict order. Ingest refuses two artifacts with one computed_at, and a computed_at that is not UTC (runs are ordered by the ISO string). The docstrings that said the newest artifact is every claim's latest now say it is the newest run that computed the claim's year.
• Hypothesis deadlines: the lane's three property tests now set deadline=None, as the repo's other property tests do. Round 3 saw 200 ms deadline failures under host load; the values were unchanged.

PR-body nits: the coverage test is described as three histories (one, two and two artifacts), and the years are split into the main worlds' years (2026–2030) and the diagnostic years (2027, 2029).

Review response, round 2 (GPT-6.1 Sol at 12b94b0)

1. Engine notes attached to pairs they were not checked on.
   • Each engine or run note now names its pairs (variants) and attaches only to their results.
   • Each pair's statements live in verified_pairs: why its run is static, its hours draw, and the pe_gap diagnosis, or none.
   • Ingest refuses an artifact on a pair with no entry, so a new pair gets no 2.89.2 explanation by default. The two-pair test is under "Pair scoping".
2. Coverage accounting. expected() now derives the tally from the claims, each run's computed years and the published file, not from a fixed per-run count. The coverage test covers a history where a run computes 2031.
3. Claim identity and missing reasons.
   • pe_missing moves from conditions (part of the id) into publication, so a new run never changes an id.
   • It is computed from the years each run actually computed. It names the pair to rerun when a year is among a certified pair's data years but no run computed it.
   • The exporter shows it in the feed as conditions.pe_missing.
4. Run-id collisions. The default run id is childcare-100k-<variant>-<UTC start>, not a fixed string.
5. Metadata test. The scan now resolves names in their own scope rather than across the file. A mutation (a summed variable left out of the metadata) is caught.

PR-body points:

• The verify wording now lists the checks in the order they run (see "What changed").
• The consumed-claim count is 6,812 on a fresh build.
• The PR Scorecard database: reform-keyed claims, populace-targets shape #8 row is pinned to its merged state (1b83dd46, 24 October). The 21 October draft figure (£1.009bn, ff881bd7) is gone from the table.

Review response, round 1 (GPT-6.1 Sol at 661c14d)

1. Runtime dataset: hashed at the path the simulation reports, before and after its reads, with the release bundle and data years checked.
2. England: counterparts are England-only. A new construction note covers it (pe-england-only), and the CBP note says so too.
3. Variant identity: the wheels are checked before any engine import and import origins before and after the worlds. Ingest refuses an artifact whose pairing differs from its variant.
4. value_verbatim: moved to publication, with an id-stability property test.
5. Annotation bases: the static basis now names the mechanism the engine uses. employment_income adds employment_income_behavioral_response, which is zero because both elasticities are zero; the old "only through apply_dynamics" wording was wrong. CBP's friction rule now cites pages 7 and 9, which are added to the extract. The full-utilisation note now rests on the inputs the run records.
6. Metadata: the missing variables are added, and a test checks every calculate() name in the source.
7. History: ingest keeps every certified artifact's run, and the README's promise is now implemented.
8. Tidy rows: both adapters emit the core fields. ARCHITECTURE.md now states that source facets may sit beside them when the ingester names them.

Follow-ups (not in this PR)

• #228: run the lane on policyengine.py's certified pair (6cba5d71, policyengine-uk 2.132.6). That means checking each pair-scoped note at that pin and adding its verified_pairs entry. The new run becomes the latest, and the 19 June pair's run and the published figure stay in the history.
• When policyengine.org serves the dashboard again, record the figure it shows as a new published row.
• Calibrating to CenTax's moments (Arun's suggestion): decided on 25 October 2026 that CenTax's administrative tabulations become calibration targets in Microcosm, where that work happens. Its model outputs, including every Table 4.2 cell, never qualify. Nothing in this lane changes.

axiom: n/a: scorecard data and a microsimulation comparison; no PolicyEngine country-model change.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
policyengine-scorecard	Ready	Preview	Oct 26, 2026 10:55am UTC