Repository navigation
Move the UK spine to geography first (#931) - #1538
Draft
juaristi22 wants to merge 55 commits into
Draft
juaristi22 wants to merge 55 commits into
juaristi22 wants to merge 55 commits into
Conversation
…ine-block machinery (#931) Per-clone engine blocks in the dense measure step, and the per-clone spine batches that follow, run policyengine-uk over part of the pool, so the formulas that read beyond their own households see the part. This adds the pieces the next commits wire in. - microcosm.frame.adapters.engine_population is country-agnostic: numpy and duck-typed simulations, no engine import. It holds the pool-share correction c_b = f_b * S_b / S with its zero rules, checks (a) and (b), and injection into a simulation, its baseline and every sibling, refused once a branch exists or a recorder is armed. A per-instance _run_formula recorder, which core's clone rebinds into later branches, and a known-periods walk feed a judge over declarations, parameter predicates and pass policies. A held value is judged unless the simulation holding it ran it after arming, and a run reaches the position window by the last year its own period covers. The verdict carries names, periods and booleans only. Household batches, the nesting check, the scatter back to one engine's order, a float32 share bound that follows numpy's pairwise-sum leaves and counts the adds forming each key, and a per-row bound for values that add share terms to others complete it. - microcosm.frame.adapters.policyengine_uk_population classifies pe-uk 2.174.1's 19 WHOLE_POPULATION_VARIABLES: three corrected shares, two stored inputs, eight zero at policy, attendance inert through private_school_vat_change, six refused ranks. It adds the window from REBALANCING_YEARS (2026 on), the constructor's structural reform factories, sha256 pins of the sources the classes rest on, the spine batch column, and uk_engine_population(), which imports pe-uk and validates only when called, failing closed. Nothing imports either module yet, so no kernel, stage or target hash moves and no output changes. Engine-free tests drive the machinery on fake core-style simulations, covering each guard condition a small change to the module could drop. Engine tests hold the declaration to the installed engine: list, adds, pins and what they cover, predicates at 2024 and 2025 and moved off current law, reform factories, window, the readers of attendance and private-school VAT, and drift lists of axis-free reductions and of whole-array branches in formulas and the engine functions they reach. They correct two synthetic clones that differ to one engine over the pool within the float32 bounds, household_tax per row, with household-local outputs bitwise equal. A build test holds the batch column and the share inputs to the build's own constants. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The annual-income simulation carries each input's other spells from the dataset's year by the factor the engine projects that input by. The factor was the median of the engine's own year-on-year ratios over the rows the simulation holds. That is a statistic of whichever block ran, so per-clone engine blocks that differ would carry the same spell by different factors, and no block would match one engine over the pool. Following María's ruling of 6 Nov, the factor is now the product of one plus the engine's own rate, for each year after the dataset's, in the growth series policyengine-uk's dataset projection uses for the input: average earnings for pay, mixed income per head for profit. The rates are read from the simulation's parameters. The engine's rows must still agree with the factor within 1e-5, or the resolver refuses, as before. This moves data on today's national and dense lines. The median sat within float32 rounding of the parameter, so an annual input with other spells can move by one float32 step. The *_annual matrix entries, calibrated weights and H5 bytes can move, and other_spells_engine_factors changes in every resolver receipt that carries it. A K=1 parity check should expect *_annual inputs within one float32 step, not an identical H5. The full target and problem kernels and the national target kernels re-key with measure_simulation; no spine stage does. An engine-free test holds the factor to the parameters and to the rows' tolerance. An engine test pins the declared series to policyengine-uk's own uprating map and checks the factor on a synthetic frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he whole-population formulas it runs (#931) After the move the dense measure step's per-clone engine blocks are no longer identical copies, so policyengine-uk's share formulas (shareholding, corporate_land_value, consumption_shareholding) would read a block's denominator instead of the pool's. - The shared engine_population module gains EngineBlock, the life of one block on its own engine. It validates the declaration, refuses the position window and periods before the data, and runs checks (a) and (b). It sets each share formula to its own value times c_b = f_b * S_b / S in the simulation and its baseline, and arms the formula-run recorder. Every simulation built later gets the same arrays and recorder. At the end it checks that each still holds the block's keys and weights, then judges them all. A record pass records refusals and a raise pass raises; work after the verdict is refused. The country passes its declaration and a pass. validate_pool_shares and PopulationDeclaration.correction_reads come with it. - UKMeasureResolver takes engine_pool_shares (scratch mode only) and opens an EngineBlock with the UK declaration and the measure pass, which records and never raises. A sibling that would set a key, its inputs or the weights is refused. close_engine_block judges the block. - full_measure builds each block's pool shares from the totals the representation already records and passes them to the block's resolver. After the blocks run, exact holds only if every block validated the declaration, passed both checks, applied the correction in every simulation it built, kept its keys and weights to the end, agreed with the others on its growth and annual-income factors, and got a clean verdict. exact_basis names this basis. exact_scope says float32-level, not bitwise, and names the Universal Credit rebalancing draw the window refuses. The identical-copies checks stay as diagnostics. block_sensitivity matches the land rows' engine variables, so present_in_this_run is no longer always empty. - The two frame modules join UK_TARGET_COMPILE_MODULE_NAMES. The fake engine the shared tests use moves to test_support so the build tests can use it too. No data changes on today's lines. Identical copies take a correction of 1 and give the same matrix and local metrics, bitwise, as without it, and a single-block resolver records none of the new keys, so national receipts keep their bytes. Today's K=15 dense receipts change by design: - the representation gains exact_basis, share_correction, population_reach and annual_income_projection_factors; - exact_scope is rewritten, and the release manifest's engine_measurement limitation copies it; - block_sensitivity is rewritten; - each block's resolver receipt gains engine_pool_shares and engine_population_reach; - exact now also needs a clean guard verdict, a stricter release guard on that line. The full and national target kernels re-key; no spine stage does. On a synthetic pool of three clones that differ, three blocks match one block over the pool: household-local and annual rows and every local metric bitwise, and the land rows and each block's share formulas within the float32 bound. Uncorrected land rows miss by more than 1%, and a decile read, a 2026 period and a counterfactual on corporate wealth are each refused. Engine-free tests run EngineBlock on fake engines. They drive checks (a) and (b), a sibling that moves its keys or weights, the window and a raise pass, and hold its receipts to the representation's finalisation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…931) One simulation over the move's K=15 spine pool does not fit in memory, so the UK adapter runs a call over more than one clone clone by clone. It decides from the frame, never from a constructor flag: a household table carrying household_clone_index with more than one clone batches, and a frame without the column, or with one clone, takes the single simulation it always has. National never batches. - The shared engine_population module gains materialize_in_batches and frame_key_sums. Before any engine is built, the driver judges the call's requests: a period in the rebalancing window, a refused rank, a stored formula the frame does not supply, and attendance or its levels and carriers while it is unstored. It forms each batch's share from the frame, c_b = S_b / S at unscaled weights. Each batch, in ascending clone order, then runs an EngineBlock under the spine's raising pass: checks (a) and (b), the pool's shares set in the simulation and its baseline, the recorder armed, the requested values, the verdict. The engine is deleted and collected before the next is built. Every batch must grow the keys and the weights by one factor. People come back in the frame's order and groups by id, as one engine returns them. The driver refuses a recording pass, since its arrays carry no verdict. - PolicyEngineUKEngine passes the UK declaration and the spine pass. It gains simulation_batches(frame), parameters(period) for the stage-52 restatement, and batch_record(). Its optional clones, the spine's K once G1 adds it, refuses a frame carrying any other number of clones, so no call runs per clone or over a sub-pool. microcosm.frame.rules and the RulesEngine.materialize signature are unchanged. - The spine sidecar's rules_engine gains engine_batches only once a call batched. The engine identity is still hashed from the provenance alone, and no batch cap is added. - graph_kernels binds engine_population and policyengine_uk_population beside the adapter in all 16 engine-calling stages. A test derives those stages from their engine calls. No data changes on today's lines: no spine frame carries household_clone_index before G1, so every call takes the single simulation, unchanged. Identities move. Every spine stage kernel hashes graph_kernels.py, so every spine node re-keys, and the 16 engine stages also bind the two modules. engine_population.py is a target compile module, so the full and national target kernels re-key again. The spine graph declaration (the H2 fixture uk_spine.json), source_stages.json and the coverage manifest are unchanged. An engine test runs a synthetic three-clone frame whose clones differ, interleave by id and list people out of order, at the dataset year and projected a year on. Against one simulation over the same frame, the share formulas match within the float32 bound, household_tax within its per-row bound, and household-local values bitwise, with people and groups in one engine's order. A one-clone frame is bitwise the single simulation. A batched attendance request, a decile, a 2026 period, unstored months and a frame with fewer clones than K are each refused before any simulation is built. Engine-free tests run the driver on fake engines, one engine alive at a time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ine values (#931) policyengine-uk's attends_private_school ranks household net income over the people a simulation holds, so once a spine's engine calls run clone by clone, one clone's simulation would rank that clone alone. Stage 52 now restates the formula's last step itself, over every person in the frame: the members' income tax and National Insurance summed per household as policyengine-core sums them, net income from the household's market income and benefits, the household's weight once per member, microdf's MicroSeries.percentile_rank (the rank the formula calls) over the people of positive weight, the attendance rates and population factor from the engine's own parameters (parameters(period), at the two paths PRIVATE_SCHOOL_ATTENDANCE_PARAMETERS names), the one-million floor, and the stored draw for under-18s. Every input is the engine's own value at the calibration year, decided by a household's own records. - Where the engine computes the frame in one simulation (national, and every spine today), the stage asks for the engine's attendance beside those inputs, stores the engine's flags, and raises if the restatement differs from them bit for bit, so each such build re-checks it on its own data. Where the adapter computes the frame clone by clone (simulation_batches above one), it asks for the inputs only and stores the restatement. An engine without simulation_batches runs one. - The evidence gains rank_route (engine_with_parity or pooled_restatement) and engine_parity_checked, a boolean: whether the engine's flags were checked. A failed check raises, so neither records a mismatch. weighted_pupils stays. The parity error names the formula, never a count. - The stage's notes in sources.yaml describe both routes. The notes in source_stages.json are re-derived from it, and the release input coverage manifest is regenerated (its source_manifest_sha256 fields). - Left for G1, which makes the clone-by-clone route live and regenerates the spine fixtures: the stage's source line ("evaluated by the pinned engine"), the notes' sentence that the stage materializes the engine's attendance, and the H2 fixture's sources/fixture.json, which keeps the stage's earlier notes until tools/graph_uk_spine_fixture.py runs. All three hold while every call takes one simulation. The store_rules_engine_input operation describes both routes: each stores attendance as an engine input from the stored draw, at the calibration year and the spine's design weights. No data changes on today's lines: no spine frame carries household_clone_index before G1, so every call takes one simulation and stores the engine's flags, as before. Identities move. Stage 52's kernel hashes private_school_attendance.py, so stage 52 and every node after it re-key. The packaged resources enter the full and national target kernels' country-spec fingerprint, so those re-key again. National's stage-52 evidence gains the two fields. The spine graph declaration (uk_spine.json) is unchanged, since stage contracts do not cover notes. An engine test runs a synthetic pool of three clones of twelve households: some alike in every clone and some not, most with several children, weights spread about one to ten, ties, a household of no weight, and each clone under the floor while the pool is over it. Household ids interleave the clones and the people are listed out of order. Each child's draw sits 3% under or over the rate the formula gives it over the pool, from the formula's own percentiles and the engine's parameters, so a rank, rate, interpolation or factor a few percent off would store other flags. A spy on microdf's percentile_rank shows the restatement hands it the formula's net incomes and weights and gets its percentiles back, bit for bit, in one simulation and clone by clone. The stage run clone by clone stores what one simulation stores, where a clone's own simulation imputes nobody; a population factor of zero, or 6% lower, breaks the parity check; below the floor nobody attends on either route, while design weights just under it impute on both, since the formula reads the engine's weights at the calibration year; and a frame of one clone stores the engine's attendance as the stage asked for it before. Engine-free tests run both routes, the parity refusal and the floor on a fake engine with a stand-in rank. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…raph doc (#931) The dense runbook and the local pre-flight still said a per-clone engine run was exact when its blocks were identical copies scaled to the pool. Exact now means every block corrected to the pool's share denominator with a clean guard (exact_basis pool_share_injection_with_population_guard): equal to one engine over the pool to float32 rounding, not bitwise, with nothing computed in policyengine-uk's Universal Credit rebalancing years (2026 on). - docs/uk-dense-release-assembly-runbook-762.md says so where it describes --engine-blocks, the pre-flight and the engine_measurement limitation. It adds that a calibration year or dated measure's year of 2026 or later leaves every per-clone run inexact and refuses every batched spine call, and that the pre-flight does not require exact_basis, so a receipt from before the correction passes too; that is open for Max. - docs/uk-full-build-graph.md gains a section on the corrected measure blocks and their guard, on how a spine engine call over more than one clone runs clone by clone and what refuses it, and on stage 52's rank over the whole frame. - tools/preflight_uk_local_release_candidate.py: the docstring and the comment over the engine check only. The gate code is unchanged. test_uk_local_release_preflight gains two cases: a per-clone resolution whose representation has another mode, or another block count, fails however exact it says it is. - Two changelog fragments cover group (0). One describes the engine blocks, the spine batches, stage 52 and the shared modules. The other describes the annual-income factor for other spells, the one data change, which can move an annual input by one float32 step on both lines. Only comments, docs, fragments and tests change. No kernel hashes these files, so no node key, H2 fixture, source_stages.json or coverage manifest changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…estatement (#931) Review follow-ups to batching spine engine calls by clone. - The spine sidecar's engine_batches is now read from the final frame, never from the calls this process ran. A resumed spine whose engine stages hit their checkpoints runs no kernel, so it under-reported batches; now it records what a fresh run does. PolicyEngineUKEngine.batch_record(frame) takes the same decision and refusals as simulation_batches. - PopulationDeclaration gains restated: stored formulas a caller may compute over the pool from household-local engine values, with the parameters each reads. Each pe-uk simulation builds and edits its own parameter tree, and parameters(period) is the adapter's fresh copy. So a new check_restated_parameters holds every batch's tree to the adapter's wherever a restatement reads it. The UK declares attends_private_school with its attendance rates and population factor, the parameters stage 52 reads. - PolicyEngineUKEngine(clones=...) accepts any whole number. A frame that drops the clone column still takes one simulation; G1's integration check rules that out after G1. - The test that finds the engine-calling stages matches a materialize call on any receiver, and any transform taking an engine. It keeps isa_subscriptions, which only checks the country, on an asserted allow-list. No data changes on today's lines: no spine frame carries household_clone_index before G1, so every call still takes one simulation and no sidecar gains engine_batches. Identities move again: every spine node re-keys through graph_kernels.py and the engine modules, and the full and national target kernels re-key through the two frame modules. The spine graph declaration (uk_spine.json), source_stages.json and the coverage manifest are unchanged. An engine test adds a frame carrying none of the keys' inputs, which gives the pool's values bitwise. A batch whose own tree moved a restated parameter is refused before it computes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The WAS donor's income, DVTotinc_bhcR8, is net regular household income before housing costs less council tax: HBAI's concept. Both WAS-trained stages matched it to the engine's household_net_income, which also takes off stamp duty, the TV licence and the business rates and corporate stamp duty that shareholding spreads over the households the engine simulates, and adds benefits in kind. At was_lisa (stage 24) those shares already read the corporate wealth was_wealth had just drawn, at the spine's weights before calibration. Both stages now read the engine's HBAI net income before the reset to zero, the concept LCFS, ETB services and NTS read (#1164), named hbai_household_net_income on donor and recipient (María's ruling, 6 Nov). - was_wealth materializes UK_HBAI_NET_INCOME_VARIABLE, and UK_WAS_ENGINE_PREDICTOR_NAMES gives it the predictor name UK_HBAI_NET_INCOME_PREDICTOR; the donor cleaning maps DVTotinc_bhcR8 to the same name. The chain's segments, their order and seeds are unchanged. - Its derive step declares the donor column in direct_maps, and the drift check refuses a declared direct map the cleaning does not read. - was_lisa takes the WAS household predictor, so the ownership logit, the balance forest and the realised profile's income tertiles read it. The profile key becomes by_hbai_household_net_income_tertile and the logit's feature log1p_hbai_household_net_income. Its declared engine reads are now what it materializes (UK_WAS_LISA_ENGINE_PREDICTORS): num_adults and num_children, from the FRS family roles since #1095, leave the list. - sources.yaml carries the declarations and both stages' notes. source_stages.json is re-derived from it, the release input coverage manifest is regenerated (its source_manifest_sha256 fields), and so is the H2 fixture: sources/fixture.json carries the two stages' declarations and notes, and stage 52's notes from 019f5d4, which waited for the tool to run; uk_spine.json carries the two stages' new contract hashes. No synthetic fact in the fixture moves. - docs/uk-lisa-1003.md and a changelog fragment describe the concept. What moves: every WAS draw can move on both lines, and with them the Lifetime ISA draws and the later stages that read wealth (means-tested capital, CGT selection, property components, high-value support). The size is unmeasured; a sampled f010 arm measures it. was_wealth, was_lisa and every later spine node re-key. Tests: the WAS recipient reads the engine's HBAI income, not a household_net_income decoy, and the donor maps DVTotinc_bhcR8 to the same name; the packaged declarations name both sides; drift to the old engine read or a wrong direct map is refused; the LISA recipient and donor carry the WAS predictor and its declared engine reads are what it materializes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of 640b2cf. Its text said the WAS and Lifetime ISA stages read one income concept on both sides. The code matches the ruling, but the two sides are not the same income: policyengine-uk's HBAI net income also counts HBAI's in-kind items (free school meals, school milk and fruit, the free TV licence's value, Healthy Start vouchers, the Tax-Free Childcare top-up) and the Warm Home Discount, and the WAS user guide defines the income behind DVTotinc_bhcR8 only as total net regular household income less the council tax paid, so how it treats pension contributions, maintenance paid, student loan repayments and negative incomes is not verified. - The was_wealth and was_lisa notes, docs/uk-lisa-1003.md, the changelog fragment and the code comments call the recipient's income the engine's closest concept to the donor's and name those residuals. The was_wealth notes keep the income among the predictors that are not fully like-for-like. - household_net_income is described as adding the imputed value of NHS, education and transport spending, not "benefits in kind", which HBAI's income also counts in part. - Only the LCFS and ETB services stages name the income as a predictor; NTS reads the same engine variable to build gross income. The comments and test docstrings say so, and the two tests are renamed to match. - The changelog's What moves adds the calibrated weights on both lines. - source_stages.json is re-derived from sources.yaml (the two notes), the release input coverage manifest is regenerated (its source_manifest_sha256 fields), and the H2 fixture's sources/fixture.json carries the two notes. uk_spine.json does not move: notes are outside the stage contract hash. What moves: nothing in the data or any contract hash. Text only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#931) María's ruling of 6 Nov on the move, in line with workstream 2 of microcosm#1521 and the first must item of Vahid's review there. Two of the three ONS land rows cannot usefully bind weights: - ons.land.corporate_land_value (E43B, £2,063,534m): policyengine-uk's corporate_land_value hands out its aggregate by each household's share of the weighted corporate-sector wealth, so a released file meets that aggregate at any weights. Bound to the column measured at the starting weights, the row pinned only the weighted mass of the WAS corporate and private pension wealth, which no other row targets, not land. - ons.land.land_value (E44R, £6,872,455m) counts the land of financial corporations, government and NPISH, which the engine's land_value does not hold. With the household row (E44N, £4,485,171m) and the corporate row it asked for £323,750m that no weights could supply. Both get entries in uk/calibration_measure_exclusions.json in the obr.council_tax_england shape (tracking microcosm#1521, approved 2026-11-06, expiring 2027-02-06) and are measured on every evaluation. ons.land.household_land_value stays bound. Its binding note in uk/uk_population_targets.json said households_and_npish; E44N is the households sector alone, as the Chronicle package labels it, and the note now says so. The contract is hand-written. tools/generate_uk_target_references.py reads no binding note, so target_references.json and the membership report do not move (their regeneration test needs the pinned Chronicle feed, which is not on this machine, and skips here). What moves: the solve drops two rows on both lines, so calibrated weights move slightly; a calibrate-only rerun on the move's spine attributes it. Released corporate land value still equals the engine's aggregate. Tests: the packaged register's count and census take the two rows, and a new test pins their fields, the published values the reasons quote against target_reference_membership.json, the household row staying bound and its corrected note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…931) Follow-up to 410e822. Its test pinned the register's two entries, their fields and the published values they quote, but not what the entries do to a run. The new engine-free test compiles the three committed ONS land references with stand-in values and applies the packaged register's land entries, as both target loaders do after reconciliation. The solve surface keeps ons.land.household_land_value alone, the exclusion receipt carries the two entries' review fields, and the compiled register, which every run keeps beside the solve surface as its band-edge register (a national run writes it as national_contract_registry.json), still holds all three. On a synthetic four-household frame the national problem encoder binds the household row alone, with the receipt and the compiled register's digest in its bindings, while the compiled register materializes and scores all three rows at their weighted values. What moves: nothing. Test only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of 410e822 and 161c777. The two entries hold the right rows, but some of their text contradicted itself, the record of what moves was short, and one claim in 410e822's message was wrong. - The total-economy reason, the changelog, the test docstring and the census comment now tie the £323,750m gap to the rows: E44R also counts the land of financial corporations, government and NPISH, which neither E44N nor E43B covers, and policyengine-uk's land_value is the sum of the two columns those rows read. The old text said the engine's land_value does not represent or hold financial corporations' land, while the corporate entry says policyengine-uk describes its corporate aggregate as the land of non-financial and financial corporations. That entry now says "describes as", not "labels". - The census comment no longer says the corporate row is met by construction: the column meets the engine's own aggregate at any weights, not E43B. The adjudications give Vahid's suggestion in his order (met by construction once the engine's parameter comes from the target's ONS series), and say the calibrate-only rerun is to attribute the change, since it has not run. - The two held rows' binding notes in uk/uk_population_targets.json record the hold, as the other held rows' notes do. The parity register's land concern names the two exclusions; uk_data_target_parity.json is regenerated with tools/census_uk_data_target_parity.py, and --check passes. The UK_BLOCK_SENSITIVE_MEASURE_COLUMNS comment and its test's comment no longer say the national targets bind the two variables. - 410e822's message said the target-reference regeneration test skipped because the pinned Chronicle feed is not on this machine. The feed is here. tools/generate_uk_target_references.py, run with the test's arguments on that feed (facts and manifest hashes and the row count match uk/chronicle_feed.json) and on this commit's contract, reproduces target_references.json and target_reference_membership.json byte for byte. It ran directly, through the engine slot, rather than inside pytest, whose own load of the 10 GB feed would double the memory. The local generator keeps only local-level contract rows, so the land rows' notes cannot reach the local surface. What moves, all from 410e822 (this commit is text, a regenerated parity register and tests): - Both lines split each family's equal share of the objective evenly over its bound rows (the dense line within each grain, where the land rows are amounts). ons.land.household_land_value, now the ons_land family's only bound row, carries the family's whole share: three times its loss weight before, on both lines. No other row's weight changes. The calibrate-only rerun measures this together with the two dropped rows. - The national frozen scoring register (national_target_registry.json, the approved register) loses the two rows, so a national run given a frozen register derived before 410e822 refuses, as with obr.ni_employer. The dense loader checks the reconciled register, which keeps them. - The incumbent and release-cut scores, uk_target_fit and calibration_diagnostics.json read only the rows the solve binds, so the two rows leave them. - Neither line resolves land_value or corporate_land_value any more: the dense measure receipt's national_inputs falls by two, and a per-block dense run's block_sensitivity.present_in_this_run is empty. Evaluations that score the un-excluded register (tools/evaluate_uk_incumbent_surface.py) still measure the two rows, as they do every held row; 161c777's test shows the compiled register can score them, doing the scoring itself. - The dense release lists the two exclusions among its limitations, each with the first 300 characters of its reason, which cite only published ONS figures and code facts. - The edited files are UK country-spec resources, so the national and dense target, measure and solve nodes re-key; full_measure.py feeds only those kernels. Spine nodes, source_stages.json, the release input coverage manifest and the H2 fixture do not move. The regional_home_values stage note still says the home value reaches three national rows, ons.land.land_value among them. It changes with group (c)'s rewrite of that stage, which regenerates source_stages.json and the H2 fixture anyway. Tests: test_the_corporate_and_total_land_rows_are_measured_diagnostics also pins the "4.7 percent" the total reason quotes and the two held notes. test_the_two_land_diagnostics_are_measured_but_not_fitted adds a bound row of another family (obr.tv_licence_fee) and pins the household row's family_equal weight at three times its share beside the other two land rows, with the other row's weight unchanged; its docstring says it applies the entries as the loaders do, without calling them. test_uk_measure_simulation, test_uk_full_measure, test_uk_data_target_parity, test_uk_population_targets, test_uk_incumbent_surface_evaluation, test_spec_source_citations, test_uk_target_references (its feed cases skip without CHRONICLE_UK_FACTS; the generator run above stands in for the national one) and test_uk_solve_scope pass, and ruff is clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every UK copy layer took its id offset from ten to the digit count of the largest id it saw. Once (b) clones households at the spine's start, every offset would grow with K, and the K=1 national spine would stop being the clone-0 slice of the K=15 spine. Copy ids now come from fixed decimal fields declared in uk_runtime/copy_id_table.py, so every copy id is a function of its source id and its copy path, at today's stage order. The table is UK-local (María's rulings of 7 Nov); the shared table, a declaration digest and the int32 check come with the follow-up stage. - The table, least significant first: vintage (10 ordinals at 10**8, kept for pooled FRS vintages; nothing mints it), geographic_support (100 at 10**9), spi_support_channel (3 at 10**11: the SPI copy at 1, the band donor at 2), high_value_support (6 at 10**12), cgt_support_split (10,000 at 10**13), cgt_incidence_clone (2 at 10**17) and cgt_residential_split (8 at 10**18). The largest mintable id, about 7.2e18, is checked below int64 when the module loads. Helpers: radix, require_below, mint (refuses an ordinal outside its field, a row that already carries one and an int64 overflow), ordinal and parent. - Mint sites: frs_spine refuses raw ids at or above 10**8. Stages 16 (with its _remap_ids, which had no overflow check), 17 (sources read from the source-id columns), 26, 42, 43 and 46 mint through the table. Stage 42 refuses a household needing more than 10,000 copies with a fixed message. The anchor (45) pairs each twin by the twin field's radix. The executor takes each id-offset stage's declared radix, refuses an incumbent id at or above it and recovers parents on exact int64 ids; the row walk it replaces upcast all-numeric rows to float64, inexact above 2**53. Refusals print counts, never ids. The tests-only replace path mints through the table and refuses a base whose ids reach the SPI field. - Dense: the dense expand keeps its multiplier above the spine's largest id. A table-minted spine's twins and arms leave no int64 room above them, so every dense entry point (uk.full.expand, the ladder and crosswalk clone routes, the row-wise tool's plans) refuses n_clones > 1 in one message naming #931 and the clone-first spine. Dense K>1 builds wait for (c). K=1, and a spine without copy rows, expand as before; the expand kernel no longer multiplies an empty column by a multiplier above int64. - Declarations: the id_remapping texts in sources.yaml, source_stages.json and the code dicts name the table field. stack_zero_weight_donors and stack_income_band_donor_households gain the key, required by sources.schema.json and checked against the code. copy_id_table is bound into frs_spine and every minting stage, spi_support into stages 16 and 17, rowwise_geography into 43 and 45 and cgt_support into 26. What moves: base FRS rows keep their ids. Every copy row's five id columns change: SPI copies +10**11, band donors +2*10**11, high-value copies k*10**12, CGT copies k*10**13, twins +10**17, arms j*10**18. Relative id order is unchanged at K=1, so sorts, ties, ranks and folds are unchanged. Values move on copy rows through every draw keyed on the copy's own id: stage 18's stage-2 fill, stages 19-24, 27 and 28-41, every twin's Advani-Summers prior (so the CGT gains layer re-realises), stages 48-54 and policyengine-uk's warm_home_discount_random_draw on copy households. Base rows move second-order through pooled steps: stages 26-28's selections and walks, the residential solve, the anchor, the CGT walks and calibration. Draws keyed on person_source_id do not move. National ids pass 2**53 from stage 42 on. Dense K>1 builds are refused until (c). Every node re-keys. The coverage manifest's source digests and the H2 fixture are regenerated: the fixture's declarations, and its synthetic CGT asset-type, property purchase and rental facts, which are sized to the synthetic spine's copy-row values. Left for 1b: the identity tool's decoders still read the retired rule, so 11 tests in test_uk_identity_stability_receipts fail until 1b; also size_evaluation's decoders, the remaining id-printing messages and the stochastic_assignment binding sweep. Tests: the table's fields, bounds, order and ceiling, and the helpers' round trips and refusals; each copy stage's ids against its roots, found by lineage; a K=1 chain through all six layers sorts as the retired rule did and keeps base ids; clone-first K=1 and K=3 frames nest exactly and the retired rule does not (test_support/microcosm_build/ uk_clone_first_frames.py); the anchor pairs at a one-digit CGT ordinal; the executor's refusal and parents, exact above 2**53; the dense refusal and the K=1 path; the fixed CGT message; an AST guard against the retired rule; the binding tests. H2 parity passes on the regenerated fixture. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three reviews of 22bcdbe found texts that overstate or misstate what it does, and refusals and dense paths that no test drives. This commit corrects the texts and adds the tests. The identity tool's decoders and the other items ruled into 1b stay in the next commit. - Stage 46's notes said the geography identity kernel and the identity tool read the arm rule. The kernel reads lineage columns, and the tool decodes the retired rule until 1b, so the notes now name the executor only, as the module docstring does. source_stages.json is re-derived, and the coverage manifest's source digests and the H2 fixture's notes are regenerated. - The row walk the executor replaced read a row of only integer and float columns as float64; an all-integer row stays int64. The _source_lineage docstring and the changelog now say so, and the exactness test carries a float column, on which the replaced walk fails. - The dense refusal fires wherever the dense offset would overflow int64, as a spine with residential arms at multiples of 10**18 does at K >= 2. Its docstring, its message and the changelog now say that. require_below's message no longer says that only a later stage may fill the field. - The changelog says each copy row now sits at its parent's ids plus its layer's offset, where it used to sit a power of ten above the largest id its layer saw. It also says that national ids pass 2**53 from the twin on (and at stage 42 from ordinal 901), that only the new refusals print counts, that create_uk_spi_support_tables no longer takes id_multiplier, and that copy rows join across versions by lineage, not by build ids. - The CGT split's bound check moves into a helper so its boundary can be tested. The lineage test helper reports counts, never ids or keys. What moves: no value or id. Every node re-keys again, through the text in graph_kernels.py, copy_id_table.py and rowwise_dataset.py and the helper in cgt_support.py. Tests: - Stage 16 on clone-first K=1 and K=3 frames nests exactly. - The executor takes the declared radix where the frame's maximum would give another, names each stage's own field, and refuses parentless rows by count. - The id_remapping texts quote the table's radices. - Stages 26 and 42 mint from the immediate parent on frames with SPI copies in FRS id spaces. - frs_spine (end to end), stages 42, 43 and 46 and the replace path refuse incoming ids by count. - The CGT bound holds 10,000 rows and refuses 10,001. - A drifted SPI id_remapping is refused. - The copy-id binding set is derived from imports. - The dense refusal and the K=1 path run through the pool expand, the crosswalk route, both dry-run plans and the graph's uk.full.expand, on ids above 2**53. - The twin's negative-prior count is pinned. - H2 parity passes on the regenerated fixture. The 11 tests in test_uk_identity_stability_receipts still fail until 1b, as at 22bcdbe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
22bcdbe and its follow-up 2f0f8ce moved every mint site, the executor and the anchor onto the copy-id table's fixed radices. Two decoders outside the stages still used the retired rule. The identity tool read the CGT support copies and the residential arms at ten to the digit count of the largest pre-split id. The dataset-size evaluation multiplied by a dense multiplier of 10**19, which a one-clone expansion of a table-minted spine records, and overflowed int64. This commit moves the identity tool onto the table and both decoders onto exact integers. It also makes the copy path's refusals count-only, adds static guards against the retired rule, and binds the shared keyed-draw helpers into every stage that draws through them. - Identity tool: _support_copy_lineage and _residential_arm_lineage take each copy's ordinal and parent at its field's declared radix (copy_id_table.ordinal and parent). They read the household table only, as exact int64; a non-integer id column is refused. Each receipt's id_multiplier now records the declared radix (10**13 for the support split, 10**18 for the arms). Before any receipt, the tool refuses a spine whose copy ids disagree with their lineage columns under the table, as every spine built before the table does. The refusal is one count-only message saying to run the tool at the commit that built the spine. The refusal and the decode are two small functions, so a retired-rule path could replace them and nothing else. - Size evaluation: dense ids decode back to spine ids exactly, at the multiplier the expansion recorded (the new id_multiplier argument of weight_tables), else the spine's, else the clone-0 ids'. 10**19 is included. A clone offset that int64 cannot hold, a non-integer id column and a multiplier that does not fit the dataset are refused by count. A dense file that keeps its true sources computes its spine stretch again; it used to fail on int(None). - Count-only refusals: - rowwise_geography: _remap_ids' overflow (it printed the multiplier and the largest id), clone_entity_frame's and assign_household_geography's duplicate checks (up to five ids), and the crosswalk route's check for duplicate household ids before cloning; - rowwise_dataset: its uniqueness and person-link checks; - spi_support: its uniqueness and person-link checks, the donor's duplicate and missing-source checks, and the selected-household check; - spi_spine: the stage-14 retained-leaf check; - cgt_structure: the CGT carrier check; - cgt_imputation: the redraw's missing-weight check. - Guards: only the dense line's own modules name id_multiplier_for_values until (c), each with its reason: rowwise_geography, rowwise_dataset, the package re-export, size_evaluation and the row-wise tool. Every module that mints, bounds or decodes copy ids reads the table, the identity tool among them, and names no retired rule. - Bindings: every spine stage whose module or bound helpers import microcosm.build.stochastic_assignment now binds it. A derived test holds the set, and a helper-only edit to it re-keys all 36 such stages and no control. No dense node calls it. - Texts: the stage-46 notes and the residential split's docstring say the identity tool reads the arm rule too. The changelog's binding sentence names only the helpers that mint or weigh copies (spi_support, cgt_support, rowwise_geography). What moves: no data value or id. Stage keys re-key on 31 stages that newly bind stochastic_assignment: frs_spine, frs_take_up, frs_person_draws, frs_household_draws, frs_brma, frs_annual_income_spells, spi_income_band_donors, hmrc_spi_income_spine, spi_housing_shell, was_wealth, was_lisa, property_components, property_purchases, nts_bus_travel, lcfs_consumption, etb_vat, etb_services, uc_reporter_redraw, spi_benefit_coherence, uc_capital_coherence, uc_take_up, uc_legacy_closure_take_up, pension_credit_take_up, child_benefit_take_up, uc_deduction_attributes, cgt_incidence_clone, cgt_incidence_anchor, salary_sacrifice, student_loans, isa_subscriptions and childcare_copy_take_up. Every spine node re-keys anyway through graph_kernels.py, as do the dense nodes that hash rowwise_dataset.py. Regenerated: source_stages.json (the stage-46 notes line), the coverage manifest's 25 source digests and the H2 fixture's notes line in fixture.json; uk_spine.json is unchanged. Tests: the 11 identity-tool tests that failed at 22bcdbe and 2f0f8ce pass again. The hand-built split frame is minted through copy_id_table; the other fixtures come from the real stages. New tests cover: - the old-spine refusal, through main and directly; - the arm decoder above 2**53; - a support copy at ordinal 9,999, which a float read misplaces; - the size evaluation's recorded, spine and clone-0 paths at 10**19, an older dense run at its own multiplier, the int64 refusal and the source-column stretch; - the count-only messages, with no id in them; - the executor's household_source_id branch; - the two static guards and the binding test. Synthetic mutations make each guard fail. The parent's weight_tables fails the four new size tests (OverflowError, TypeError). The 114 touched engine-free test files pass. The H2 fixture regenerates byte-identically, and H2 parity and the engine graph and ladder-clone tests pass in the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three reviews of 50b7ea1 found no must. This commit takes their should findings and the cheap nits, and corrects four claims 50b7ea1 made. - Bindings: stage 14 (frs_hmrc_spine_leaves) draws its employer pension contributions on keyed uniforms through employer_pension_contributions, but did not bind stochastic_assignment, because the derived test read direct imports only. It now binds it, and the test follows one more step: a stage draws when its module or a bound helper imports the helpers, or imports another first-party module's function that draws through them (a fixed point over the import graph, re-exports and module attributes included). spi_support_channel, which shares spi_spine with stage 14's draw and stage 18's SPI imputation but calls neither, is the one exemption, pinned to those two names. employer_pension_contributions itself stays unbound until the wider binding sweep. - Size evaluation: the spine-stretch refusal printed no count and chained a pandas KeyError listing every missing household id; it now counts the dense rows before the lookup and raises unchained. The misfit refusal says an inferred multiplier reads household ids only and names weight_tables(id_multiplier=), the one route for a recorded multiplier. - Count-only refusals: the row-wise tool's dry-run input checks now call the build's own (rowwise_dataset._require_unique and _require_person_links) instead of listing up to five ids, and the spine sampler's three refusals in national_sampling count families. - Guards: an AST guard refuses ten raised to a len(str(...)) power outside rowwise_geography (the dense rule), copy_id_table (field spans) and national_sampling's legacy candidate decoder, so the retired rule cannot come back written out inline. - Texts, correcting 50b7ea1: - a retired-rule decode would not replace two functions and nothing else; it would also thread the person and benefit-unit tables back through the two lineage functions and their four callers, and pair the twins in the tool at the three pair_clone_households call sites (the tool's comment now says so); - only the identity tool used the retired rule; size evaluation kept the dense line's own rule and overflowed at 10**19, and its non-integer id refusal names the dtype, not a count; - no caller passes a recorded dense multiplier yet: tools/evaluate_uk_dataset_size.py infers it from household ids (the docstrings say so); - the arm tests decoded a synthetic copy chain, not stage output; the stage-built tests below cover that now. The changelog names what each SPI and CGT stage binds and why, limits the count-only sentence to the UK modules (the shared graph executor's EXPAND checks are unchanged), describes the inferred multiplier, and says the identity tool refuses a pre-table spine with copy rows. What moves: no data value or id. Stage 14's key moves with its new binding; every spine node re-keys anyway through graph_kernels.py, and the dense population nodes that hash national_sampling.py re-key, as they already do through rowwise_dataset.py in this push. No generated file changes. Tests: new size tests decode an older dense run at K=15 on odd ids above 2**53 at all three bases (a float64 subtraction fails it), and pin the uncovered-spine refusal (no id in the message, its cause or the traceback) and the negative clone_index refusal. New identity-tool tests hold the old-spine gate to each stored ordinal exactly and to an empty vintage field, refuse a support copy whose parent the split did not copy, and run the gate on frames that stages 16, 17, 26, 42, 43, 45 and 46 built (stage 44's redraw done by hand) and the E8 residential, clone-pair and support-split receipts on the last of them. The dry run, the sampler, the CGT carrier and redraw checks, the SPI uniqueness checks and the support-copy refusals now have exact count-only messages pinned. Synthetic mutations make each new guard and test fail. The 30 touched engine-free test files pass (1,331 tests). The coverage manifest's check passes, the H2 fixture regenerates byte-identically, and H2 parity and the engine graph tests pass in the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CGT support split divided each selected household into ceil(w / 60)
copies of its own weight. Once (b) clones households at the spine's start,
each geographic clone carries 1/K of its source's weight, so a clone's count
would shrink with K and the K=1 spine would stop nesting in the K=15 one.
As María ruled on 7 Nov, the count now reads the geographic clone count
times the row's own weight, so every clone of a household gets the count its
source gets at K=1. Nothing moves at K=1.
- Count: cgt_support_copy_counts(weights, maximum_copy_weight, *,
clone_count) returns max(1, ceil(K x w / 60)). For K > 1 the quotient is
snapped to the nearest integer when it is within K ulps of it, so an exact
multiple of 60 keeps its count at any K (at K=7, 7 x (60m / 7) lands above
60m for 626 of m < 20,000 in a synthetic sweep). The snap gives up the
converse case: a quotient within K ulps above an integer counts one copy
fewer than at K=1. At K=1 nothing is snapped, and the count is
ceil(w / 60) bit for bit.
- Stage: split_cgt_support_households takes a required clone_count, and the
transform reads it from the frame (geographic_clone_count): one more than
the largest household_clone_index, or 1 without the column. A non-integer
column, a missing or negative index, and indices that are not exactly 0
to K - 1 are refused by count. Copies keep w / n, so each weighs at most
60 / K. The selection walk still accumulates each row's own weight, and
nothing sums geographic siblings.
- Receipt and gate: the receipt's parameters gain clone_count and nothing
else. stage_health compares K x the heaviest copy and K x the heaviest
selected household with 60, so a receipt at K > 1 no longer fails
falsely. At K=1 both checks and their messages are byte-identical, and a
receipt without a positive integer clone_count is refused. The identity
tool's recompute reads K from the pre-split table and passes it on; its
receipt is unchanged.
- Texts: copy_rule in the code dict, sources.yaml and source_stages.json
now reads 'copies = ceil(geographic clone count x household weight /
maximum_copy_weight); every copy carries weight / copies'. The stage notes
give the cap as 60 divided by the geographic clone count before the
incidence clone. The docstrings, the cap's comment and a changelog
fragment are updated to match. The mass-change reason is unchanged.
What moves: no value at K=1. On synthetic frames the pre-commit and new
code give the same results: the stage's ids, copy counts, weights (bit for
bit), mass records and frame content identity, the identity tool's receipts,
and the gate's verdicts and messages. The only difference is that stage
evidence gains clone_count. Implementation hashes move for
cgt_support_split, and for high_value_home_support, which binds cgt_support.
So node keys move from stage 26 on. The spine gates hash stage_health, and
every stage after the assembled gate waits on its verdict, so in a gated
build every node from stage 14 on re-keys. cgt_support_split's stage
contract moves with its texts. Regenerated:
- source_stages.json (copy_rule and notes, re-derived from the typed
projection);
- the coverage manifest's 25 source digests;
- the H2 fixture: the two declaration lines in fixture.json, and the
stage's contract hash in uk_spine.json.
Tests:
- K=1 counts equal ceil(w / 60) bitwise on random and edge weights: exact
multiples of 60, one ulp either side, tiny and large.
- K=3, 7 and 15 replicas count as their sources, including the K=7 exact
multiples the unsnapped rule overcounts. The snap's documented trade is
pinned.
- The clone-count refusals ({0, 2}, negative, missing, float, bool) give
counts.
- On K=3 and K=7 clone-first frames, the transform gives every clone its
source's copies, with weights nested within rtol 1e-12 and every copy at
most 60 / K.
- The packaged gate passes a K=3 stage receipt with w <= 60 < 3w, and fails
it when the count is ignored. Hand-built receipts pin the K=1 messages and
both scaled checks.
- The identity tool's K=1 receipt equals the one ceil(w / 60) gives, with no
new key. At K=3 it receipts green, and it reports a recompute that ignores
K.
Eight source mutations each fail a new test. The 51 touched engine-free
files pass (2,317 tests). The coverage manifest's check passes, the H2
fixture regenerates and passes its check, and H2 parity passes in the
engine slot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three reviews of 914cae4 found no must. This commit takes their should findings and the cheap nits, and corrects three claims 914cae4 made. - Identity tool at K > 1: the E8 recompute counted each family exactly, on a weight rebuilt by re-summing the family's copies. At K > 1 that sum drifts by more ulps than the count's K-ulp snap absorbs. On the commit's own nesting scenario, the tool flagged the stage's correct count of the 7,620 family in every clone at K = 11 and K = 13. At K > 1 the stored count must now lie between the counts at the folded weight times 1 - 1e-9 and 1 + 1e-9, the tolerance the tool already gives family weights. At K = 1 the check stays exact, so K = 1 receipts and problems are unchanged. The tolerance cannot tell an exact multiple's n from n + 1. The count's unit tests pin the snap, and the flags and family weights still report a moved count. copies_recomputed stays the exact recount. - Gate tolerance: test_split_gives_every_geographic_clone_its_sources_copies now runs the packaged stage gate on the transform's receipt. At K = 7 it also asserts that 7 x the heaviest copy lands above 60, so the copy check's 1e-12 tolerance stays exercised. - Texts, correcting 914cae4: - Its changelog fragment and message said every clone of a household gets its source's count. That holds for a selected clone at exactly 1/K of its source's weight. The walk still accumulates each row's own weight, so the household at which a band's walk reaches its support mass, and households tied on wealth, can be selected in some clones and not others, lowest household id first. The fragment and the split's docstrings now say so, and a new test pins the boundary case. - Its Tests line omitted that the clone-first split scenario exhausts every pool, so its selection is the same at every K. - The spine gates re-key because they hash the country spec as well as stage_health; the fragment now names both. The fragment also says the snap adds a copy at K = 7 only for some exact multiples. The module docstring, the clone-split factor's comment and the split's refusal sentence name the incidence clone stage (cgt_incidence_clone). geographic_clone_count's docstring says a refusal gives a count or the column's dtype. Not changed here: the snap rule itself and a mention of it in copy_rule, which wait for a ruling on the snap; the stage notes' "the clone stage", which changes when the stage's declarations are next regenerated; and the tool's cross-check of K against the stage receipt, which lands in (b). What moves: no value, at any K. cgt_support.py changes only in docstrings and a comment, so its implementation hash moves again, and the stages that bind it re-key as in 914cae4. sources.yaml, source_stages.json, the coverage manifest and the H2 fixture are untouched; the fixture holds stage contract hashes, not code hashes. The tool's verdicts move only at K > 1, which no spine runs yet. Tests: the identity tool's nesting receipts are green at K = 7, 11, 13 and 15; a count moved off a non-multiple is reported at K = 1, 3 and 11; at K = 1 a moved count on an exact multiple is still reported by the count check. Six synthetic mutations each fail a new test: the gate without its tolerance, the tool exact at every K, the tolerance at K = 1, the tolerance ignoring K, a tolerance of 1e-1 and a strict walk. The touched and related engine-free files pass (243 and 275 tests). The coverage manifest's check and the source_stages projection check pass, and ruff is clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
) The re-reviews of da5a350 and 9721448 found no must and no code should. This commit takes their four nits. - Bindings: the stochastic_assignment exemption of spi_support_channel now names the stage's transform, and the binding test checks that the transform names no drawing function, directly or through its module, as well as the drawing names the module imports. Before, a support-channel transform calling stage 14's employer pension draw, which spi_spine already imports, would have passed both binding tests. - The row-wise tool's --source-lineage-modulus help still described the spine's retired copy-id scheme. It now says spine inputs carry copy ids in the copy-id table's fixed fields. - Test comments on the CGT nesting scenario: the 9,000 household is the first candidate for the exact-total correction, not where it always lands. At 7 and 13 clones the shared fallback puts it on the last copy of the 7,620 family's last clone. At 13 clones an exact recount moves 12 of the 13 clones, not every one. - The identity tool's E8 docstring says copies_recomputed stays the exact recount, so at K > 1 it can exceed copies by one for each exact-multiple family while the count check passes. What moves: nothing. No runtime module changes, so no node key moves. Ruff is clean, the stochastic_assignment binding tests pass, and the re-review's mutant (the exempt transform naming the employer pension draw) now fails the binding test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
13 tasks
The land changelog said the two held rows "are measured on every evaluation", and 161c777's test was named for that. That phrase is the exclusions register's convention, and the register's entries keep it, but no build step reports a held row's estimate. As María ruled on 8 Nov, the changelog and the test now say what happens: the rows stay on the run's compiled register, and a tool that scores the un-excluded register, such as tools/evaluate_uk_incumbent_surface.py, measures them, as it does every held row. - The changelog fragment's sentence is reworded. - test_the_two_land_diagnostics_are_measured_but_not_fitted is renamed test_the_two_land_diagnostics_leave_the_solve_but_stay_on_the_compiled_register, and its docstring and the scoring comment say the same. What moves: nothing. Ruff is clean and the renamed test passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
As María ruled on 8 Nov, national_sampling's compact sampler, sample_uk_national_frame, is retired with its family decoder, uk_source_family_units. No build or tool has called the sampler since 8adc836 (#762 PR A) moved the row-wise candidate onto the spine sampler. The decoder re-derived retired id arithmetic (a clone multiplier of ten to the digit count of the largest canonical id, and max + 1 SPI and capital-gains offsets), which kept the module on the copy-id tests' inline-rule allowlist. - national_sampling.py loses the two functions, their __all__ entries and the helpers only they used: the post-sample SPI replacement quota check, the required-column list and the clone-index column alias. The module docstring now describes the spine sampler. The spine sampler, the rung tokens, the default seed and every name other modules read from the module are unchanged. - test_uk_national_sampling.py drops the ten tests of the retired functions and their compact-frame builder. The numeric-region refusal, which only those tests covered, is now tested on the spine sampler. - test_uk_copy_id_table.py: national_sampling leaves _INLINE_DIGIT_COUNT_MODULES, so the inline-rule guard now admits only rowwise_geography and copy_id_table. - experiments/901-uk-main-rebase-receipts.md cited one of the removed tests in its row 4; the row gains a dated note. - A removed changelog fragment. What moves: no value. _PopulationKernel hashes national_sampling, so uk.full.sample, uk.full.normalize, uk.full.expand, uk.full.locations, uk.full.geography_mapping, uk.full.pool and uk.full.geography_gate re-key, and with them every node downstream of uk.full.sample. No other UK kernel binds the module, so no spine node re-keys. No generated file moves: the source_stages projection and coverage manifest checks pass, and the H2 fixture holds no code hashes. Tests: test_uk_national_sampling, test_uk_copy_id_table and test_uk_runtime_exports pass (57 tests). An AST scan finds no tracked Python file that names the two functions, and the inline-rule detector flags the module at the parent commit but not here. A permissive region check fails the new spine test. Ruff is clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ll it (#931) A spine stage's key hashes its stage module and the helper modules listed for it, and source_hash does not follow imports, so a stage that called a function in an unlisted module replayed a stale output when that module changed. As María ruled on 8 Nov, this sweep binds those helpers, with a test that derives them. It only changes cache keys. - test_uk_graph gains the derived rule. From the transform graph_kernels builds for each stage, it follows every definition the code names (annotations included, through re-exports and module attributes) into the module defining it, within microcosm.build, to a fixed point, and requires the stage's hash to cover each module it meets. A check that returns and writes nothing, a function returning one column name, a record class of undefaulted fields, a constant of strings and the stage declaration need no binding; a numeric constant does. The other workspace packages stay outside the rule, and the docstring says why. A second test applies the rule to the seven dense kernels that share _PopulationKernel's hash, with the spine gate admission check listed as refusal-only, and a control test pins each form the rule follows and each kind it skips. - graph_kernels: national_frame joins the modules every stage hashes, and the helper table gains 219 bindings of 49 modules in 48 stages, with two shared groups (the vendored-fact reader with the chronicle feed and its schema epochs; frs_spine's pinned tab reader with trace). The six modules the ruling named are bound where stages 26, 42, 43 and 46 compute through them: property_localisation (26), spi_support (26, 42, 43, 46), cgt_imputation (42, 46), cgt_asset_type (46), advani_summers (43) and cgt_structure (42; stages 26 and 46 reach only its closed-world operations check). Stage 14 binds employer_pension_contributions. - The older import-level tests then ask for more: stochastic_assignment in 8 more stages (the support channel's exemption goes), the copy-id table in stages 14 and 18, 15 stages on the copy-id name-only list and four more counting stages. Their re-key tests get new controls and counts. - graph_population: _PopulationKernel hashes rowwise_geography, and the dense test also requires frame_sampling, national_frame, local_authority_input and gates. - Changelog: a new helper-bindings entry; the copy-id entry drops its sentences about the helpers that stayed unbound. What moves: node keys only. Every spine kernel hashes graph_kernels.py, so every spine node re-keys, and with it every node downstream. Every graph_population kernel re-keys (uk.full.sample, normalize, expand, locations, geography_mapping, identity, property.value, property.band, geography.local_authority, geography.brma, geography.rfdr, private_rent, pool and geography_gate), and so do the calibration and size kernels that hash the module (uk.full.dense, selected, calibrated, size_search, size_draw, size_refit and size_checkpoint_import). No value, declaration or generated file moves: uk_spine.json and the H2 fixture hold stage contract hashes, not code hashes, and the fixture check regenerates byte for byte; the source_stages projection and the coverage manifest are current. Tests: test_uk_graph (33 tests) and 15 related engine-free files pass, and ruff is clean. Mutations under scratch: a value helper newly imported into a stage (by name, by module attribute or through a re-export), a numeric constant newly read and a dropped binding (a helper, a group member, national_frame) each fail the stage test, while a helper the transform never names, a check and a constant of strings pass. Dropping rowwise_geography or frame_sampling from the dense hash, or a dense kernel newly naming an unhashed helper, fails the dense test. The H2 fixture check and the H2 parity test pass through the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erance (#931) This was to be the commit that keeps stage 42's exact-total correction off the support copies, as María ruled on 8 Nov: try the other unselected households, then the selected households left whole, never a root or copy. On synthetic frames that rule cannot always make the total exact. In the CGT nesting scenario at K = 7 and 13, every household the split leaves whole sits in the first half of NumPy's pairwise sum. That half lies in the total's binade and the second half carries exactly half an ulp, so round-half-to-even skips the target: no change to the whole households can reach it, and only a copy in the second half can. The rule as written refuses those frames, and 1 to 3 of 24 random 150-household frames at K = 1, 3, 7, 11 and 13. So the correction is unchanged here and goes back to María. This commit lands the two rulings that do not depend on it. - copy_rule names the K-ulp snap ruled on 7 Nov, identically in the code dictionary, sources.yaml and source_stages.json: "copies = ceil(geographic clone count x household weight / maximum_copy_weight), the quotient snapped to the nearest integer within clone-count ulps when the clone count exceeds one; every copy carries weight / copies". The stage notes say "the incidence clone stage (cgt_incidence_clone)" where they meant it. - The identity tool's _e8_support_split holds the stored count between the counts at the folded weight times 1 - 1e-9 and 1 + 1e-9 at K = 1 as well. At K = 1 the fold can land ulps from w: for a weight a few ulps above a multiple of 60 (about half of them in a synthetic sweep), or where the correction fell on a copy. On an exact multiple a moved count is then left to the flag and family-weight checks. The docstring, the receipt's tolerance policy and the test class docstring say so. - The split's docstring and _exact_support_weights' no longer claim the correction always keeps every copy at w / n. - test_at_one_clone_the_count_check_stays_exact becomes test_a_count_moved_on_an_exact_multiple_is_left_to_the_flags, at K = 1 and 3. A new test folds the 900 family just above 900 at K = 1: green, with the exact recount one copy higher. - The changelog fragment describes the snap clause, the K = 1 tolerance and the unchanged correction. What moves: no stage value at any K. The identity tool at K = 1: of 25 synthetic frames as built, the 2 it flagged are green now, both false positives; every tampered frame still fails, but a count moved on an exact multiple is reported by the flags and family weights, not the count check. Its K > 1 verdicts are unchanged. Re-keys: the implementation hashes of cgt_support_split and of high_value_home_support, which binds cgt_support; cgt_support_split's stage contract digest, with its texts; and every node keyed on the UK solve fingerprint, which covers sources.yaml, source_stages.json and the coverage manifest, the spine gates among them, so in a gated build every spine node from stage 14 on re-keys. Regenerated: source_stages.json (copy_rule and notes), the coverage manifest (25 source digests) and the H2 fixture (the two declaration lines in fixture.json and the stage's contract hash in uk_spine.json). Tests: 63 touched engine-free files pass (2,745 tests). The new K = 1 test fails against the parent's tool. The source_stages projection, the coverage manifest and the H2 fixture checks pass, and H2 parity passes in the engine slot. ci_test_plan verify and ruff are clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rating left it (#931) The 1b decoders review found two identity-tool gaps on the synthetic H2 oracle frame. As María ruled on 8 Nov, they land as a tool-only commit before the arms. Both reproduce with origin/main's code and with the parent's. - E5's stored-column check re-applies the regional uprating (stage 25) to the stored values as a fixed point. regional_home_values (stage 27) then rewrites main_residence_value and property_wealth, so E5 was comparing stage 27's values. On the oracle, every mismatching row is one that stage 27 rewrote: 284 of 394 scoped rows on main, 294 of 404 at the parent. Stage 27 keeps the value it replaced in main_residence_value_rank_basis and moves the wealth by the value's change. The scoping now reverses that rewrite for every stage before 27. With the reversal, the mismatches drop to zero on both frames, so the stages are right and the tool was reading the wrong stage's output. - _STACKING_STAGE_BY_FLAG had no entry for household_is_high_value_support_copy (stage 26). The was_wealth scope kept the copies (105 of 394 rows on main, 115 of 404 at the parent), and E4's FRS-only frame kept any copy of an FRS household. The flag joins the map. Dropping the layer folds the copies onto their roots by family, after the CGT folds: each copy adds its w/n back, so the root returns to its pre-split weight, to rounding. A new decoder reads the copies at the copy-id table's high_value_support field. It refuses, by count, a copy without a root, a flag or index that disagrees with the id, and a family whose size disagrees with high_value_support_copies. Dropping the copies while a CGT layer stays in scope is refused. On the oracle every high-value root is an SPI row, so the copies never reach the uprating's FRS-base mean there. The new tests use FRS roots, where they do. What moves: tool verdicts only. On the oracle, through main() and the H5 loader, E5 fails with the parent's tool and passes now. E4 reads the same frame as before. The E6, E7 and E9 receipts are identical, and E8 stops at the same pre-existing error on the fixture with both tools. On a spine whose high-value roots include FRS households, E4's scope and weights change too. Nothing hashes the tool, so no stage, value, node key or generated file moves. The source_stages projection and coverage manifest checks pass unchanged. Tests: 12 new tests on frames the real stages build (stages 25, 26 and 27, then the CGT support split and the clone): - the was_wealth and FRS-only scopes drop every copy and give each root its pre-split weight; - E5 is green on that chain, red with either half of the old scope, and red when an uprated value is moved off its fixed point before stage 26; - the tool's entry point passes on that chain and fails on the moved value; - the reversal restores stage 26's values; - the lineage refusals print counts only. 9 of the 12 fail against the parent's tool. The 3 that pass are the mutation cases, which are red with both tools. The 9 touched engine-free files pass (218 tests). Ruff is clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ts (#931) Three reviews of 2afda31, fe15413, e279b69 and bd292eb. Their one must, the stage-42 correction rule, needs a ruling and is left out with the other ruling items listed below. Bindings (fe15413): - The rule took any constant made only of strings as unable to change a value. A tuple, set or mapping of strings selects rows, columns or codes, so it is now a value, like a number. Only a single string, a function returning one column name and a record whose field types are of these kinds still stop the rule. Newly bound: tenure_constants in was_wealth, was_lisa, high_value_home_support and regional_home_values; council_tax_constants in frs_spine; rowwise_geography in lcfs_consumption; etb_vat in etb_services, which joins the counting stages through it; cgt_imputation in cgt_incidence_clone; frs_annual_income and hmrc_source_contract in hmrc_spi_income_spine; household_income in was_wealth and was_lisa. The dense population hash adds spi_support. - Where the rule stops at a name, a new test requires the name's module to be hashed by the stage or by a node upstream of it, since node keys chain on their predecessors', or the name to be listed with a reason. One stage name is listed (the geographic clone index columns, which no spine node writes, checked) and two dense ones (the LAD23 lookup's pin). - The derived check kind goes: a check is skipped only when listed with its reason, now also cgt_structure's closed-world check and spi_support's entity check. _writes_nothing, which tests each entry, also counts an augmented assignment, out=, copyto, ufunc.at, shuffle, reverse and the like as writes, and lets only a one-parameter function return an argument. A record's field types must be harmless too. - The spine root's rung sampler moves from spine_build, which no node hashes, to national_sampling (UKSampledSpineRootTransform and sample_uk_spine_root, behaviour unchanged). It declares national_sampling and frame_sampling through graph_implementation_dependencies, so the production root node hashes them. A test holds every transform spine_build installs to its stage's module, declared dependencies or a listed adapter. - The scope comment and changelog said microcosm.fit's fitters are bound by hand; three are, in two stages. They now name what stays unbound outside microcosm.build and the dense kernels the rule is not yet applied to, and the changelog names stage 18's employer pension binding. Fallback (e279b69), texts only: the tool's docstring, comment and tolerance policy, the receipts test class and the changelog say that within 1e-9 of a multiple of 60 the count check accepts n or n + 1, so a family split consistently into one copy more goes unreported at every K; a new test pins it. The changelog's what-moves names the tolerance policy and the country- and solve-fingerprint kernels that move, and puts the correction row about an ulp of the frame's total off w / n. Stage 26's docstring names the shared fallback as stage 42's does. Tool (bd292eb): the E5 reversal refuses, by count, a roster where a stage after regional_home_values declares main_residence_value or property_wealth. A test holds the stacking map to the structural stages' declared flags, with two listed exceptions. Sampler (2afda31): the seed comment drops the retired draws, and a test pins uk.full.sample's f100 branch, which experiments/901's row 4 now cites. Left for María: the stage-42 correction rule (on the synthetic nesting scenario no whole household can carry it at K = 7 and 13), an exact K = 1 count check, binding the unbound modules outside microcosm.build, binding cgt_structure in stages 26 and 46 instead of listing its check, and copy_rule's wording, which waits for the correction ruling. For the PR body: 2afda31 adds a changelog fragment of type removed; fe15413 binds employer_pension_contributions in stages 14 and 18; e279b69 also re-keys the spine, full, preflight and holdout gates and the target compilation, target selection, problem, measures, national targets, national problem and national gate kernels. What moves: node keys only. graph_kernels.py changes, so every spine node re-keys; the 12 new bindings would re-key their 9 stages by themselves. national_sampling.py and graph_population.py change, so every graph_population kernel and the calibration and size kernels that hash it re-key. The production root node hashes the rung sampler at every rung. No value, declaration or generated file moves: the source_stages projection, coverage manifest and H2 fixture checks pass, and H2 parity passes. The identity tool's receipts change only in the tolerance policy text. Tests: 34 touched and related engine-free files pass (1,115 tests), and ruff is clean on the 11 edited Python files. In-memory mutations under scratch turn the new tests red: a dropped tenure_constants or cgt_imputation binding; a stage newly reading a collection of strings, calling an np.add.at helper in an unbound module or reading an unhashed single string; an emptied or padded names table; spi_support or rowwise_geography dropped from the dense hash; the rung sampler declaring nothing or only its own module; an unlisted source adapter; the stage-26 flag dropped from the stacking map; uk.full.sample sampling at f100; and the tool before the K = 1 tolerance on the n + 1 pin. The source_stages projection and coverage manifest checks pass, the H2 fixture regenerates byte for byte and H2 parity passes in the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 16 copied exactly round(10,000 x the sample fraction) households,
picked by row position with rng.choice, so whether a household was copied
depended on every other row. Once (b) clones households at the spine's
start, the K=1 spine's copies would not be clone 0's copies at K=15. As
María ruled on 7 Nov, the stage now copies a keyed Bernoulli share whose
probability it computes itself.
- Share: the transform passes expected = count x sample fraction, so the
manifest's 10,000 is the share's numerator. create_uk_spi_support_tables
takes expected_spi_households in place of spi_household_count (None
still copies every household). It counts N, the distinct
source_household_id values entering the stage, and copies a household
when pi = min(1, expected / N) is one or the household's uniform falls
below pi. The uniform is keyed on the household's own household_id,
under the stage's seed (42) and the new salt
spi_support_channel:inclusion. The decision is spi_share_inclusion,
which takes pi explicitly; at pi = 1 it reads no uniform, since an
integer over 2**64 can round to one. Copies keep frame order. The
positional draw and the count-versus-rows refusal are gone. The
tests-only replace path keeps its own draw.
- Gate: e7_spi_synthetic_preclone is renamed e7_spi_synthetic_share. It
declares maximum_zero_weight_rows_rule {rule: binomial_share_band,
sigma: 4} in place of the integer. After the draw, the stage resolves the
rule to min(n, ceil(n x (pi + band(pi, n)))) over the n household rows
entering it, then runs the unchanged zero-weight gate.
stage_health.latent_attribute_tolerance moved into stochastic_assignment
as binomial_share_tolerance(*, target, rows, sigma). The stage_health
name stays as an alias at four sigma, with unchanged values.
_support_stage_parameters checks the draw, salt, gate name, selector,
rule and reason, and returns the declaration with its rule. Its
hmrc_source_contract caller needs no change.
- Per-clone strata (ruling 2): where household_clone_index is present,
_allocate_spi_prior_mass adds a derived clone stratum, which is never
written to the frame. The manifest declares it as
geographic_clone_stratum, and spi_spine checks it. The empty-stratum
refusal now counts strata and names none. Without the column, as on
every national spine today, the allocation is unchanged bit for bit.
- Manifest and schema: the stage has a new source sentence, draw text,
salt, gate reason and notes. In sources.schema.json,
stack_zero_weight_donors requires salt, the allocation accepts
geographic_clone_stratum, and a zero-weight declaration takes the
integer maximum or the rule (oneOf).
- Receipt: new keys expected_spi_households, inclusion_probability
(receipt_counts.floored_share), source_households (rounded_count), and
the gate's name, rule and sigma, not its resolved maximum.
spi_households stays exact, because the stage gate reads it.
- Bindings: the receipt's counting sits in spi_spine. That adds one
counting stage (frs_hmrc_spine_leaves), against three for spi_support
(cgt_support_split, cgt_incidence_clone, cgt_residential_split), which
ten stages bind. Node re-keys are the same either way: the root already
binds receipt_counts. high_value_home_support and cgt_residential_split
now bind stochastic_assignment, as they bind spi_support.
What moves: at K=1 the SPI copy set is redrawn. Its count is Binomial(N,
pi) around 10,000 x f instead of exact: about 1,000 at f010, by design
arithmetic. Per-stratum SPI weights move with it, and so do every
SPI-channel value from stage 18 on, the high-value and CGT selections that
rank SPI rows, and calibration. The claim that stage 17's seating does not
move holds on synthetic frames. On 8 frames run through the parent's stage
16 and this one, stage 17 seated the same households, carriers and bands,
with equal seating scales, mass scales and funding factors. Stage 16's
base weights were bitwise equal except for one row in one frame (relative
7e-14, the exact-total correction). The H2 fixture's 135 households at
f = 135/10,000 give pi = 1 (10,000 x 0.0135 is 135.0 in float), so its
copies are unchanged. Every node re-keys, since graph_kernels,
stochastic_assignment and spi_support changed and the root binds
stochastic_assignment; so does every kernel keyed on the spec or the
coverage manifest. Regenerated: source_stages.json, the coverage
manifest's 25 source digests, the SPI declarations in the H2 fixture
(fixture.json), and the stage's contract hash (uk_spine.json).
Tests:
- With pi held fixed, permuting rows or appending other households leaves
every household's draw unchanged. A row permutation leaves the stage's
selection unchanged.
- Over 200 seeds the count stays inside the 4-sigma band.
- pi = 1 copies every household, even with uniforms patched to 1.0.
- The resolved gate passes at its maximum and refuses one more row. The
stage gate refuses an injected copy above the bound, and an unmatched
zero-weight base row still fails.
- Malformed rules, and drift in the draw, salt, rule, sigma, name or clone
stratum, are refused.
- On clone-first frames at K=3 and 7, N is the same as at K=1, and clone
0's copies are exactly K=1's. The other clones draw their own, and K x
clone-0 weights equal K=1's to rtol 1e-12.
- Each clone keeps its own region masses. An empty clone stratum is
refused by count, and so are a non-integer, missing or negative clone
index. A one-clone index allocates bit for bit as no index.
- The band-donor fixture moves to expected_spi_households=10, and a
comment records that seed 42 keeps copies in both regions at 60 and 200
households. Its weight test now takes the median over the FRS base the
seats are drawn from: over every row, the redrawn copies moved the
median and the statistic fell just under its 0.53 bar, while the
seating itself is unchanged.
- Ten in-memory mutations each fail a new test: keying on the source id,
a row-count denominator, no pi = 1 short-circuit, no clone stratum, the
positional draw, a gate without its band, a three-sigma alias, an exact
share or source count in the receipt, and another salt.
The touched engine-free files pass, with ruff. The coverage manifest is
current. The H2 fixture regenerates and passes its check, and H2 parity
passes in the engine slot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 17 seated its donors in one walk over every candidate, and sized the seats on the eligible households counted in rows. Once (b) clones households at the spine's start, the walk would pool the clones, and the seats and the 15% household-share cap would grow with K, so clone 0 of a K=15 spine would not seat the K=1 donors. As María ruled on 7 Nov, the stage now runs today's walk inside each geographic clone, with one common mass scale. - Seating: where household_clone_index is present, the candidates are grouped by their household's clone; without it there is one group. The eligible households are counted in distinct source_household_id values, so the seats and the cap (spi_income_band_donor_seats) are the same at every K. Today's walk runs once per clone with those seats and the same person-keyed uniforms, Efraimidis-Spirakis keys and lexsort, so clone 0, whose ids are the one-clone ids, seats exactly the one-clone donors. The per-band eligibility check runs per clone and refuses by count. selected_household and carrier_person stay shared, since ids differ across clones. A non-integer, missing or negative clone index, or a gap in 0 to K - 1, is refused by count. - Weights and funding: each donor weighs (T_b / count_b) / K. Its funding stratum is its region within its clone: _funding_strata appends the clone where the column is present, and the manifest's funding_strata stays ['region']. So every clone's regions conserve their own mass. One mass scale, the minimum over every clone-region stratum, serves every clone (ruling 2), so each band keeps one donor weight, and stage_health's band-mass reconciliation and the identity tool's E8 replay hold unchanged. The exact-total correction still moves one incumbent row and never a donor, so a band's donors stay equal bit for bit at any K. With exact replicas it lands on clone 0, whose weights then nest to rtol 1e-12, not bitwise. The MassChangeRecord reason is unchanged, so uk_frame_content_identity and the coverage manifest's required_mass_change_reason do not move. - Receipt: new key clone_count, and funding_strata lists household_clone_index where the column is present (the stratum labels read region|clone). expected_donor_households, expected_donor_count and the donor counts cover every clone; donor_weight and planned_donor_weight are one clone's. At the receipts' floor of three, as the design review asked, both eligible counts go through rounded_count over distinct true sources in place of the exact ones; seating_scale reads 'full' or 'capped', since an exact scale below one gives back the eligible count; a band's carrier_mean_age reads '<3' below three carrier source households; and carrier_region_counts and the funding rows' donor_households are rounded. Besides the tests, stage_health reads seating_scale and the identity tool copies it; nothing else reads the changed fields. - Gate: the band-donor check refuses a receipt without a positive integer clone_count. It requires donor weight x K within the maximum, donors that split evenly across the clones, and at full seating donors equal to the receipt's plan (K x one clone's) and at least K x 120. The band-mass check is unchanged. At K = 1 the checks and messages are as they were, except that the seating scale is read as 'full' or 'capped' rather than a number in (0, 1]. - Texts: the draw, weight_rule, funding, seating_scale and mass_scale constants and the stage dictionary, sources.yaml (and the stage's notes) and source_stages.json. The design's draw wording named "geographic_expand", which nothing in the code defines, so the draw names the node that writes the clone index, uk.full.expand. - Bindings: the derived tests in test_uk_graph.py pass with no change. The stage already counts by lineage (it is in _COUNTING_STAGES) and binds spi_support and stochastic_assignment, and the clone column's name is a string constant. What moves: no value at K = 1. On 48 synthetic stage-16-then-17 runs (8 frames, 3 band configurations, 2 seeds; two of the frames carry a one-clone index), the seating, carriers, every household weight (hex), donor weights, mass and seating scales, funding factors, mass log and content identity after the stage equal the parent commit's. The receipt gains clone_count and takes the floor-3 forms above. spi_income_band_donors and every node after it re-key, the spine gates included, as does any kernel keyed on the spec or the coverage manifest. Regenerated: source_stages.json, the coverage manifest's 25 source digests, the stage's declaration in the H2 fixture (fixture.json) and its contract hash (uk_spine.json). Tests: - K = 1: the existing seating, carrier, weight and receipt expectations hold, with seating_scale now 'full' or 'capped'. A pinned test checks uk_frame_content_identity after stage 17 on three synthetic frames (full seating, capped seating, scaled mass) against the parent commit's values. A one-clone index seats, weighs and funds bit for bit as no index. - K = 3 and 7, with clone-first frames run through stage 16: clone 0 seats the K = 1 households, bands and carriers; every clone seats the same seats, and the other clones seat their own. K x clone-0 weights, donors included, equal K = 1's at rtol 1e-12, and so do clone 0's funding factors. - A capped seating scale is the same at K = 1 and K = 3. Each (clone, region) conserves its mass. Where one clone's region binds, its factor is 0.5, the one scale serves every clone and band, and the gate passes the scaled receipt. - Refusals: a clone without candidates, and a float, missing, negative or gapped clone index, each by count. A row permutation at K = 3 leaves the seating unchanged. - Receipt: the eligible counts are rounded, checked against an independent recount; the funding and region counts are rounded; the key sets are pinned. The mean age reads '<3' at K = 1 and where three clones' carriers share one source household. - Gate: the K = 3 receipt passes, a dropped clone_count raises, and a misread one fails. Synthetic K = 3 evidence passes and fails on weight x K, uneven splits and the scaled minimum, with the K = 1 messages pinned. - The identity tool's E8 replays a K = 3 seating with no problems and flags a reweighted clone-2 donor. - 17 mutations on scratch copies of the source each fail a test. The touched engine-free files pass, with ruff. source_stages.json and the coverage manifest are current. The H2 fixture regenerates and passes its check, and H2 parity passes in the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three reviews of 50b6e62 (stage 16's keyed share) and cd43acf (stage 17's per-clone seating) found no must. This takes their should findings and the cheap nits, all inert at K = 1. The ruling items, and the text changes the reviews placed at the next regeneration, are left out and listed below. Band donors (cd43acf): - Each band's rounded eligible count is clone 0's, in distinct source households. Pooled over the clones it was a union, which grew with K in every band below the top. Clone 0 walks what one clone walks, so the count is now the same at every K. The per-clone refusal is unchanged. - A band's carrier_mean_age is also withheld where the band seats fewer than three households per clone. At K = 1 a band's carriers rest on at most its seats in source households, so no K = 1 receipt changes. - The result docstring and the receipt comment name the fields that are rounded, and say that the band rows' donor and carrier counts and donor_count stay exact row counts, because the gate reads them. The module docstring says the seats are sized on eligible households counted in source households, and when clone 0's weights and funding factors nest. - Gate: a seating_scale in neither form is named by its type, or as missing, in the failure, the full-scale message and the details, so a replayed old receipt's exact scale does not reach the verdict. Share (50b6e62): the spine gate kernel hashes stochastic_assignment beside stage_health, since the latent-attribute checks' band moved there. Changelog fragments: stage 16's says what stays bit for bit without the clone column (the strata, given the same copies), that stage 17's funding factors and mass scale move by rounding under the redraw, and that the empty-stratum refusal reads the new draw. Stage 17's drops the clone-0 premise for the exact-total correction and states the nesting condition, the per-band counts, the mean-age rule and the gate message. Corrections to the earlier messages, which stay as they are: - 50b6e62: under the redraw, stage 17 keeps its seating, carriers, bands and seating scale, but its mass scale and funding factors are not equal. The funding strata's masses sum the redrawn copies' weights, so the factors, and the mass scale where it binds, move in their last bits, and the incumbent and donor weights with them (the reviews' synthetic probes: within about 1e-11 relative). The empty-stratum refusal reads the new draw, so a frame that built before can refuse, and one that refused can build; the reviews found both on synthetic frames. - cd43acf: the seating nests exactly at any K, but clone 0's weights and funding factors nest only while the common mass scale is the one K = 1 takes (one at both, or bound by a clone-0 region). The synthetic tests hold it at one. After stage 16 the clones are not exact replicas, and in the reviews' synthetic probes the exact-total correction often landed outside clone 0, at stage 16 too. - Both: rtol 1e-12 is the small synthetic frames' figure. A corrected row misses by a few ulps of the total, so its relative miss grows about as rows x K x 2.2e-16 (design arithmetic). What moves: no value at K = 1. On six synthetic frame and plan configurations at K = 1, the stage-17 evidence and the frame content identity equal cd43acf's; at K = 3 and 4 only carrier_mean_age differs, withheld in more bands. Every UK stage kernel hashes graph_evidence, so every spine node re-keys (54 of the spine graph's 79 kernel hashes change, the root's among them; 50b6e62 had re-keyed every node already), as do the dense population kernels and the spine gates. No declaration changes: source_stages.json and the coverage manifest are current, and the H2 fixture is untouched. Tests: - K = 3: the carriers per region are recounted in source households (SOUTH_EAST reads '<3' over three or more rows). With the rounding patched out, each band's eligible count equals clone 0's recount and is below the pooled one; the rounded counts equal K = 1's. - K = 3 and 4: the one-seat bands withhold the mean age, though their carriers come from three or more source households. - The K = 1 per-band refusal message is pinned. - The share gate resolves a sigma-2 rule below sigma 4, and refuses a NaN or infinite sigma. A sample fraction whose 10,000 x f is not whole keeps the fraction in the expected count and the share. - The gate names a float or a missing seating scale by type. The gate kernel's hash includes stochastic_assignment. - Ten mutations on scratch copies of the source each fail the intended test: the six the reviews found missed, a pooled per-band count, and one each for the mean-age rule, the gate message and the gate hash. Left out: the draw text's "(one before uk.full.expand)", which waits for María; clone_count in the stage-16 receipt, with (b); and the declaration texts the reviews put at the next stage-16 and stage-17 text regeneration, so that H2 is not regenerated for text alone (the weight rule's band donor count, the stage-17 notes' seats counted in source households, the stage-16 notes' draw on the id alone). Disclosure-control items from the reviews went to María separately. The touched engine-free files pass, with ruff. source_stages.json and the coverage manifest are current. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…st (#931) Stages 42 (cgt_support_split) and 26 (high_value_home_support) share cgt_support._exact_support_weights. It put the exact-total correction on the heaviest unselected household. Where that household could not carry it, the correction went to whichever row spi_support's shared fallback picked, which can be a root or copy of a divided household. As María ruled on 9 Nov: whole households first, capped at the 64 heaviest, then today's shared fallback, with copy_rule naming that one exception. - The helper tries the 64 heaviest households the split left whole (CGT_SUPPORT_CORRECTION_CANDIDATES), heaviest first, tied weights in row order. The first that can carry the correction takes it, so every root and copy keeps bitwise w / n. Only where none can does it call the shared fallback, unchanged. spi_support is not touched. - Whole households at stage 42 are the unselected positive households and the selected ones left at one copy. At stage 26, whose selected households all become six rows, they are the unselected positive ones. - A selected household left at one copy carries the correction only where its corrected weight leaves the selection walk and its one copy as they were (keeps_the_split). Without that check, 11 synthetic frames had a one-copy household lifted to a weight that counts two copies, and the identity tool's recompute reported a family the stage never made. Unselected households sit past their band's crossing, so the check passes them without a rerun. - Both copy_rule texts gain the same clause, built on the cap, in the code dictionaries, sources.yaml and source_stages.json: "..., except where none of the 64 heaviest households left whole can carry the exact-total correction, when the shared fallback can put it on a root or copy, off weight / copies by about one ulp of the total weight". Both stages' notes say where the correction lands. The was_wealth and was_lisa notes are untouched. - Docstrings (the split, the helper, the high-value split), the nesting scenario comments in both test files, the K = 13 recount comment and the changelog fragment's Exact total and What moves bullets follow the rule. The fragment also says what stage 26 does. What moves: - At K = 1, nothing where the heaviest unselected household is still the first candidate and carries the correction. It is the first candidate unless a selected household left at one copy weighs as much. Such a household weighs at most 60 / K, so that needs every unselected household to be as light. That household then takes the correction if it keeps the split, which moves a rounding amount between two whole households. - Where the heaviest unselected household cannot carry it, at either stage and any K, a rounding-sized correction moves from the row the shared fallback picked to a whole household. Those rows' weights move by rounding amounts, and later stages can move with them. - Re-keyed: the implementation hashes and stage contract digests of stages 26 and 42 (of the 79 spine kernels, only these two implementation hashes move; high_value_home_support binds cgt_support), and every node keyed on the UK solve fingerprint, since sources.yaml, source_stages.json and the coverage manifest moved. - Regenerated: source_stages.json (the two copy_rule and two notes lines), the coverage manifest (25 source digests) and the H2 fixture (the same four declaration lines in sources/fixture.json, and the two stages' stage_contract_sha256 in uk_spine.json). No H2 value moves: on the fixture's frame, all six calls at stages 26 and 42 find the total already exact. Synthetic evidence, counts only, against a git-archive snapshot of af667b7: - Stage 42, 751 runs: the nesting scenario at K = 1, 3, 7, 11, 13 and 15; 40 random binding frames, the four tool_probe classes and a 60-frame sweep at each of those K; and the cap frame. 419 need no correction. In 231 today's first candidate carries it, and the output is the parent's bit for bit. In the 101 where today's rule fell back: - 68 now land on a later whole household within the 64, so the correction moves off a root or copy. That is 63 selected households left at one copy and 5 unselected ones. In 11 of them the check passed over earlier one-copy households. - 33 still fall back, onto a root or copy as before, bit for bit. These include the nesting scenario at K = 7 and 13 and the cap frame. No selected household left at one copy outranked a carrying first candidate. Every total is bitwise on target. - Stage 26, 720 runs (60 random owner frames at the same six K, two stratum settings): 441 need none, 257 land on today's first candidate (the parent's output), and 22 move from a root or copy to a later unselected household. None falls back. - The cap frame: 64 heavy whole households that cannot carry the correction, then a 65th that can. It falls back, and with the cap at 65 the 65th takes it. - Worst case, timed on a synthetic vector of 15 x 78,188 rows (a K = 15 spine): sorting the whole households, 64 candidates that cannot carry it and the fallback take 0.43 s, against 0.02 s for today's one candidate. A one-copy household that could carry it costs one rerun of the selection walk (0.41 s at that size). - Identity tool: on the 68 stage-42 frames where the correction moved, E8's support-split block and clone pairs are green, as they are under today's rule; without the check, 11 were red on support_split_selection_stored. On 23 random owner chains (K = 1, 3 and 7) where stage 26's correction moved, its high-value lineage and fold, the scoped weights and E5 are green. Today's rule failed the fold's 1e-12 tolerance on 2 of them. Tests (engine-free, synthetic): today's first candidate keeps the weights bit for bit as the rule before gave them; a later whole household carries the correction where the heaviest cannot, with every divided row bitwise w / n; a selected household left at one copy can carry it; one that the correction would divide is passed over, and the identity receipt stays green (red without the check); the 65th whole household is past the cap; the nesting scenario falls back only at K = 7 and 13 and stays exact; stage 26's analogue; and the copy_rule pins in the code dictionaries, sources.yaml and source_stages.json. The 66 touched engine-free files pass (2,925 tests), with ruff, and ci_test_plan verify is ok. source_stages.json equals the typed projection, the coverage manifest is current, and through the engine slot the H2 fixture regenerates byte-identically and test_h2_uk_spine_parity passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rry one weight (#931) Since e279b69 the identity tool's support-split count check brackets the stored count between the counts at the folded family weight times 1 - 1e-9 and 1 + 1e-9 at every K. Near a multiple of 60 it accepts n or n + 1, so a family the stage split consistently into one copy more there went unreported. As María ruled on 9 Nov, the K = 1 check is now exact wherever a family's members carry one weight, and the bracket stays for the rest. Tool only. - At K = 1, a family the recompute and the stored layer agree on is held exactly where all n members are present, n is above three and every member's clone pair sum is one value c. The stored n must be the count of some w with fl(w / n) == c. Those w are one run of floats, found by stepping with np.nextafter, and the count rises with w, so the run's ends decide. - Why it is sound. The split's own exact-total correction, the incidence clone's and the residential split's each move one row. So a family of more than three members has a member no correction moved. Where at least four members show their two clone halves at equal weights, one of those is uncorrected, and its pair sum is exactly w / n. The clone halves a weight exactly. The #970 anchor scales a clone half by a factor of at most one and adds the rest to the original. The residential split divides only liable clone halves, which the anchor leaves alone. Elsewhere the anchor's and the arms' roundings can move every member of a family to one value a float off w / n. There the hold reaches one float either side of c. - The bracket stays for families of three or fewer, which may have no uncorrected member, for families whose members differ, and for every family at K > 1. - Texts. The _e8_support_split docstring, the count-check comment, the receipt's tolerance_policy, the receipts test's class docstring and the changelog fragment say which families are held exactly. They also say that where the bracket applies it opens "for a family whose source-scaled weight K x w lies within a relative 1e-9 of a multiple of 60 (at K = 1, its own weight)", nit (a) of the 8 Nov re-review, and that copies_recomputed stays the exact recount from the re-summed fold. What moves: tool verdicts only, and the tolerance_policy text; the receipt's keys are unchanged. - At K = 1 the count check reports a family of more than three members that the stage split consistently into one copy more on a multiple of 60, where its members fold to one weight. With the 8 Nov review's floor(q) + 1 stage on the nesting scenario through the clone, it reports all four exact multiples (support_split_copies_stored = 4). The parent's tool reported nothing. - At K = 3 nothing is reported there, as before. - No stage, value, declaration or node key moves, and nothing is regenerated: no node or fingerprint hashes tools/. Synthetic evidence, counts only, against a git-archive snapshot of the parent's tool: - The pair arithmetic, replayed with the stages' own operations for 10 million members per case with n of 4 or more. Untouched pairs sum to exactly w / n. Trimmed pairs, liable pairs with their arm, and both together are at most one float off. Equal halves are never off. - No false support_split_copies_stored on frames the real stages build: - the 8 Nov tool_probe classes and the nesting scenario, split then cloned, at K = 1, 3, 7, 11, 13 and 15, as built and tampered. Their verdicts are the parent's at every K; at K = 1, 986 of 1,383 families are held exactly; - 60 random K = 1 frames, half the weights multiples of 60 and a third of those one to three ulps above, split then cloned. 4,765 of 5,660 families are held exactly; - the nesting scenario and those frames through a hand redraw, the anchor and the residential split, in four mixes of gain groups. 40 to 58 percent of families are held, 161 to 920 of them to within one float. With the reach forced to zero, one correctly counted family there would be reported; - the H2 oracle frame (K = 1), which holds no support family: green under both tools. - Detection through that full chain: the floor(q) + 1 stage gives 524 to 933 count flags across the four mixes. The parent gives none. - The blind spot, by design arithmetic over every n from 4 to 9,998. Held exactly, at most the multiple itself (for 31 of the 9,995 counts) for one copy more, and never for one fewer. Within one float, the multiple and the float either side. The bracket's relative 1e-9 is 4.5 to 9.0 million floats of w on each side. Tests (engine-free, synthetic): - floor(q) + 1 is reported at K = 1, root and copies together, and still unreported at K = 3; - families one to three ulps above a multiple stay green, held exactly; - a corrected member drops its family to the bracket; - a family of three or fewer is left to the bracket; - members all one float off are held to within one float and stay green, where a reach of zero would report them; - through the anchor and the residential split the correct stage is green and three of four floor(q) + 1 families are reported. Two test comments are updated. The eight engine-free files that load the tool or the split pass (338 tests) and ruff is clean. source_stages.json, the coverage manifest and ci_test_plan verify are current. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…that call them (#931) The derived binding rule followed microcosm.build only, so a stage that computed through microcosm.fit, microcosm.frame, microcosm.graph or microcosm.calibrate replayed a stale output when that code changed. As María ruled on 9 Nov, the rule now extends to every first-party package, and the core modules every node runs through are listed with reasons instead of bound. It only changes cache keys. - test_uk_graph: _HELPER_SCOPE is "microcosm.", which covers all seven workspace packages. _CORE_MODULES lists where the walk stops: the Frame (frame.bundle, schema and weights), the graph executor (graph.executor and population) and the modules that compute node keys (graph.keys, decl and kernel). Each reason says what runs through the module and what catches an edit that changes values: H1's stored bytes for the Frame; H2, which compares the graph's spine with the legacy stage plan's, for the executor's modules; every key moving where node_key or source_hash returns something else. A new test checks that the walk from run_graph meets each, that the stages' transforms meet only the Frame's three, all three together, and that every spine stage kernel declares SeedSource.NONE. - The wider walk adds one refusal-only check (frame.rules.assert_rules_engine_country) and one listed name (frame.concepts.SOURCE_HOUSEHOLD_ID_COLUMN). The names test checks that name's reason against each reader's declared columns. The dense population test stays on microcosm.build until the other UK kernels take the rule. - graph_kernels binds what the test then requires, 71 bindings in 47 stages: - fit.qrf and fit.model (_QRF_MODULES) in was_wealth, was_lisa, nts_bus_travel, lcfs_consumption, etb_vat, etb_services, property_components, uc_reporter_redraw, spi_housing_shell, spi_employment_shell and aps_occupation; - frame.materialize in frs_hmrc_spine_leaves, spi_support_channel, hmrc_spi_income_spine and spi_income_band_donors; - frame.rules in uc_take_up and pension_credit_take_up; - calibrate.geography_constants in hmrc_cgt_gains_spine, isa_subscriptions and spi_housing_shell; - graph.randomness and graph.canonical in regional_home_values; - the engine adapter, beside national_frame, in every stage's hash, since national_frame stamps each frame with its ISA income contract and savings income projection. The 38 stages that call no engine gain it, and the engine re-key test now expects every stage to move on an adapter edit. - From the 8 Nov re-review: the rung sampler declares national_sampling alone at f100, where sample_uk_spine_root returns before frame_sampling, and both below it, and the sampled-root test pins both. The spine-build install scan follows what the source adapter and the rung sampler wrap, which neither declares to the node, and requires it to come from the stage's module; the test also checks that the observation adapter forwards a wrapped declaration. - Text: the band donor result docstring says which band fields the gate reads exactly, and that carrier_regions is a count of regions no check reads. The two nesting comments and the band-donor changelog's Funding bullet add that the 1/K split rounds when K is not a power of two. The helper-bindings changelog lists what is now bound and what is listed. What moves: node keys only. graph_kernels.py, which every spine kernel hashes, changes, so every spine node re-keys, and every node downstream with it; so do the identity and claim nodes, whose kernels live there. national_sampling.py changes, so the dense population nodes re-key, and the production root now hashes the sampler's module alone at f100 and both modules below it. The spi_band_donors.py docstring re-keys stage 17 and the SPI income stage, which the above already covers. No value, declaration or generated file moves: uk_spine.json and the H2 fixture hold stage contract hashes, not code hashes, and the fixture check regenerates byte for byte; the source_stages projection and the coverage manifest are current. Tests: test_uk_graph (38 tests) and 8 related engine-free files (440 tests) pass, and ruff is clean on the edited files. Mutations under scratch, each caught: age_tail newly calling fit.qrf or frame.materialize without the binding; was_wealth losing its QRF binding; regional_home_values losing canonical; the adapter leaving the common hash; a core entry no node runs through, one only some stages compute through, canonical listed as core, and the weights entry renamed to a module the executor never reaches; the f100 and below-f100 declarations swapped; a declaring class from another module inside the source adapter (which the parent's install test passes) or inside the rung sampler. The H2 fixture check and the H2 parity test pass through the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The derived binding rule covered the spine stages' transforms and, over
microcosm.build alone, the dense population kernels. The UK's other graph
kernels hashed hand-written module lists, and the 8 Nov reviews found some
computing through modules they did not hash, so a node could replay a
stale output after such a module changed. As María ruled on 9 Nov, the
other kernels take the same rule, keys only. Nine kernels need a ruling
first and stay held, each listed with its reason.
- test_uk_graph: a new test enumerates every kernel the UK registries
register, 99 by ref: the spine's with its gate battery; the full build's
with its gate batteries and its terminal, certification and inspection
kernels; and the national line's. Another checks they are every kernel
class that uk_runtime and the shared atomic geography define. For each
kernel the rule walks what the kernel runs, at the first-party scope: its
methods in method resolution order, without the implementation hash,
which computes the key, and without listed refusal-only methods. It
checks the modules it meets against what the hash really hashes,
recorded at every source_hash call in whichever module makes it. The
stage rule's tables apply, and a name's module must be hashed by the
kernel or listed: the LAD23 names' pin moves from the dense table. Three
kernel stops are listed, each with a reason a test checks:
- the H2 fixture's transform resolver, which names only the transform
builder the stage rule starts from;
- spi_spine's SPI support mass reason, which _minimal_frame reads only
inside its branch on hmrc_spi_income_spine, whose node hashes its reach;
- UK_GATE_REGISTRY. The bound spine reaches it only through
_manifest_digests, which returns the battery run's three digests. The
national gates evaluate through it, and their evaluators are held with
the gate batteries (below).
The dense population test folds into the new one, with its pins. A
control test pins the walk: an inherited method followed, an override
without super() not, super() reaching the base, the hash and a listed
refusal-only method left out, a listed stop reported with what names it,
and the recorder seeing a source_hash call in another module.
- Admission reads. The export description and the rotated holdout now call
graph_terminal._require_full_gate_admission in place of their inline
decode-and-refuse, with the same messages. It and the dense solve's
_admit join the refusal-only checks. Without them, the two kernels and
the dense solve would hash calibration_run, full_gates and graph_evidence
for a gate report they read only to refuse.
- Bindings, by kernel:
- the dense population kernels' shared hash (sample, normalize, expand,
locations, geography mapping, pool, gate): the engine adapter,
graph.canonical and graph.weight_update;
- uk.full.identity: household_lineage, source_lineage, frame.concepts,
graph.canonical;
- uk.full.property.value: source_lineage, frame.concepts, rowwise_dataset,
tenure_constants, graph.randomness, graph.canonical;
- uk.full.property.band: tenure_constants;
- uk.full.geography.brma and uk.full.geography.rfdr: receipt_counts;
- uk.full.private_rent: source_lineage, frame.concepts, rowwise_dataset,
national_frame, receipt_counts, graph.randomness, graph.canonical;
- uk.identity and uk.claim: national_frame;
- the four atomic geography kernels' shared hash: graph.store, through
load_source_bytes' store, and frame.materialize;
- the seven calibration and size kernels' shared hash: national_frame,
calibrate.hierarchy, calibrate.monetary_binding, graph.canonical, and
for the size search and refit target_weights, local_doctrine,
local_targets, national_doctrine and calibrate.geography_constants;
- uk.full.bound_spine: frame.materialize, the engine adapter,
content_identity, hmrc_uprating, gate_battery, gates, trace,
graph_kernels (its create-frame normalisation), graph.canonical;
- uk.full-export.prepare: gates, atomic_area_support, graph_population,
hmrc_uprating, local_authority_input, rowwise_dataset,
rowwise_geography, frame.materialize, graph.canonical; the inspection
export the same, and geography_ladder, national_frame and weight_year;
- uk.full-export.readback: artifact_files, trace, geography_ladder, the
engine adapter, frame.materialize, graph.canonical;
- uk.full-export.package: artifact_files, trace, graph.canonical;
- uk.full.certification-readiness: artifact_files, trace, gate_battery,
gates, graph_evidence, calibration_run, graph.canonical;
- uk.full.rotated-holdout: the 35 modules its problem reconstruction,
folds and rowwise solves compute through, microcosm.calibrate's solver
modules, gate_battery and stage_evidence among them;
- uk.full.national-gates: gates, calibrate's artifacts, matrix, registry,
target, hierarchy, monetary_binding and solve, graph_targets,
graph_population, national_frame, etb_services, cgt_projection,
donor_uprating, graph.canonical;
- uk.full.national-readback: artifact_files, trace, frame.materialize, the
engine adapter, geography_ladder, graph_population, hmrc_uprating,
graph.canonical.
- Held for a ruling (_HELD_KERNELS):
- uk.spine-gates, uk.full-gates.preflight and uk.full-gates. Each battery
shares one key across phases that declare different gates. Binding what
one phase's evaluators reach would re-key the assembled gate, or the
preflight and with it the dense solve, as the re-review of #1420 (E)
and #1301 and #1306 kept out.
- The target compilation, selection, measures and problem kernels, and
the national targets and problem. test_uk_target_compile_modules holds
their keys to microcosm#1497's import record, which may only shrink.
Binding what the rule finds would add 253 unhashed import edges across
their three groups and remove 79.
- test_uk_target_compile_modules: the hash inputs gain the seven modules
graph_national now imports only inside the national gates' and readback's
hashes (gates, trace, donor_uprating, etb_services,
calibrate.monetary_binding, the engine adapter, frame.materialize).
- The helper-bindings changelog takes nit (b) of the 8 Nov re-review. Its
open-kernel list becomes what this commit binds and holds.
What moves: node keys only. No value, declaration or generated file moves.
The 35 kernels bound above re-key. So do every spine stage and the create
kernel, since graph_kernels.py changed; uk.full.geography.local_authority
(graph_population.py); the full build's preflight and calibrated gates
(graph_terminal.py); and the national targets and problem
(graph_national.py). The target compilation, selection, measures and
problem kernels and the spine gates keep their hashes. Graph nodes that
re-key: every spine node and every full-graph node downstream of it; the
national line from its root, uk.full.spine_checkpoint; and the transport
graphs' atomic geography nodes and what they feed. uk_spine.json and the
H2 fixture hold stage contract hashes, and the fixture check regenerates
byte for byte; the source_stages projection and the coverage manifest are
current.
Tests: test_uk_graph (41 tests) and test_uk_target_compile_modules (52)
pass, as do 29 related engine-free files (1,041 tests) and the engine
terminal hash tests through the engine slot; ruff is clean on the edited
files. Mutations under scratch, each caught: the BRMA kernel newly reading
tenure_constants, the package inventory newly calling calibrate.solve, and
the stage kernel's run newly calling cgt_projection, all unhashed; the RFDR
hash losing receipt_counts, the calibration hash the canonical encoder,
the atomic geography hash the content store, the national readback trace,
and uk.identity national_frame; either admission read leaving the
refusal-only table, and _admit writing its argument; _minimal_frame reading
the SPI mass reason outside its stage branch; the bound spine naming the
gate registry itself; the fixture resolver stop dropped; a kernel class no
registry registers; and a held kernel's ref leaving the hold. The H2
fixture check regenerates byte for byte and the H2 parity test passes,
both through the engine slot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… commits (#931) Three reviews of c9b4862, 753e89d, c197760 and 0a9d208 found no must. This commit takes the shoulds and nits that need no ruling. The rest go to María with the review batch: stage 42's one-copy check and a copy_rule clause for it, the fallback that can undo that check, the K = 1 hold's narrowings, the engine adapter's imports, the nine held kernels with the national gates' evaluators, and regional_home_values' seed source. Rule B (stages 42 and 26): - Tests pin what no test held. Tied whole households take the correction in row order: the earlier of two at one clone, bit for bit as before; clone 0 at three clones; and the earliest of forty tied in a deterministic scramble of eighty, more ties than an unstable sort keeps in order. The walk half of the one-copy check passes over a household an ulp below its band's support mass, and the identity receipt stays green; with the rerun answering as the stage's walk did, it reports a missing family. A one-copy household that ties the heaviest unselected one from an earlier row takes the correction, the one case where K = 1 weights move although today's candidate could carry it. Stage 26 tries exactly its unselected positive households, so a selected root that could carry the correction never does. - The docstrings and the changelog say the shared fallback can also land on a whole household, a one-copy one the check refused among them, and the rerun can then divide a different set, as under the rule before. They give the order condition (a one-copy household comes first where it outweighs the heaviest unselected one or ties it from an earlier row), the case where nothing moves, the guarded worst case (64 walk reruns, about 26 s on a synthetic vector the size of a K = 15 spine, which a national frame does not reach) and the key chaining from stage 26 on. Identity tool (the K = 1 hold): - A derived test reads the hold's premises from the spine graph and the stages' code. After the split, only the incidence clone, the #970 anchor and the residential split create, expand or reweigh rows. The split, the clone and the residential split each correct their total once, through spi_support's one-row helpers. The anchor corrects none and caps its factor at one, and every later stage passes its input's household weights on. - Tests pin the equal-halves threshold (three members with equal halves do not vouch for a family) and move the root, as well as a copy, in the folded-above-a-multiple test. - The changelog's What moves, the receipt's tolerance_policy and the receipts test docstring name the one-float tier's blind spot. Where fewer than four members show their pair untouched, a family split consistently into one copy more on the multiple can go unreported: for 6,393 of the 9,995 counts n from 4 to 9,998, against 31 where the family is held exactly. Bindings: - The create-frame normalisation and the owned-cell cast move unchanged from graph_kernels into a new graph_frames module. The bound spine hashes it in place of graph_kernels, and the spine's create, claim and stage kernels hash it too, so no national-line kernel hashes graph_kernels now. - The size search and refit hash their five extra modules in their own implementation hashes, through a shared _calibration_hash, which the kernel rule leaves out as it leaves out every implementation hash. The dense solve and the other calibration kernels no longer carry them, so an edit to local_doctrine or target_weights re-keys no national-line node; the national targets and problem hash the other three. - test_uk_graph: the national gates' hold at UK_GATE_REGISTRY moves from the stop table into _HELD_KERNEL_STOPS. A test checks that each held kernel and held stop still guards a module its hash leaves out. Another holds the executor's and key modules' definitions that kernels outside the hold name to a listed table, each with a reason: dtype_for_token, ArtifactType, ArtifactOutput, StructuralDelta and KernelRole. The Frame's reason says H2 cannot catch a Frame edit, and a test pins Weights.total as NumPy's sum. The fixture resolver's stop names the injected engine as its exception. A comment by the SeedSource.NONE assertion records that regional_home_values draws keyed uniforms, held for a ruling. - The helper-bindings changelog: the intro, "or be a listed adapter", the claim kernel's graph_frames, the NZ transport graph's geography nodes, that held kernels can replay stale outputs, and the national line's re-keys. What moves: node keys only. No value, declaration or generated file moves; cgt_support.py changes only in comments and docstrings, and the tool only in its receipt's tolerance_policy text. Of the 99 UK kernels, 64 implementation hashes move: every spine stage kernel, the create, identity and claim kernels, the bound spine, and the seven calibration and size kernels. Every spine node and every full-build node re-key, and so does the national line from its bound spine root. Neither the country nor the solve fingerprint moves. source_stages.json equals the typed projection, and the coverage manifest is current. The H2 fixture holds stage contract hashes and no declaration changed, so it is not regenerated. Tests (engine-free, synthetic): the four touched files pass (242 tests). So do 21 related files (572 tests): those that load the tool, scan uk_runtime's modules, hold the import records, or run the create, claim, bound-spine and calibration kernels. Ruff is clean on the edited files, and ci_test_plan verify is ok. Mutations under scratch, each caught: - reversed ties, and an unstable sort (by the forty-tie frame); - the walk rerun stubbed to agree; - stage 26 trying selected roots; - unselected households tried before one-copy ones (only the one-copy test fails); - the hold where members differ (by the moved root), or on one or three equal halves; - for the premises test: four corrected rows, a second correction in the clone, one in the anchor, a rescaled pass-through, and the anchor's cap at 1.5; - Weights.total as fsum; - a kernel naming graph.keys' frame_key; - the size kernels dropping target_weights; - the dense solve calling local_doctrine; - the bound spine dropping graph_frames; - a hold that guards nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ations (#931) María ruled on 9 Nov that stage 42's one-copy check (keeps_the_split) stays, as a reading of the whole-households-first rule, and that copy_rule names it. She also accepted that the shared fallback does not make the check, with the stage notes saying so. This commit changes declarations, text and tests only, not behaviour. - copy_rule gains one clause after the shared exception, built on the cap as that exception is (CGT_SUPPORT_ONE_COPY_CHECK). It reads the same in the code dictionary, sources.yaml and source_stages.json: "...; of those 64, a selected household left at one copy can carry the correction only where its corrected weight keeps its one copy and the walk's selection, a check the fallback does not make". The closing words keep the clause true where the fallback lands on a household the check passed over. Stage 26's copy_rule is unchanged, since it makes no such check. - Stage 42's notes gain one sentence: the shared fallback does not make the check, so where none of the 64 can carry the correction, it can put it on a one-copy household the check passed over. The identity tool's selection recompute can then find a family the stage never made, or miss one it made, as under the rule before. No other stage's notes change, and the was_wealth and was_lisa notes are untouched. - The changelog fragment's Declarations bullet quotes the new copy_rule. Its Exact total bullet names the check in the same words and gives both synthetic cases. - Tests: the copy_rule pin and the packaged-declarations test end on the new clause. A new test pins the accepted case on two synthetic K = 1 frames where the one household left whole is selected at one copy and is the heaviest row. In the first it sits an ulp below its band's support mass of 40 (2 x 1.25 x 16), so its corrected weight would stop the walk at it. In the second it weighs 60, so its corrected weight counts two copies. In both the check passes it over, and the fallback gives it the weight the check refused, bit for bit as the rule before did. A fallback that made the check fails the test. What moves: - No value. On the two frames the stage's output equals the parent's and af667b7's bit for bit. The identity tool reports support_split_selection_stored {missing: 1} and {extra: 1}, as it did at af667b7. No H2 value moves. - Stage 42's stage contract digest, with its copy_rule and notes. - The implementation hashes of stages 42 and 26. cgt_support.py's bytes moved, and high_value_home_support binds it. Node keys chain, so every spine node from stage 26 on re-keys (43 of the 79 hermetic spine nodes; the 36 before it keep their keys), and so does every node downstream of the spine (75 of the 111 nodes of the full graph with its gates). - Every node keyed on the UK country spec's or solve's fingerprint, since sources.yaml, source_stages.json and the coverage manifest moved. Of the 99 kernel implementation hashes, 13 move: the two stages, plus the spine, full, preflight and national gates, the rotated holdout, the measures, the problem, the target compilation and selection, and the national targets and problem. With the spine gates attached, every spine node from the assembled gate on re-keys (64 of 81). - Regenerated: source_stages.json (stage 42's copy_rule and notes), the coverage manifest (25 source digests) and the H2 fixture (the same two lines in sources/fixture.json, and stage 42's stage_contract_sha256 in uk_spine.json). Tests: all synthetic and engine-free except H2. test_uk_cgt_support and 15 related files that read the declarations, the projection, the coverage manifest, the fingerprints or the stage's code pass (950 tests). Ruff check and format are clean on the two edited Python files. source_stages.json equals the typed projection, and the coverage manifest is current. Through the engine slot, the H2 fixture regenerates, its --check regenerates it byte-identically, and test_h2_uk_spine_parity passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s, at any clone count (#931) María's 9 Nov evening rulings on the identity tool's count hold: keep its two narrowings, hold a family on the members whose clone halves are still equal, extend the hold to K > 1, and hold the family test to the same tolerance as the count. Tool only; no stage code, value, declaration or node key moves. - Equal halves (ruling 3). Where at least four members of a family of more than three show their two clone halves at equal weights and share one pair sum c, the stored count n is held exactly on c, whatever the other members carry (trimmed or liable members, a corrected one). At most three members carry a corrected weight, so one of the four is uncorrected, and equal halves leave its pair sum at exactly w / n. Before, a family was held only where every member carried c. Both accepted narrowings stay: families of three or fewer keep the bracket, and where every member shares c but fewer than four show equal halves, the hold reaches one float either side of c. - K > 1 (ruling 4). The stage counts n on each clone's own weight and gives every member fl(w / n), and the snapped count still rises with w, so the hold applies at any K, with the K-aware count at the ends of the run of weights. The one-float tier reaches two floats where c sits within two floats of a power of two. c lies in (45 / K, 60 / K], which holds 16 at K = 3, 8 at K = 6 and 7, and 4 at K = 12 to 15, and below a power of two the floats are half as wide. The hold's docstring gives the bound: a trimmed pair's sum lies within three quarters of the float spacing at c before its last rounding, a liable pair's within half. - The family test (ruling 3). Whether a household is a family is no longer the exact recount of the re-summed fold. A household the rule selects is possibly a family where its K-aware count at the folded weight times 1 + 1e-9 exceeds one, and possibly whole where its count times 1 - 1e-9 is one. Its stored state is accepted in between, and the hold's counts decide where the hold applies. The keys and their missing and extra counts are unchanged. The hold is now asked about every stored family, so it takes only a positive normal pair sum: a family of zero-weight members falls to the bracket and is reported missing, where the run of weights would step below zero and the tool would raise. - The premises test (the re-review's first nit) counts the exact-total corrections in every function, method and nested function of the three weighing stages, and matches attribute calls as well as bare names. - Texts: the _e8_support_split docstring, the hold's docstrings and constants, the count-check comment, the receipt's tolerance_policy, the receipts test's class docstring and the changelog fragment (Receipt and gate, What moves, and one Exact total sentence this commit makes stale). What moves: tool verdicts and the tolerance_policy text only. The receipt's keys are unchanged, and copies_recomputed is still the exact recount. No node or fingerprint hashes tools/, and nothing is regenerated. - At K > 1 the count check now reports a family the stage split consistently into one copy more on a multiple. With the floor(q) + 1 stage on the nesting scenario it reports all four families in every clone at K = 2, 3, 4, 6, 7, 11, 13 and 15. The parent tool reported none. - At K = 1 it also reports such a family where four members show equal halves and others differ. A family with one corrected member now reports. - The family test no longer reports a family missing, or extra, where a household's fold crosses 60 / K from its stored state. That removes the 9 Nov probe's false missing family (a two-copy family a float above 60 whose fold lands on 60) and one false extra at K = 2. It also no longer reports a household split into two copies at 60 / K (the floor(q) + 1 stage's two-copy households), nor stage 42's one-copy household that the shared fallback lifts across 60. The rule rerun still counts the latter in copies_recomputed. Ruling 2's walk-moving case is still reported, on all seven of its frames. Synthetic evidence, counts only, against a git-archive snapshot of 8ea25b9's tool: - No false support_split_copies_stored or support_split_selection_stored on frames the real stages build, at K = 1, 2, 3, 4, 6, 7, 11, 13 and 15: - the 8 Nov tool_probe classes and the nesting scenario, split then cloned, where tampered counts get the parent's verdicts; - 30 random frames with source weights on and one to three ulps above multiples of 60, and 20 with clone weights on and above multiples of 60 / K; - the nesting scenario and 12 or 24 random frames through a hand redraw, the anchor and the residential split, in two to four gain mixes; - frames whose c sits at or within three floats of 16, 8 or 4 at K = 3, 6, 7, 12, 13 and 15, as built, with every member moved one float off c (two below where c is the power) and through the full chain. - Families held: 83 to 84 percent exactly, split then cloned. Through the full chain, 64 to 74 percent exactly and 7 to 13 percent within the reach; at K = 15, 927 of 33,600 within two floats. - Detection: floor(q) + 1 gives 2,350 to 20,628 count flags split then cloned at K > 1, and 7,934 to 21,217 through the full chain. The parent gives none. At K = 1 the full chain gives 3,258 against the parent's 1,322. Every floor(q) + 1 frame is flagged. - The false missing family's frame is green, where the parent reports {missing: 1}. - The pair arithmetic replayed at 11 clone counts from 1 to 15, 2.4 to 4.8 million members per K and case, 0.7 to 1.3 million of them within two floats of a power of two where (45 / K, 60 / K] holds one: never more than one float off c, and equal halves never moved. Across 267,035 families near a power of two, a two-float reach never accepts a count one float rejects. - The blind spot, by design arithmetic over every n from 4 to 9,998. At K = 1 it is as before. At K = 2 to 15 the snapped count's step sits 3 to 17 floats above the multiple of 60 / K. Held exactly, at most the float either side of the step, for up to 2,975 of the 9,995 counts. In the one-float tier, up to three floats from it. floor(q) + 1 on a clone of a source multiple is reported for every count where it changes the count. - Mutations: the premises test now catches a second correction written as spi_support._importance_weights_with_exact_total(...) in clone_cgt_incidence, a bare-name one in UKCGTIncidenceCloneStageTransform.__call__, and a wrapper call in a nested function. The test before let the first two through. Eight of nine tool mutations fail the receipts tests. The ninth, not letting the held counts decide the family test, changes no verdict: a held family's fold is n times its shared pair sum. Tests (engine-free, synthetic): floor(q) + 1 is reported at K = 3, 7, 13 and 15; a family with a corrected member is held on its equal halves; four equal halves vouch for a family whose others differ, three do not; families of three or fewer stay on the bracket at K = 1 and 3; through the anchor and the residential split the correct stage stays green at K = 1 and 3 and floor(q) + 1 reports 3 and 12; members near a power of two are held to two floats; the two-copy family that folds onto 60 is green; a family dropped or added away from a multiple is reported at K = 1 and 3; a zero-weight family is reported, not refused; the stage-42 one-copy check case is now green, with its stage weight checked instead. The seven engine-free files that load the tool pass (257 tests), and ruff is clean on both Python files. source_stages.json, the coverage manifest and ci_test_plan verify are current. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#931) María's rulings of 9 Nov (5, 7, 8 and 9) asked for four things: - bind the first-party modules the injected engine computes through into the engine stages, through a second walk root at the engine class; - have the walk see the names microcosm.graph exports lazily, with the create kernel's H2-only use of load_source listed as a stop; - keep SeedSource.NONE for every spine stage kernel; - keep the graph_frames move. This commit does all four and the re-review's changelog nit. It changes keys only. - The engine root. test_uk_graph roots a second walk at PolicyEngineUKEngine, the class that graph_kernels' hermetic H2 builder and the spine build both construct (checked). - The walk follows the engine methods each engine stage's code names, by attribute on any receiver or through getattr. It also follows the constructor and every method they call on self, at the first-party scope with the stage rule's tables, and _stage_walks adds it to the stage's own walk. - The 16 engine stages call materialize and variable_metadata, and private_school_attendance also calls parameters and simulation_batches through getattr. - Beyond the three hand-bound engine modules, the walk reaches one module: microcosm.frame.materialize, through which the adapter builds each simulation's dataset. graph_kernels gains _ENGINE_COMPUTE_MODULES = (frame_materialize,) beside _ENGINE_MODULES, and every engine stage binds both (_ENGINE_STAGE_MODULES). - A new test derives the group exactly. The engine-modules test keeps _ENGINE_MODULES exact. Another stage may bind a module of the group only where its own transform computes through it, as the SPI spine's three stages and the band donors do. - The engine reaches the concept vocabulary (policyengine_uk_concepts, concept_mapping and concepts) and frame.rules only through methods no engine stage calls (concept_mapping and export_contract). So they are not bound, and the engine stages do not re-key on edits to the concept vocabulary, unlike the consequence stated with the ruling. - The fixture resolver stop's reason no longer names the engine as held. SOURCE_HOUSEHOLD_ID_COLUMN's listing is unchanged. - Lazy exports. - _resolve now follows a name that a module exports through its module __getattr__ to the submodule that defines it. It reads the table that __getattr__ reads (_LAZY_EXPORT_TABLES: microcosm.graph's _RUNTIME_EXPORTS). - The walk now reads type aliases as definitions. Three of those names are type aliases. - A test checks that every lazy name resolves to the object the package returns. It also checks that every first-party module with a module __getattr__ is listed; us_runtime.sources, which only raises, is listed as exporting nothing. - Resolving the lazy names changes no stage's walk. In the UK kernels, two things surface: - six definitions of microcosm.graph.kernel (the capabilities record, three of its enums, and the context and result records), now listed as core names with reasons; - the create kernel's load_source, now a listed stop. - The stop's reason: the create kernel calls load_source only where no root transform is bound, the hermetic registry's data-only path over the H2 fixture's captured root tables. The spine build installs the root's transform, and H2 parity binds it too (checked). - One test checks that the call sits only in the "self.transform is None" branch. Another checks that no spine kernel and no national-line kernel hashes microcosm.graph.codecs. - The blind-spot paragraph is gone from the kernel test's docstring. - SeedSource. The comment by the NONE assertion records the ruling and why NONE holds: regional_home_values' stream is a stage parameter, which its contract hash covers, and it reads no generator the executor seeds. - The changelog fragment. - The What moves sentence on local_doctrine and target_weights is reworded so that each clause is true. It was checked against the recorded source_hash calls and the kernels' walks. - The fragment now describes the engine root, the lazy exports, the load_source stop and the SeedSource ruling. What moves: - Node keys only. No value, declaration or generated file moves. - graph_kernels.py's bytes changed. So the implementation hashes of the create, identity and claim kernels and of all 53 stage and expand kernels move: 56 of the 99 UK kernels. The 16 engine stages now also hash frame.materialize. - Every node re-keys, by its own hash or by chaining: - the hermetic spine, 79 of 79; - the spine with its gates, 81 of 81; - the full graph with its gates, 111 of 111. - The other 43 kernels keep their hashes. So the national line, whose root is the bound spine checkpoint, keeps its keys, as do the transport graphs' atomic geography nodes. The country spec's and the solve's fingerprints do not move. - uk_spine.json holds only the 54 stages' stage_contract_sha256 values, and the H2 fixture holds no code hash. The fixture regenerates byte-identically, and source_stages.json and the coverage manifest are current. Tests: all engine-free, synthetic or code facts, except H2. - test_uk_graph (50 tests) and 15 related files pass (445 tests). - Ten mutations on a scratch copy each fail the tests that should catch them: - materialize newly reads the concept mapping; - frame.materialize leaves the group; - a stage calls the engine's concept_mapping through getattr; - the walk ignores lazy exports; - load_source moves out of the unbound branch; - a kernel newly reads a lazy export it does not hash; - a stage newly reads a lazy export it does not hash; - a non-engine stage binds frame.materialize without computing through it; - a spine stage hashes the codecs; - a module __getattr__ is not listed. - Ruff check and format are clean on the two edited Python files. - Through the engine slot, the H2 fixture check regenerates byte-identically and test_h2_uk_spine_parity passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María ruled on 9 Nov (ruling 6) that each phase of a gate battery keys on what its own gates compute through, with what only one phase computes through carried on that phase's params, and that the single-phase national gates bind the gate registry's whole reach. This commit does both. It moves keys only, and the full gates now refuse a stale preflight code as they refused a stale terminal code. - The model. test_uk_graph derives what each battery phase computes through (_battery_phase_walk). It follows the kernel's code, taking the branch of each `if phase == ...` the phase takes, and the binding of each gate the manifest declares for the phase, read from UK_GATE_REGISTRY's literal mapping. In stage_health it follows only the checks the phase declares, by each entry's `check`. In the spine gates' artifact table (uk_spine_gate_artifacts) it follows only the entries the phase's gates read, so the CGT asset type domain is the transferred phase's alone. The kernel rule also leaves out a method only the implementation hash calls, the full gates' _country_resources. Another test checks each part of the model against the code. - The check. A battery's shared hash holds what every phase computes through and nothing only some phases do. Each phase's own code, carried as `<phase>_code`, holds exactly the rest. The kernel rule's test holds each phase's key to cover what it computes through. - The spine gates. The shared hash has 11 modules. It gains gates, stage_evidence, graph_kernels, national_frame, receipt_counts, council_tax_constants and calibrate.geography_constants, and stochastic_assignment moves to the transferred phase. The assembled node carries a new assembled_code (frs_relationships). energy_pricing_code becomes transferred_code, 17 modules. As before, the spine gate kernel does not check them. - The full gates. The shared hash has 26 modules. local_rowwise, weighted_integrity and diagnostics move to terminal_code. high_value_homes and calibrate.geography_constants move into the shared hash, since the preflight's target surface and ledger parity compute through them. The preflight node carries a new preflight_code (microcosm.build.ledger_targets), and terminal_code hashes 37 modules. The kernel refuses a stale or missing code in either phase. - The national gates. Their hash adds the 50 modules battery_bindings.UK_GATE_REGISTRY_MODULES lists, which a test holds to the registry's derived reach. Their held stop goes, and _HELD_KERNEL_STOPS with it. The three batteries leave _HELD_KERNELS, which keeps the six target kernels. - Listed with reasons: rowwise_dataset's clone index column name, which the calibrated and national gates' support-family fold reads, and source_hash, a core name the full gates' code check reaches. - Tests and text. The spine evidence and terminal tests follow the new params. The stale code test covers both full phases and a missing code, and a new test checks each full gate node's code. The control test pins the key-only method. The changelog fragment moves the batteries and the national gates from held to bound. What moves: node keys only. No value, declaration or generated file moves. - 74 of the 99 kernel hashes move. graph_evidence.py changed, and 65 kernels hash it: every spine stage and expand kernel, uk.create, the seven dense population kernels, the spine gates, the preflight and calibrated gates and the certification. graph_terminal.py also moves the export, package, inspection and holdout kernels. graph_national.py moves the national targets, problem, gates and readback. - So every node of the gated spine re-keys (81 of 81), and every node of the country adapter's in-graph build (113 of 113). On a bound spine checkpoint, every node after the checkpoint re-keys, the preflight and the dense solve among them. On the national line 5 of 6 nodes re-key; the bound checkpoint keeps its key. The transport graphs keep theirs. - Later edits: one to a module only the transferred phase computes through re-keys the transferred gate, not the assembled gate or the stages that wait on it. One to a module only the terminal phase computes through re-keys the calibrated gates and what follows, not the preflight or the solve. On a bound spine, edits to stage_evidence, parity_reference, release_identity and release_input_coverage now re-key the preflight, and so re-solve: the preflight's release input coverage gate computes through them, and its key used to miss them. - No fixture or generated file holds a gate node. uk_spine.json, the H2 fixture, source_stages.json and the coverage manifest do not move. Tests: test_uk_graph, test_uk_graph_evidence and test_uk_graph_terminal pass (98 tests). 25 related engine-free files pass (1,276 tests) but for one test that also fails at the parent on this machine (test_uk_cgt_projection asserts the UK engine is not installed). Through the engine slot, the engine terminal and full-graph admission tests pass, the H2 fixture regenerates byte-identically under --check, and test_h2_uk_spine_parity passes. 17 mutations under scratch are caught: a phase newly computing through a module its key misses (a transferred check or the terminal branch reading an unhashed module, an assembled check reading a transferred-only one, the preflight branch reading a terminal-only one), a phase-only module moved into a shared hash, a shared module moved into one phase's code, a terminal-only module put in the preflight's code, the national gates dropping a registry module or the registry's reach, an unmodelled stage-health branch, another evaluator reading a domain artifact, the run calling the key-only method, the run dropping its code check, and a spine node dropping its code. Ruff is clean on the seven edited Python files, and source_stages.json and the coverage manifest are current. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
) María ruled on 9 Nov (ruling 6) that the derived binding rule replaces microcosm#1497's per-kernel import record for the UK target and problem kernels, that 0a9d208's edit to that test's hash inputs stays, and that the false reason in transport/compile_modules.py is corrected with it. This commit does all three. It moves keys only. - The six kernels leave _HELD_KERNELS. The table, its reason and the hold's liveness test go, so no kernel stays held. test_uk_graph's kernel rule now walks the full build's target compilation, selection, measures and problem, and the national targets and problem, as it walks every other kernel. - Bindings. What all four full kernels compute through goes in the hash they share. What only one computes through goes in its own implementation hash, as for the size search and refit. - The shared hash gains the canonical encoder. - The compilation adds chronicle_epoch, whose schema versions and fact-key epochs ledger_artifact checks and records, and frs_relationships, whose ONS household type domain ledger_targets checks the bindings against. - The measures add the 15 modules that resolving the measures and compiling their national problem compute through: graph_population, national_frame, atomic_area_support, frame.materialize, the engine adapter, dfe_spring_term, frs_annual_income, hmrc_income, income_tax_rate_classes, lcfs_consumption, uc_target_measurements, chronicle_epoch, national_calibration, local_rowwise and calibrate.monetary_binding. - The problem adds graph_population, national_frame, local_geography and calibrate.monetary_binding. - The national kernels' shared hash gains chronicle_epoch, graph_targets and the canonical encoder. The national targets add frs_relationships. The national problem adds the measures' modules except chronicle_epoch and national_calibration, which its shared hash binds, plus diagnostics and gate_battery. - The measures and the national problem also bind frs_spine for one name, the annuity and trust column. The resolver reads zeros where a frame lacks it, so on a bound spine a renamed constant would change the problem without re-keying it. - The clone index columns' name stays listed. Its reason now covers the dense measures, whose population is the expand's and which read it only to split into engine blocks, refusing a frame without it. It also covers the national problem, which resolves in one block and never reads it. The kernel rule's test checks these facts. - No kernel loses a module, though some hash modules the walk does not reach: the selection hashes 51 and computes through 6. - test_uk_target_compile_modules. UNCLASSIFIED_IMPORTS, the kernel groups and the record check go, and the docstring says the derived rule now holds these kernels. The closure check of the shared list, the hash tests and HASH_INPUTS stay, with 0a9d208's seven entries. Three of them (calibrate.monetary_binding, the engine adapter and frame.materialize) are now also imported in the national problem's hash, and their reason says so. A new test checks what the derived rule does not: - each hash input is still imported where listed; - a kernel hashes one only where another module it hashes imports it. So no kernel hashes the compile list or the solve fingerprint's module. - target_compile_modules.py's docstring no longer names the retired record. The file is a hash input, so the edit re-keys nothing. - transport/compile_modules.py. _GRAPH_RUNTIME said no kernel in the repository hashes a microcosm.graph module, which was false. It now says: - what the listed modules use those modules for; - that the compile kernels hash none of them; - that other kernels hash the canonical encoder, each UK graph kernel whose code computes through it among them. No kernel hashes compile_modules.py, so the edit re-keys nothing. - The changelog fragment moves the six kernels from held to bound. What moves: node keys only. No value, declaration or generated file moves. - 11 of the 99 UK kernel hashes move: the six, and five that hash graph_targets.py or graph_national.py (the preflight and calibrated gates, the rotated holdout, the national gates and the readback). - Full build with its gates, in the country adapter's in-graph composition: 14 of 113 nodes re-key. They are the target compilation, selection, measures and problem, the pool checkpoint, the preflight, the dense solve, the size search, draw and refit, the selected and calibrated populations, the calibrated gates and the holdout. Every node after them re-keys by chaining. The spine and the dense population and geography nodes keep their keys. - National line: 5 of 6 nodes re-key (the targets, the problem, the dense solve, the calibrated population and the national gates). The bound spine checkpoint keeps its key. - The gated spine (81 nodes) and the transport graphs keep their keys. source_stages.json, the coverage manifest, uk_spine.json and the H2 fixture do not move. Tests: all synthetic or code facts. - test_uk_graph and test_uk_target_compile_modules pass (104 tests), as do 14 related engine-free files (365 tests). Through the engine slot, engine/uk/test_uk_full_target_graph passes (8 tests). - 19 mutations on a scratch copy are caught: - each of the six kernels, or a helper the measures compute through, newly computing through an unhashed module, or reading a lazy export; - a module leaving each kind of hash: the shared, the compilation's, the measures', the problem's, the national shared and the national problem's, frs_spine among them; - a kernel hashing a hash input; - the national problem resolving in blocks, and the build moving the measures' population. A twentieth, naming the module by a computed string, passes, as the rule documents; its static form is caught. - Ruff is clean on the six edited Python files. source_stages.json and the coverage manifest are current. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three reviews of 8ea25b9, e46f54a, aaa371a, 021182d and 08b437a found no must. This commit takes the shoulds and the nits that need no ruling. Five go to María: the engine walk's root (the methods the stages call or the whole class), the reach near a power of two, where the phase-code tables live and a stale-code check in the spine gates, the two bindings the target rule forces, and the transport compile kernels' canonical encoder. The stage-42 notes' narrowing waits for (b)'s next stage-16/17/42 regeneration, so H2 regenerates once. Gate batteries: - The phase-code texts said an edit only one phase computes through re-keys that phase alone. A phase's own key leaves out what only the other phase computes through, but key chaining still re-keys the other phase wherever a node upstream hashes the module. graph_evidence's and graph_terminal's docstrings and comments, the battery table's comment, the battery test's docstring and the changelog now say so and name where: stages before the assembled gate hash stochastic_assignment and take_up_contract, and of the 38 modules only the terminal phase computes through, nine never re-solve, on a bound spine or in the in-graph build, high_value_home_gates among them. - 021182d put calibration_run, the national calibration runner, on the preflight's key, so in the in-graph build an edit to it re-solved, against microcosm#1492's design. The runner's names the preflight computes through, the spine and local gate scopes and the scoped gate manifest, move unchanged into a new gate_scopes module, which calibration_run re-exports. The full gates' shared hash binds gate_scopes in its place, and the bound spine, the national gates and the certification hash it too. The calibrated gates' terminal_code binds calibration_run, whose admin totals and CGT projection artifact their gate context is built with. A new test holds that an edit to the runner leaves both full gate kernels' hashes and the preflight's own code, and moves the terminal code. - The spine gates' enum gates resolve a domain no artifact supplies through the rules engine they are handed (PolicyEngineUKEngine's enum_domain). The battery walk now follows the engine each battery's kernel is built with wherever a phase's gates read it (_BATTERY_ENGINES: the spine build's rules engine, and the driver's coverage engine for the full gates), so the spine gates' shared hash binds the engine adapter. A test checks that each engine is the one its build hands the kernel, and the artifact that hands it on. The engine root: _engine_reach repeats the engine walk over the engine methods its own reach names outside the engine's classes until the set stops growing, so a helper the engine hands itself to is followed into the methods it calls on that argument. It adds no module today. The control test gains that case. Identity tool (tests and text only): - The premises test builds one wrapper set from all three stage modules and counts references to a helper or wrapper, not only calls, so a second correction through another module's wrapper, an alias or a partial fails it. - A test pins that the exact tier takes c from the equal-halves members. On a synthetic family a float above 900, one member's pair sum a float below c leaves the tool green, which the least pair sum over every member would not. - The tolerance policy, the docstrings and the changelog say the exact tier holds where every member that shows equal halves shares c, which is what the code checks; a member the split's own fallback corrected keeps its family on the bracket. The changelog says c can sit a few floats above 60 / K (at most 16 for K up to 15, by design arithmetic), and its Declarations bullet says what the tool reports of the selection recompute's finds and misses. Texts: fixture_stage_plan_inputs' docstring says H2 parity binds the graph's CREATE kernel to the root transform it adds; the gate registry stop's reason says the rule follows the gate kernels past it; the codecs test reads the national line's kernels from its graph, the calibrated kernel among them; test_uk_target_compile_modules' docstring states the narrower hash-input rule; docs/uk-full-build-graph.md names frame.materialize among the engine stages' bindings. Corrections to the earlier messages, for the PR body: in 021182d's later-edits bullet, the chaining above, and in the in-graph build calibration_run (until this commit), parity_reference and release_identity joined the solve's key; in aaa371a, the 16 engine stages call materialize and 6 of them variable_metadata, and on a bound spine 24 of the local line's 33 nodes re-keyed, from uk.full.expand.owned on; in e46f54a, the exact tier's condition above and the snap's range. What moves: node keys only. No value, declaration or generated file moves, and neither the solve nor the country fingerprint does. - 77 of the 99 UK kernel hashes move: every spine stage and expand kernel, the create, identity and claim kernels and the seven dense population kernels (graph_kernels' docstring and graph_evidence); the spine gates (with the engine adapter); the preflight and calibrated gates (gate_scopes for calibration_run); the export, package, inspection, readback and rotated holdout kernels; the bound spine and the certification; and the national targets, problem, gates and readback (graph_national). The calibrated gates' terminal_code moves too. The calibration and size kernels, the full target kernels, the geography, property and private rent kernels and the four atomic geography kernels keep theirs. - So every node of every UK graph re-keys: the hermetic and gated spine (79 and 81), the in-graph build (113), the local line on a bound spine (33) and the national line (6). The transport kernels keep their hashes. - source_stages.json and the coverage manifest are current. uk_spine.json and the H2 fixture hold stage contract hashes and no declaration changed, so neither regenerates. Tests (engine-free, synthetic or code facts): 26 touched and related files pass (1,133 tests). Ruff is clean on the 13 edited Python files, and ci_test_plan verify is ok. The unmutated tree passes, and 19 mutations under scratch are each caught: - a second correction in the incidence clone by attribute, by bare name in its transform, through cgt_support's wrapper, an alias, a nested function, a lambda or a partial; - an adapter helper calling concept_mapping on the engine it is handed, and a one-pass engine reach; - the exact tier taking the least pair sum over every member; - the spine gates dropping the adapter, the model ignoring the engine, and either build handing its gates another engine; - calibration_run back in the full gates' shared hash, the preflight reading a runner scope, and the full gates or the bound spine dropping gate_scopes; - the national line's calibrated kernel hashing the codecs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ffsets, on their labels (#931) María's ruling 6 of 7 Nov: key cgt_imputation's shared stream, and BADR's band offsets on the band bounds. Stage 44 built one Generator from its seed (552) and the build period and handed it to every Table 3 cell walk, pooled walk and plan in run order, so the n-th uniform belonged to whichever walk or plan ran n-th: a cell that gained or lost its gainers, or skipped, moved the uniform of every walk and plan after it. Stage 47 drew one offset per band by position, so a band added to the facts moved every band after it. - Shared scalar: stochastic_assignment.stable_identity_uniform(key, *, seed, salt) is stable_identity_uniforms' draw for one label, and refuses a sequence. The random() adapter, _KeyedOffset, moves from isa_subscriptions into cgt_imputation beside _walk_bands; the ISA subscriptions take it and the scalar from there. - Stage 44: a cell walk's offset is keyed cell:<income band>:<age group>:<region group> and a pooled walk's pool:<income band>, salted hmrc_cgt_gains_spine:offset:<period>; a plan's jitter plan:<gain band>:<income band>, salted hmrc_cgt_gains_spine:jitter:<period>. Bands and age groups are named by their lower bounds, region groups by their labels. The seed stays 552 and the period moves into the salts. Each walk and plan reads one uniform. _walk_bands and _draw_plan_amounts keep their signatures; their rng annotation names the adapter too. - Stage 47: a band's offset is keyed <lower>|<exclusive upper> (open for the top band), salted assign_badr_qualifying_gains:offset:<period>, seed 555 (_badr_band_offset). - Declarations: stage 44's seed_mixing and stage 47's realization texts say this, and the sub-AEA mapping text no longer names a walk and plan draw order, which no longer exists. The code dictionaries, sources.yaml and source_stages.json agree; the docstrings and comments follow. - Bindings: the derived rule now finds stochastic_assignment in the reach of stages 44 and 47, so both bind it. test_uk_graph's invalidation test takes regional_property_uprating and private_school_attendance as controls in their place. - property_components' receipts-walk offset is unchanged (ruling 6). No nesting is promised. The offsets and jitters are the same at every K and whichever other cells walk, but the walks stay pooled: the rounding carry within an income band, pass 2's walk over the band's unassigned gainers, plans pooled across cells, and each BADR band's walk over its pool. What moves (synthetic counts only): - Stage 44: every walk offset and plan jitter, so band membership near each walk boundary and the within-band quantiles. On three synthetic frames, 9 of 1,002, 9 of 3,000 and 126 of 1,512 gainers changed gain band and 338, 2,920 and 1,300 changed amount; no non-gainer moved. - Stage 47: every band's offset, so each band's claimants. On three synthetic frames of equal weights no claimant row of the old walk claims in the new one, each frame's claimant count moved by one or two, and the main asset types moved with the claims. - ISA: unchanged. On three synthetic frames all five output columns are bit for bit the parent's (4,881 person rows). - Node keys: all 79 spine nodes' implementation hashes move, since graph_kernels and stochastic_assignment changed and the root binds stochastic_assignment, so every node re-keys, as does every node keyed on the spec or the coverage manifest. Values move from stage 44 on; isa_subscriptions re-keys through its cgt_imputation binding and its own module with no change in its output. - Regenerated: source_stages.json; the coverage manifest's 25 source digests; in the H2 fixture, the three texts in fixture.json, the two stages' contract hashes in uk_spine.json, and 22 values of the synthetic cgt_asset_type_facts.json, which the tool sizes to the frame after the redraw. No gates.json note goes stale. Tests (engine-free, synthetic or code facts): - New: the scalar is the array helper's draw bit for bit and refuses a sequence. Each walk's and plan's uniform, recorded under a label taken from the call's own members, is the keyed scalar of that label, salted with the period. It is unchanged when an earlier cell gains members or skips, and the one-cell higher income bands keep their members and amounts. The redraw is invariant to a row permutation. A band inserted into Table 4.1 leaves the other bands' offsets and claims as they were. ISA's offsets and jitters are the uniforms they were. - 16 mutations of a scratch copy are each caught, and the unmodified copy passes. The shared Generator as before, and offsets keyed on the walk's position, fail the invariance tests (under the shared stream 54 of the higher bands' 60 plan amounts move); mis-keyed cells, pools or plans, salts without the period and one salt for offsets and jitters fail the label tests; BADR's positional offsets as before, a position key and a lower-bound key fail the insertion test; and the ISA, scalar and binding mutations fail theirs. - The seven touched test files pass (230 tests), as do the coverage manifest, declared-seed and replay-artifact tests (31). Ruff is clean on the 10 edited Python files, and ci_test_plan verify is ok. source_stages.json and the coverage manifest are current, the H2 fixture regenerates and passes its check, and H2 parity passes through the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Six walks ordered exactly tied rows by id: stage 42's support walk, stage 26's high-value stratum, stage 28's receipts walk, stage 44's rank and stage 47's BADR and main asset type walks. Ties broken by id alone put the lowest id first every time, and the copies and geographic clones of one household differ only in their copy ordinal, so the first copy or clone won every tie (synthetic probe of the design: the lower clone index ranked first in all of 70,035 tied clone pairs under id ties, in 49.9% under keyed ties). Accepted as designed on 7 Nov, with the verifier corrections. - Helper: stochastic_assignment.keyed_tie_order(primary, ids, *, seed, salt) returns np.lexsort((ids, u, primary)), u the records' keyed_tie_uniforms(ids, *, seed, salt), stable_identity_uniforms of integer_record_keys(ids, label=salt). It is a secondary sort key, the form stage 27 uses, so distinct values keep the id order bit for bit; it draws only the tied records' uniforms. It keys each row on its own id, never on spine_household_keys, and is not the rank-inside-a-tie form. - Sites: stage 42 seed 0, salt cgt_support_split:wealth_tie; stage 26 seed 0, salt high_value_home_support:value_tie, its mid-rank unchanged; stage 28 walk_seed and a new tie_salt allocate_property_rental_income:tie, which ReceiptsAllocationSpec reads and refuses when empty or equal to a salt the stage's finance-cost draws key the same person ids on (the op has no other salt, so the stage passes those two); stage 44 one uniform per person, seed 552, salt hmrc_cgt_gains_spine:rank_tie:<period>, drawn once beside rank_key and passed to _ranked, _draw_plan_amounts and _map_remainder_amounts as a keyword-only tie (None keeps the person_id order, so their direct test calls are unchanged); stage 47 seed 555, salts assign_badr_qualifying_gains:tie:<period> and assign_main_asset_type:tie:<period>. - Declarations: the two split operations declare seed and salt and the receipts operation its tie_salt, in the code dictionaries, the strict schema (three operations), sources.yaml and source_stages.json. The selection, stratum_rule, order, ordering, mapping, realization and values texts, the two stage notes, the docstrings and the receipts walk's and asset types' receipt order texts follow; "no draw, no seed, no salt" and "deterministic, no seed" are gone. spine_build._declared_seeds registers the splits under their salts and walk_seed under the tie salt. - Rule B at stage 42: both selections take seed and salt with no default. The split, keeps_the_split's rerun and the identity tool's recompute pass cgt_support_tie_break(), read from the declared operation. The exact-total correction's candidate order (the heaviest whole households first, tied weights in row order) stays: it places a rounding-sized correction, not a draw, and copy_rule states it. - One stage 42 walk on a synthetic vector the size of a K = 15 spine takes 0.5 to 0.7 s, against 0.38 s before (the uniforms of tied households and the id check); the correction cap's comment gives the new cost. - The five stages already bind stochastic_assignment, so the derived binding rule needs nothing new. No receipt key is added; the band receipt's wealth_threshold is untouched. - Stale, left for (c): uk/gates.json's high-value home support gate note says the stage "makes no draw". gates.json is not edited here. What moves (synthetic counts only): nothing where no exact tie touches a cut or a position schedule. - Stage 42: a band's crossing household in a tie group. With three clones per household, 2 to 8 of about 850 households per frame changed copy count over six frames, and clone 0, which carried the most families in all six, now does in one; at one clone none moved (the tied family members there were too light to copy). - Stage 26: the tie group across a region's cut, 0 to 5 of 3,000 owners per frame over six frames of values on 300 atoms. - Stage 28: receipts among tied landlords; about two thirds of the tied landlords' receipts moved on three frames, none outside the tie groups, band counts unchanged. - Stage 44: 43, 99 and 62 of about 1,800 gainers moved amount where wealth varies (the blended key ties), none where wealth is equal; no non-gainer moved. - Stage 47: bit for bit on three frames. - Node keys: every spine node re-keys, since stochastic_assignment changed and the root binds it; 49 of the 79 nodes' own implementation hashes move. Values move from stage 26 on. - Regenerated: source_stages.json; the coverage manifest's 25 source digests; in the H2 fixture, 12 texts and 5 new keys in fixture.json, the stage contract hash in uk_spine.json, and the synthetic facts the tool sizes to the frames stages 28, 29 and 46 read: property_rental_facts.json (8 values, in their last digits), property_purchase_targets.json (3 values, two by 4 to 5 per cent) and cgt_asset_type_facts.json (28 values, by up to 4.4 per cent). Tests (engine-free, synthetic or code facts): - The helper is the id order on input without ties, is a permutation, equals the full lexsort on tie-heavy input (NaNs and signed zeros included), breaks a tie otherwise than ids would, does not depend on the row order, and refuses float, repeated and misaligned ids. - At each site, a tie across the cut, with ids whose keyed order differs from id order, follows the keyed order computed in the test; distinct values give the id order's result bit for bit; permuting the rows changes nothing. Stage 44's plans and remainder give ties the keyed order reversed, and every _ranked call reads the one keyed draw. - Clone-order guard (verifier P10): with three clones of 60 tied families, clone 0's share of the taken (stages 42 and 26), of the top receipts (stage 28) and of the first ranked (stage 44) lies inside the four-sigma binomial band around a third; under id order it is one (stage 26: the last clone, every time). - The CGT support scenario still selects households 2, 3 and 6; the partway walk test now takes the tied clone the keyed order puts first. The split's walk and its rerun pass the declared pair. The identity tool stays green on a tie at the crossing at three clones, passes the declared pair, and reports the family as missing and clone 0's as extra when its recompute walks by id. The seed and salt pins in test_uk_source_stages and test_uk_frs_spine follow. - 12 mutations of a scratch copy are each caught and the copy as is passes: id order in the helper and in _ranked, keying on the geographic-branch-stripped id (spine_household_keys) in either, the rank-inside-a-tie form at stages 26 and 28, the rerun on another seed, a tie salt equal to a finance-cost salt, salts without the period, the asset type tie held at seed 555 whatever the BADR seed, and the declared seeds without the splits. - The 11 touched test files and test_uk_graph pass (694 tests), as do 8 related files (357: the coverage manifest, spec bundle, country spec, copy-id table, rules-engine provenance and fixture-reading tests). Ruff is clean on the 19 edited Python files and ci_test_plan verify is ok. source_stages.json and the coverage manifest are current, the H2 fixture regenerates and passes its check, and H2 parity passes through the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling 7 of 7 Nov. Half of each subscriber's ISA amount key at stage 53, and of each Lifetime ISA payer's payment key, is the person's balance rank. _weighted_rank sorted balances stably and gave each row its own mid-rank, so tied balances ranked by row position, although its docstring said ties share a rank. Ties are common: the copies and clone halves made after the WAS wealth draw carry their root's ISA totals, so a single-adult one ties its root, and QRF draws repeat a few atoms. A root sits before its copies, as clone 0 will before its clones, so it ranked lowest among them. The stage was not row-order invariant where balances tie: the design's synthetic probe, re-run, moves 372 of 400 amounts under a row permutation on five atoms at the parent and none now (none on distinct balances at either). - _weighted_rank(values, weights, ids) orders by (value, id). Every member of a tie group takes (F(x-) + F(x)) / 2 over the total weight; a singleton keeps (c - w / 2) / total, computed as before. Both callers pass the person ids, so the keyed uniform half of each key alone orders tied holders, exchangeably, and every rank is the same bit for bit under any row order. The module docstring's step 3 says so. - No seed, salt or manifest key. No declared text says anything about ties (within_band stays "uniform"), so nothing regenerates: source_stages.json and the coverage manifest check current, and the H2 fixture holds no stage 53 value or kernel hash. No gates.json note mentions the rank. The band walk and the LISA strata stay pooled. - The rank stays in isa_subscriptions, its only user, so no shared module moves and only stage 53's kernel hash does. What moves (synthetic counts only): - Nothing where no balance ties: on the three synthetic stage frames of distinct balances every output is bit for bit the parent's. - On the same frames with balances on five atoms, every tied holder's rank moved and no singleton's (the rows arrive in id order, as the spine's tables do). Amounts still move beyond the ties, since the band walk and the within-band order are pooled: 248 to 260 of the 363 to 376 tied subscribers and 329 to 344 of the 420 to 430 others moved amount, and 46 to 56 of the 52 to 68 Lifetime ISA payers, all tied, moved payment. No balance moved. - On those frames a row permutation moved 588 to 599 of the 792 to 796 subscribers' amounts and every payer's payment at the parent, and moves none now. - With three geographic clones of 2,000 holders on five atoms, clone 0 gets the least of its three in 33 (ISA amounts) and 32 (LISA payments) per cent of the holders, against 43 and 40 by row position. - Node keys: 1 of the 99 UK kernel hashes moves, stage 53's. The gated spine re-keys 3 of its 81 nodes (isa_subscriptions, trading_allowance_receipts and the transferred gate) and the in-graph build 35 of its 113, from isa_subscriptions on; the national line on a bound spine keeps its 6. On the spine only stage 53's two subscription columns and its LISA payments move; stage 54 reads none of them. Tests (engine-free, synthetic or code facts): - New: a tie group's members share its mid-rank, by hand and over 3,000 tie-heavy rows; on id-sorted rows singletons keep the old rank bit for bit, and without ties every row does; the ranks are unchanged under five row permutations, where the old rank moved nearly every tied row's; with three clones of 2,000 holders, clone 0's share of the least ISA amount and of the least LISA payment lies inside the four-sigma binomial band around a third, and the old rank, patched in, puts both outside. - test_reordering_the_rows_moves_no_draw gains a five-atom case, which checks the ties are there. It fails at the parent (27.5 per cent of cash_isa_subscriptions differ) and passes now; the distinct case passes at both. The clone test fails at the parent too. - 8 mutations of a scratch copy are each caught and the copy as is passes: the old rank; the group mid-rank on row-order sums; ties at F(x) or at F(x-); singletons by the group formula; either call site on the old rank; ties by descending id. - The ISA stage-health entries pass on the five-atom frames at the parent and now. - test_uk_isa_subscriptions (22), test_uk_isa_stage_health (17) and test_uk_graph (53) pass. Ruff is clean on the 2 edited Python files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mits (#931) Three reviews of 65e50b9, b8bd453 and 0c850b1 found no must. This commit takes the five shoulds and the ten nits (two of them the same national-line point); none needs a ruling. The commits are not amended, so the corrections to their messages are listed below for the PR body. 0c850b1 needed no change. Tests: - Stage 44's label test also runs on a frame with a short second cell in the 37,700 income band, so that band's pooled walk runs too. Above band 0 a band's position and lower bound differ, so a pooled walk keyed on its position, or every pooled walk keyed on the lowest band, now fails it. - Stage 53: a new test records the uniform each of select_subscribers' cell walks and draw_band_amounts' walk reads, and each band jitter the amounts draw, and compares them with the pre-move expressions, computed from each call's own seed, salt and key. A call that keys or salts its draw otherwise now fails, where the helper test passed. - Stage 47: the keyed-ties test also hands each walk in turn the id order and asserts that the tied persons' BADR gains, then their asset types, move. A walk that draws the keyed order and walks another now fails it. No stage 47 clone-order guard is added: gains are plan amounts and rarely tie. - Stage 28: a new test on unequal-weight ties (half of 2,400 landlords on 60 round profits, at weights that differ) holds that, against the id order, no untied landlord changes band, while tied landlords cross and the bands' record counts move. Code: - Stage 47's band offset key refuses an edge that is not an integer, with a TypeError naming the type. _validate_table4 compares edges by value, so a roster read with float edges passed it and would have keyed every band's offset anew. A NumPy integer keys as the Python integer does. Both loaders give ints, so no value moves. - cgt_asset_type's module docstring: the residential flag is carried as weight by cgt_residential_split since microcosm#1063 and read here; the weighted systematic walk is described under the BADR claims, which use it; and the main type is assigned by error diffusion, with the keyed tie-break its only draw. - keyed_tie_order's docstring: the lowest copy ordinal always came first, not always won. Changelog fragments: - 931-cgt-keyed-offsets: the rake couples income bands; gainers cross the exempt amount; the national line's keys; ISA takes the adapter from cgt_imputation and the scalar from stochastic_assignment. - 931-keyed-tie-breaks: stage 28 with unequal-weight ties; stage 42 at one clone; which copy ordinal won (the last at stage 26); copy_rule names the cap of 64 candidates, not their row order; the national line's keys; five stage contract hashes. Corrections to the earlier messages, for the PR body (synthetic counts only): - 65e50b9: - "No nesting is promised" also misses the rake. It seeds every cell from the frame's own gainer shares and fits the seed to age and region margins over all income bands, so a change in one cell's gainers moves the raked targets of cells in other income bands. On a synthetic frame, one added cell in the lowest band moved all 20 positive targets of a two-cell higher band, by up to 0.8 per cent. - On the carry frame 70 of the 1,512 gainers crossed the annual exempt amount (none on the other two frames), so the liable set moves, and with it stage 46's arms and row counts. - On a bound spine the national line re-keys 5 of its 6 nodes, all but the bound checkpoint: the national target, problem and gate kernels hash the country spec, which the commit edits. - ISA takes _KeyedOffset from cgt_imputation and the scalar from stochastic_assignment. - b8bd453: - Stage 28: untied landlords keep their band. "Band counts unchanged" holds only where a tie group shares one weight, as on the commit's family frames. Where a tie group of unequal weights straddles a band boundary, tied landlords cross it, band counts and masses change, and the other landlords' receipts in the bands either side move. On three frames of 2,400 landlords, half on 60 round profits at weights that differ: 24 to 49 tied landlords changed band; 4 to 7 of the 11 bands' record counts changed; no untied landlord changed band; and 398 to 941 of about 1,200 untied landlords' receipts moved, by at most about 0.3 per cent. - Stage 42's zero at one clone comes from light synthetic families. At one clone the copies move where a band's crossing falls in a tie group whose members weigh above 60, as a high-value family's can. - The H2 fixture moved five stage contract hashes in uk_spine.json (cgt_support_split, high_value_home_support, property_components, hmrc_cgt_gains_spine, hmrc_cgt_asset_type_spine), not one. - The first copy or clone won every tie at stages 28, 42 and 44; at stage 26, which takes the top of an ascending order, the last did. - copy_rule names the cap of 64 candidates, not their order: tied weights take row order. - On a bound spine the national line re-keys 5 of its 6 nodes, as for 65e50b9. What moves: node keys only. No value, declaration or generated file moves. - 50 of the 99 UK kernel hashes, those that hash stochastic_assignment or cgt_asset_type: 48 stage kernels, uk.create, and the national gates through the gate registry's reach. - Every node of the hermetic and gated spine (79 and 81) and of the in-graph build (113) re-keys. On a bound spine the national line re-keys 1 of its 6 nodes, the calibrated gates. - source_stages.json and the coverage manifest check current. No declaration changed, so the H2 fixture does not regenerate. No gates.json note goes stale. Tests (engine-free, synthetic or code facts): the four touched test files and test_stochastic_assignment pass (189 tests), as do test_uk_graph, test_uk_isa_stage_health, test_uk_cgt_residential_split and test_uk_identity_stability_receipts (183). Ruff is clean on the 6 edited Python files. Under scratch the unmutated copy passes, and 9 mutations are each caught: - the pooled walk keyed on the band's position, or on the lowest band; - the ISA jitter keyed <band>:<key>, the amounts walk salted :walks, and the cell walks salted :cells; - either stage 47 walk drawing the keyed order and walking the id order; - the band key without the integer check; - stage 28 keying near ties. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The derived binding rule's walk lived in test_uk_graph.py. Holding the
transport kernels to it needs the same walk under the transport's own
tables, and country-neutral machinery belongs in one shared module, with
each country's details passed in as a declaration. This commit moves the
walk and its controls as they were. It changes no UK table, walk or key.
- test_support/microcosm_build/derived_binding_rule.py holds the
machinery. A country declares its tables in a RuleTables (core
modules, refusal-only checks, kernel stops and declaration modules)
and walks through a BindingRule built from it. The functions are
test_uk_graph's, word for word, except that each reads its tables
from the rule:
- table-free: module_definitions, writes_nothing, statement_names,
key_only_methods, merged, hashed_by and the rest;
- the rule's methods: resolve, named, helper_kind, walk, helper_walk,
kernel_body, kernel_walk and the engine-root walk;
- four checks both countries need, taken out of UK test bodies:
kernel_classes_defined (the scan for classes that declare a ref),
findings (each walk's modules outside its key, its stops outside
the key, the listed stops met and the core names outside the
Frame), and the rule's refusal_only_violations and
core_modules_off_run_graph_path.
- The lazy-export tables and the in-place calls are facts about the
first-party packages, so they are shared constants, and a rule's
lazy exports default to them.
- The generalisations the design flagged, as settled here:
- declaration modules are optional and empty by default, so a walk
then skips no module;
- each country gives its own reasons for its core modules; what is
shared is the check that each is on run_graph's path;
- a rule caches each definition's kind, and the lazy exports, per
instance, since both depend on its tables;
- findings leaves the Frame's core names out for every country, as
the UK did, since a country's Frame reason covers them.
The graph-fact reasons of the UK's core names stay in its table here;
the transport commit shares them.
- tests/engine_free/shared/test_derived_binding_rule.py holds the five
controls that moved: the refusal check, the helper rule, the kernel
rule, the engine root and the lazy exports. Each builds a rule from
its own scratch tables instead of monkeypatching the UK's.
- test_uk_graph.py keeps its tables and its UK-only parts (the stage
table, the engine root and the gate-battery phase walk), builds _RULE
once from its tables, and binds the rule's methods to the names its
code already used. Its functions and test bodies read as before, but
for the four that now call the shared checks.
What moves: nothing. No key, value, declaration or generated file.
Tests: all code facts.
- Parity: a scratch dump of every UK walk the rule computes is the same,
byte for byte, at 8ddd0a6 and at this commit. It covers the 54 stage
transforms and the engine methods each names, the 16 engine roots, the
54 merged stage walks, the bodies and walks of the 48 kernel classes
behind the 99 UK kernels (with the default halts, and without the gate
registry's stop), the 99 kernel rule walks with the 4 battery phases
and their phase-code modules, the helper walks the UK tests take
directly, the six refusal-only verdicts, the registry bindings and the
stage-health dispatch, the spine-build installs and the 99 UK kernel
hashes.
- test_uk_graph (48 tests), test_derived_binding_rule (5),
test_uk_target_compile_modules (53) and test_first_party_imports (6)
pass. Ruff is clean, and the test plan verifies.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…usal check (#931) The refusal check (writes_nothing) refused every store through a subscript. So a check that records what it has seen in a dict of its own could not be listed as refusal-only: uk_runtime.ledger_targets._assert_household_type_bindings_declared, which stores covered[value] = target_id, is one. This commit teaches the check that such a container is the function's own. It changes the shared machinery and its controls only. - local_containers finds them. A name is local when: - the function's own scope binds it to a new container: a dict, list or set display or comprehension, or a call of dict, list or set; - every other binding of it anywhere in the function, nested scopes included, is one too; - it is no parameter of the function or of anything nested in it. Bindings of every kind count against it: loop, with, walrus, unpacking and comprehension targets, deletions, imports, nested definitions, and exception and match captures. A call counts only where the function binds no dict, list or set of its own. - writes_nothing now lets the function store into or delete one item of a local container, call an in-place method on it, or augment it as a whole, in its own body or in a closure. It still refuses: - a nested store, into an item of an item or an attribute of one; - an attribute store on the container; - a store through any other name, including an alias of the container or of a parameter; - returning the container. - Two departures from the design: - An augmented store into an item (counts[row] += 1) is still a write. It calls the item's own augmented operator, which changes a list or array item in place, and the item may be an object the function was handed: seen = list(rows) holds the rows' own objects. The plain store counts[row] = counts.get(row, 0) + 1 is allowed. - The fresh binding must be in the function's own scope. The design also counted one in a nested scope, but a name that only a nested function binds is a global in the function itself. - A new control holds 10 functions that write only into their own containers and 14 that still write. What moves: nothing. No table, walk, key, value, declaration or generated file moves, since writes_nothing only checks the listed refusal-only entries. This commit neither lists _assert_household_type_bindings_declared nor unbinds frs_relationships; the UK follow-up does both. Tests: all code facts. - _assert_household_type_bindings_declared fails the check at 8ddd0a6 and passes it here (checked in scratch). - The six UK refusal-only entries still pass, and the scratch dump of every UK walk is unchanged. - Of the design prototype's 27 controls, all but its augmented-item case keep their verdict. - test_derived_binding_rule (6 tests) and test_uk_graph (48) pass. Ruff is clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On 9 Nov María asked for the bigger transport change: hold every transport kernel to the derived binding rule, as #931 did for the UK, instead of only adding the canonical encoder to the transport compile list. This commit does so with the transport's own tables, and binds what the rule finds. It moves keys only. - test_transport_binding_rule holds the 45 kernels build_transport_registry registers: the transport package's 37 kernel classes, the shared atomic geography's four, calibrate.ordered_adam@1 and the three rules kernels. test_support/microcosm_build/transport_kernels.py replays the registry's registration calls with a stand-in AxiomEngine, and the test holds the replay to a real build_transport_registry, built engine-free with the toy package's engines. Each table entry has a reason a test checks: - the eight core modules, each on run_graph's path, with transport reasons (the transport graph tests run composed graphs through run_graph); no transport kernel declares SeedSource.EXECUTOR; - 19 core definitions: 15 graph records, enums and constants, and four with transport reasons; - two refusal-only checks, graph_inputs.require_outputs and reference_profiles._validate_interval_resolution, both of which writes_nothing passes; target compilation gives the second no interval document; - one stop, target_kernels._COMPILE_MODULES, met only through the first quantile map's class and read only inside implementation hashes; - no unbound names and no declaration modules; - what the registry hands kernels whose classes their code need not name: the AxiomEngine behind the three rules kernels and the input closure, and TRANSPORT_GATE_REGISTRY behind the battery, both checked against build_transport_registry's code; and the battery's bindings' source closures, which its key binds beyond its source_hash calls (checked: a changed digest re-keys it). - The shared module gains two things: - keyed_walk, which walks a kernel with what its build hands it. Each handed object is walked on its own, with the kernel's halts, and joins the kernel's walk, so it can only add to what the kernel walk finds, as in the design's prototype. One walk with the engine as one more root would stop following the AxiomEngine class whole where the input closure names it for an isinstance check, and so miss frame.materialize. - GRAPH_RECORD_REASONS, the graph-fact reasons. The UK's _KERNEL_CORE_NAMES takes its eight from there and keeps its 12 keys; ArtifactOutput's text now reads for any kernel. - What is bound: 79 (kernel, module) pairs in 36 kernels, in 14 files, each with a short comment on what computes through it: - the canonical encoder in 32 kernels. The seven that hash the compile list gain it through the list. In ten (the four column kernels, the measure, two money kernels, two reporting kernels and the currency) only canonical_document_param's spelling check computes through it; - reference_factors, ledger_artifact and chronicle_epoch, through which decode_target_surface or validate_reference_resolution checks a surface's factor evidence and its Chronicle feed identity, in targets.problem, takeup.assign, diagnostics.calibration and geography.support_from_facts. calibration.prepare, which hashed reference_factors already, gains the other two, and the battery gains reference_factors alone; - artifact_types in support_from_facts, takeup.assign, takeup.cell_diagnostics and the three reporting kernels; calibrate.target in support_from_facts and takeup.assign; graph_atomic_geography in support_from_facts; - country_spec, gate_battery and gates in export.prepare and transport.package; build.trace in export.readback; takeup_kernels in the battery; - the Axiom input surface in concepts.encode and encode_groups, and so scenario_override; frame.rules and frame.materialize in the input closure; calibrate.hierarchy in the weights audit and the diagnostics, which also gain monetary_binding; - frame.concepts in targets.problem, for one name, the source household column the interval diagnostics count by; - frame.exact_money in the counterfactual bridge, through a new implementation_hash that the zero solve now inherits unchanged but for the encoder; - the ordered solve's internals (exact_k, the L0 gates, their initialization record, the hierarchy, monetary_binding and the encoder) in calibrate.ordered_adam@1's adapter half. calibrate.adam@1's hash, which UK kernels and microcosm-graph's parity pins embed, does not move. - compile_modules.py. The encoder joins TRANSPORT_COMPILE_MODULE_NAMES (now 14 modules) and leaves EXCLUDED_DIRECT_IMPORTS. _GRAPH_RUNTIME now covers only the kernel protocol and the declarations, with why: both are core modules the rule lists, and binding either would re-key every node. The docstring loses the #1497 sentence. No kernel hashes the file. - test_transport_compile_modules drops UNCLASSIFIED_IMPORTS, the kernel groups and the record check, as 08b437a did for the UK. It keeps the closure checks, the geography-label re-key test and HASH_INPUTS. Its list test becomes test_no_transport_kernel_hashes_a_hash_input: the list is still imported where listed, exactly the seven listed kernels hash its modules, and none of the 45 hashes the list itself. first_party_imports' recorded_import_mismatches and unhashed_imports, which nothing else used, go with their two controls. - The pinned compositions follow: test_transport_evidence_kernels' module lists, and the by-ref kernel's three in microcosm-frame's test_rules_kernels. - transport_composed gains toy_rules_bindings, the bindings document make_composed_fixture built inline, which the new test uses too. - The changelog fragment 931-transport-helper-bindings says what moves. What moves: node keys only. No value, declaration or generated file moves. - 40 of the 45 transport kernel hashes move. The four atomic geography kernels and geography.person_location@1 keep theirs. - New Zealand's packaged graph, composed as test_nz_transport_declarations composes it: all 357 nodes re-key, 353 directly and four (nz.geo.assign, derive, gate and person) by chaining. Belgium composes no graph. - 0 of the 99 UK kernel hashes move, calibrate.adam@1 keeps f6554bfe..., simulate.rules@1's hashed files are untouched, and nothing of the US's moves. No committed file records a transport kernel hash or node key. Tests: all code facts. - A scratch dump of every UK walk the rule computes, with the 99 UK kernel hashes, at 8ddd0a6 and at this commit, differs only in ArtifactOutput's reason text. - Six mutations of a scratch copy are caught: a dropped binding (the column kernels' encoder), a new unhashed helper in a money kernel's run, a reader of the compile list outside a hash, an AxiomEngine method computing through a new module, the battery's binding sources left out of its key, and a rules kernel left out of the handed-engine table. - On 8ddd0a6's sources the new test fails, on the 77 unhashed (kernel, module) pairs in 36 kernels; 26 stop names lie outside their keys there. - The 54 engine-free test files that import a changed module pass: 4,268 tests, with 105 skipped where a Belgian or NZ Chronicle artifact, the Axiom native extension or a pinned-source checkout is absent. Two failed in a first run, from a source edit made while the run held a battery kernel, which then refuses its changed binding registry; both files pass on a rerun (500 tests). test_graph_parity_pins passes (11 tests). Ruff is clean, and the test plan verifies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e shared checks (#931) The review of 354296a and 8c10ecc found that the extraction moves nothing for the UK, and four gaps in the refusal check and its controls. This commit closes them. It changes the shared machinery and its controls only: writes_nothing checks only the listed refusal-only entries, so no table, walk, verdict or key moves. - Augmenting a local container as a whole now needs a new container as the value. x += v can rebind x to whatever v's reflected operator returns, v itself included: Python tries that operator before a list's in-place concatenation, and after a set's in-place operator declines a value that is not a set. So seen = []; seen += values; seen[0] = 1 wrote into values and passed. 8c10ecc's message and docstring said a container's augmented operators change it in place or refuse; they need not. - An in-place call on a local container is a write again when it is handed another object's in-place method: seen.sort(key=values.pop) and seen.extend(iter(values.pop, None)) empty values. Both were refused before 8c10ecc. A method handed to any other call, as in sorted(rows, key=values.pop), was not seen before 8c10ecc and still is not; the docstring now says so. - A global or nonlocal declaration disqualifies a name, as 8c10ecc's message said every binding does. writes_nothing already refused such a function, so no verdict moves. - The controls: - the containers control gains a writing case for each branch that had none: an import, a nested function, an exception capture, a match capture, a class body (which the function's own scope leaves out), a global declaration, the two augmentations by a handed value and the two handed methods; and one more refusing case, a set joined by a new set; - test_the_shared_checks_report_what_they_find plants a violation for each check 354296a moved into the shared module (refusal_only_violations, core_modules_off_run_graph_path and findings), and holds the per-rule cache of each definition's kind: two rules whose tables differ classify the same alias each by its own tables, whichever classifies first. Corrections to the earlier messages, which stay as they are: - 354296a says four UK test bodies now call the shared checks; five do. It also says each moved control builds a rule from its own scratch tables instead of monkeypatching the UK's: the refusal-check control builds no rule, and only the helper-rule and kernel-rule controls monkeypatched the UK's tables before. - 8c10ecc says bindings of every kind count against a local container. Global and nonlocal declarations did not until this commit, and its list of the other bindings leaves out the augmented assignments and bare annotations it accepts. Tests: all code facts. - test_derived_binding_rule passes (7 tests). - The six UK and two transport refusal-only entries still pass, and uk_runtime.ledger_targets._assert_household_type_bindings_declared still passes the check. - Eleven mutations of the shared module each fail a new control: removing the import, nested-definition, capture or global branch, letting class bodies into a function's own scope, dropping the augmented values or the callback check, emptying refusal_only_violations, core_modules_off_run_graph_path or findings' stops, and sharing the kind cache across rules. - Ruff is clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ort reasons (#931) The review of 6c76c90 found no binding the code misses today. It found one way a new kernel could get past the transport's tables, two reasons that were wrong, and claims in the changelog fragment and the message that overstate. This commit addresses them. It changes the transport rule test and the changelog fragment only: two reason texts change, and no table entry is added or dropped, no walk moves and no key moves. - The kernels the registry hands its engines to are now held to the registry as built, not only to its code's spelling. test_the_registry_hands_the_kernels_what_the_tables_list matched only a positional argument named engine_mapping, so a new kernel handed the engines by keyword, whose run calls their methods, passed every test while its key hashed only its own module. The test now: - builds build_transport_registry with the toy package's engines (a module-scoped fixture the registration test shares), and requires the kernels that hold one of those engines, at any depth of their attributes, mappings, collections, bound methods, partials or closures, to be exactly the four HANDED_CLASSES lists; - scans keyword arguments as well as positional ones for the mapping; - requires AxiomEngine to be the only class with the rules-engine protocol's methods that the builder constructs, so a second engine class built for some bindings cannot go unwalked. The review's probe, a kernel handed the engines by keyword and registered in both the registry and the replay, now fails the test, as does the same kernel handed them through **{"engines": engine_mapping}, which only the built-registry check sees. - Two reasons are corrected: - the Frame's said no transport key hashes it. The matrix kernels, the ordered solve, the battery, the rules kernels and export hash its modules for their own reasons; it now says the rule asks no kernel to hash them; - require_outputs' said only the kernels that write the problem's and the diagnostics' outputs compute through those outputs' modules. The kernels that read them do too (the battery, the ordered solve and the weights audit), and the rule holds every one of them to hash those modules. - The changelog fragment's first sentence now says the kernels hash what they compute through as far as the rule can see, apart from the core modules and the two refusal-only checks, and its New Zealand count says the test composition's 18 synthetic quantile stand-ins re-key too. Two bindings are wider than the kernels need, and stay, since they cost keys only: simulate.counterfactual@1 now hashes frame.exact_money, which only the zero solve's quantized branches run, so an edit to it re-keys the counterfactual's nodes needlessly; and the input closure hashes frame.materialize because the rule follows AxiomEngine whole. Corrections to the earlier messages, which stay as they are: - 6c76c90 says all 357 nodes of New Zealand's packaged graph re-key. Composed as test_nz_transport_declarations composes it, the graph has 375 nodes, 18 of them synthetic quantile stand-ins, and all 375 re-key. - 6c76c90 says each table entry has a reason a test checks; the two reasons above were neither true nor checked. - 8f6c369 says 8c10ecc's message and docstring claimed a container's augmented operators change it in place or refuse; only its docstring did. Tests: all code facts. - test_transport_binding_rule passes (8 tests). Ruff is clean. - The review's probe, with the engines handed by keyword and positionally, and its **-mapping variant each fail the test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…le spine phase code (#931) María ruled on 9 Nov (late) that the gate phase-code functions and tables move into a small module that only the stale-code checks read, listed refusal-only (3a), and that the spine gates refuse a stale or missing phase code, as the full gates do (3b). This commit does both and takes the first two nits the re-review of a0933a5 left. It moves keys only, plus that one refusal. - The module. uk_runtime/gate_phase_codes.py holds the four phase-code functions, with their @cache and docstrings, and the two tables, which graph_evidence and graph_terminal defined. Their code is unchanged, so each phase code keeps its value. It adds one check both batteries call, require_current_phase_code(params, phase, *, battery), which raises ValueError, naming the battery and phase, unless params carries `<phase>_code` equal to the phase's code in that battery's table (a phase the table leaves out must carry none). Nothing else is in it. The graph construction imports the tables where it sets the params and the two runs import the check, so neither old module carries the names. - The check takes the battery's name, not its table. A run that named a table would compute through the module by the rule, so its kernel would have to hash it, and an edit to a phase's list would again re-key every node whose kernel hashes the gate kernels' modules. Only the listed check names the tables. - The rule. test_uk_graph lists the check in _REFUSAL_ONLY_HELPERS: it reads its params and the phase's code and refuses, writing nothing, and both gate kernels' runs call it first. The battery walk reads each phase's code table from the new module. source_hash leaves _KERNEL_CORE_NAMES rather than taking a new reason: only the full gates' run named it, through UK_FULL_GATE_PHASE_CODE, and the rule does not follow a listed check, so no kernel's code names it now, and test_the_core_definitions_the_kernels_name_are_listed holds the table to exactly what the walks name. Its reason moves into the check's entry. Two new tests: each gate kernel's run calls the check on its own battery, in a statement of its own, before it reads anything of its node but its parameters, and evaluates the phase it checked; and no UK kernel's key covers gate_phase_codes, the gate kernels' implementation hashes among them, while graph_evidence and graph_terminal carry none of the moved names. - The spine gates. UKSpineGateKernel.run calls the check before it reads or evaluates anything, and its docstring says it refuses a stale or missing code. test_uk_graph_evidence mirrors the full gates' stale-code test: each spine phase refuses a stale and a missing code before any artifact read, and passes its node's own declared code. The check that each spine gate node carries its own phase's code moves there from test_uk_graph's battery test, beside the full gates' in test_uk_graph_terminal. - Nit 1. uk_terminal_only_code_hash's docstring and the changelog said nine terminal-only modules never re-solve, on a bound spine or in the in-graph build. uk_runtime.diagnostics re-solves the national line, whose problem kernel hashes it. Both now say eight never re-solve on any line and diagnostics re-solves only the national line. Two comments beside the full gate nodes said an edit to one phase's own code re-keys only that phase, or never the preflight and the solve; they now say key chaining still re-keys them where a node upstream hashes the module. - Nit 2. gate_scopes' docstring, calibration_run's comment above its gate_scopes import, the docstring of test_an_edit_to_calibration_run_leaves_the_preflight_and_the_solve and docs/uk-full-build-graph.md now say what the changelog says: the move keeps the runner out of the full gate kernels' own hashes and the preflight's own code, so in the in-graph build an edit to the runner re-keys only the calibrated gates and what follows them, while on a bound spine the checkpoint, whose kernel hashes the runner, still re-keys the preflight and the solve. The doc also says the spine and local scopes and the scoped manifest are defined in gate_scopes and re-exported by calibration_run. - The changelog's gate battery fragment records the module, the check, the new refusal and source_hash leaving the core names, and counts seven refusal-only checks. Checked by simulated edits (a comment line through the module's __file__, the phase-code caches cleared), at the parent and here alike, counted as in-graph build / local line on a bound spine / national line: each of the eight re-keys only uk.full.gates.calibrated where it re-keys anything (1 of 113, 1 of 33, and 1 or 0 of 6); uk_runtime.diagnostics re-keys 1, 1 and 4 of 6 (the national problem, the dense solve, the calibrated population and the gates); calibration_run 1 of 113, 33 of 33 and 6 of 6. A static reach of the 38 terminal-only modules splits them the same way at the parent and here: eight re-solve on no line, one on the national line only, two on a bound spine only, seven in the in-graph build only, three in the in-graph build and on the local line, and seventeen on every line. What moves: node keys only, plus the spine gates' new refusal. No value, declaration or generated file moves, and neither the solve nor the country fingerprint does. - 72 of the 99 UK kernel hashes move: exactly those that hash graph_evidence.py (65), graph_terminal.py (8), gate_scopes.py (5) or calibration_run.py (2). No kernel's set of hashed modules changes, and none hashes gate_phase_codes. The other 27 keep theirs: the four atomic geography kernels, the claim and both identity kernels, the calibration and size kernels, the full target kernels, the geography, property and private rent kernels, and the national targets, problem and readback. No transport kernel hashes these files, and none moves. - Of the phase codes only terminal_code moves, since it hashes calibration_run.py; assembled_code, transferred_code and preflight_code keep their values. - So every node of every UK graph re-keys: the hermetic and gated spine (79 and 81), the in-graph build (113), the local line on a bound spine (33) and the national line (6). The bound spine kernel hashes gate_scopes.py and calibration_run.py, whose texts changed, so both lines on a bound spine re-key from their root. - After this, an edit to a phase's list re-keys that phase's node and what follows it, and nothing else. One module added to the transferred phase's list, simulated: at the parent, in graph_evidence.py, 65 kernel hashes move and 79 of 79, 81 of 81, 113 of 113, 32 of 33 and 0 of 6 nodes re-key (hermetic spine, gated spine, in-graph build, local line on a bound spine, national line); here, in gate_phase_codes.py, no kernel hash moves and 0, 1 (the transferred gate), 33 (that gate and every node after it), 0 and 0 re-key. A comment edit to gate_phase_codes.py re-keys nothing. - uk_spine.json and the H2 fixture hold stage contract hashes, and no fixture holds a gate node, so neither regenerates. source_stages.json and the coverage manifest are current. Tests (engine-free; code facts and synthetic values): test_uk_graph, test_uk_graph_evidence, test_uk_graph_terminal, test_uk_target_compile_modules and the shared rule and transport rule tests pass (168 tests), and 15 related files pass (655 tests). Through the engine slot, test_uk_full_graph_admission passes. 15 mutations on a scratch copy are caught: either gate kernel reading its table beside the check, or the full gates reading theirs in its place (the rule finds gate_phase_codes unhashed in both phases' walks); the check writing into its params or returning its code; the spine gates dropping the check, running it after their first artifact read or on the full battery's table, each caught both by the static test and by the spine refusal test; the full gates on the spine battery's table; a gate kernel hashing the module; graph_evidence re-exporting a table; and a phase's list gaining a module its gates do not compute through. Ruff is clean on the 10 edited Python files, and ci_test_plan verify is ok. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María ruled on 9 Nov (late) that the annuity and trust column's name moves into a small module of its own (4a) and that ledger_targets' household-type check is listed refusal-only (4b), so that frs_spine and frs_relationships leave the target kernels' hashes. 08b437a bound both only because of the rule's limits (run A findings CM8 and TGH7), and on a bound spine each cost a national re-solve. This commit does both. It moves keys only. - The column. uk_runtime/annuity_trust_column.py holds ANNUITY_AND_TRUST_INCOME_COLUMN, unchanged, and nothing else. frs_spine, measure_simulation, spi_spine and spi_income import it from there, and frs_spine no longer defines it (its __all__ never listed it). The resolver reads zeros where a frame lacks the column, so the name cannot be listed as unbound: the measures and the national problem hash the small module in place of frs_spine. The frs_spine root, which writes the column and has no node upstream, binds it, as the stage rule asks, and so does hmrc_spi_income_spine, which zeros it on the rows it draws and could chain on the root's key. - The check. test_uk_graph lists ledger_targets._assert_household_type_bindings_declared in _REFUSAL_ONLY_HELPERS, the eighth entry. It compares the contract's ons_household_type bindings with the ONS household type domain and refuses a binding outside it or composition rows that do not cover it once each. It writes only into the dict and set it creates, which the shared writes_nothing counts as local since 8c10ecc and 8f6c369. Its one caller, compile_uk_target_registry, calls it before the loop that compiles the references, and the compilation and the national targets reach it there, through full_targets' two loaders. The compilation's own hash keeps chronicle_epoch alone. The national targets compute through nothing their shared hash leaves out, so their own implementation hash goes. The shared rule module is unchanged. - Tests. test_the_household_type_check_refuses_before_the_targets_compile holds the entry's reason: the check's one caller, its statement before the compile loop, both kernels reaching the caller with no walk meeting frs_relationships or key hashing it, and the kernels that hash it for their own reasons keeping it (the frs_relationships stage and the assembled spine gate's own code). test_every_reader_of_the_annuity_and_trust_column_hashes_its_module checks that the small module holds the name alone, that no other uk_runtime module defines it or imports it from elsewhere, that every stage and kernel whose walk reads it hashes the module, and that the measures and the national problem meet nothing in frs_spine and leave it out of their keys. test_uk_target_compile_modules checks that an edit to frs_spine.py or frs_relationships.py leaves each of the six target and problem kernels' hashes alone, and that an edit to the small module moves the measures' and the national problem's. - Texts. The binding comments in graph_targets, graph_national and graph_kernels, and frs_spine's and measure_simulation's comments on the column. The helper-bindings changelog fragment's target bullets, its refusal-only count and its What moves. test_uk_target_compile_modules and docs/uk-full-build-graph.md stated neither binding, so neither needed a correction. What moves: node keys only. No value, declaration or generated file moves, and neither the solve nor the country fingerprint does. - 68 of the 99 UK kernel hashes move, exactly those that hash a file this commit edits: the 57 spine kernels, which all hash graph_kernels.py (20 of them frs_spine.py too); the six target and problem kernels; the full preflight and calibrated gates and the rotated holdout, which hash graph_targets.py; and the national gates and readback, which hash graph_national.py. Six kernels' hashed modules change: the create kernel and hmrc_spi_income_spine gain annuity_trust_column, the measures and the national problem swap frs_spine for it, and the compilation and the national targets lose frs_relationships, which now only the frs_relationships stage's kernel and the assembled gate's own code hash. The other 31 keep theirs, the bound spine, the dense population, calibration and size kernels, the export kernels and the four atomic geography kernels among them. No transport kernel moves. - Every node of the hermetic and gated spines (79 and 81) and of the in-graph build (113) re-keys, since the root's module changed. On a bound spine the local line re-keys 26 of its 33 nodes (the checkpoint, the sample, normalize, expand and the three geography support nodes keep theirs) and the national line 5 of 6 (all but the checkpoint). - After this, by simulated edits (a comment line through each module's __file__), counted as in-graph build / local line on a bound spine / national line: frs_spine.py re-keys 113, 0 and 0 nodes (113, 12 and 4 at the parent) and frs_relationships.py 108, 0 and 0 (108, 14 and 5), so on a bound spine neither line re-solves for either; annuity_trust_column.py re-keys 113, 12 and 4. - uk_spine.json and the H2 fixture hold stage contract hashes only, so neither regenerates. source_stages.json and the coverage manifest are current (both --check pass). Tests (engine-free; code facts and synthetic values): test_uk_graph and test_uk_target_compile_modules pass (119 tests), and with them 20 related files, the shared rule and transport rule tests among them, pass (838 tests). Through the engine slot, engine/uk/test_uk_full_target_graph passes (8 tests). Eleven mutations on a scratch copy are caught, each failing exactly the tests expected: - the column defined in frs_spine again and read from there by the measures (the kernel rule finds frs_spine's name outside both keys); - the measures importing it through frs_spine, which only re-exports it (the rule follows it to the small module; the reader test refuses it); - the root, the SPI income stage, the measures or the national problem no longer hashing the small module; - the household-type check unlisted while frs_relationships stays unbound (the kernel rule finds frs_relationships unhashed in the compilation's and the national targets' walks); - the check writing into what it is handed, or running after the compile loop; - the compilation hashing frs_relationships again; - the national targets reading the domain themselves. Ruff is clean on the 10 edited Python files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…es test (#931) The identity tool's count hold rests on _SUPPORT_SPLIT_CORRECTED_ROWS = 3: the split, the incidence clone and the residential split each correct their total once. test_the_hold_counts_every_weight_change_after_the_split reads that premise from the stages' code. Run A's re-review of a0933a5 found it still passed a second correction in the incidence clone made through an import alias, through a module-level alias, or by a wrapper that corrects twice, since it counted references only inside function bodies and a wrapper call as one correction whatever the wrapper did. This commit takes that nit, the third. It edits the test only. - Aliases. A name one of the three stage modules binds outside every function, at module level or in a class body, to one of spi_support's two helpers, a wrapper or another alias is an alias, and a reference to it counts as a reference to a wrapper does. That covers a from-import of one (its asname is the alias) and an assignment whose value refers to one, a functools.partial among them. Wrappers and aliases are found to one fixed point over the three modules. A function-local import under another name now counts once, where it names the helper, as a local alias assignment already did. At the tip the scan finds no alias but the helpers' own from-imports. - Wrappers. The test pins the wrapper set to what the code has, cgt_support._exact_support_weights and cgt_residential_split._exact_residential_weights, and each to one reference to each of the two helpers: each tries _exact_total_correction on a household its split left whole and falls back to _importance_weights_with_exact_total. So a second correction inside a wrapper fails, and so does a new _exact... wrapper. Wrappers are keyed by module and qualified name. - Text. The comment over the count says which forms it catches, and that it counts references, not calls: a loop that corrects twice, or a local alias called twice, counts once. No docstring or changelog fragment said the test catches aliases in general; only that comment did. Correction to a0933a5's message, for the PR body: its identity tool bullet said a second correction through an alias or a partial fails the premises test. Then only a local alias or partial inside the correcting function did; an import alias, a module-level alias or partial and a wrapper that corrects twice passed. Now all of them fail. What moves: nothing. No kernel hash, node key, value, declaration or generated file moves; source_stages.json and the coverage manifest are current (both --check pass at this commit). Tests (engine-free; code facts only): the touched file passes (101 tests). Run A's harness and mutations, copied to scratch and applied to a git archive snapshot of f274857 with this test laid over it: the unmutated snapshot passes, Pimport, Pmodule and Pwrapper2 now fail (all three pass at f274857), and the control Pbare still fails. Five more mutations fail here and pass at f274857: a top-level functools.partial alias, an alias of a module-level alias, a function-local import under another name, a class-body alias called from the clone transform's __call__, and a new _exact... wrapper that corrects twice. Run A's seven fix-round mutations still fail (an attribute call, a bare name in __call__, another module's wrapper, a local alias, a nested function, a lambda and a local partial). A loop running the clone's one correction twice passes, as the comment says. Ruff is clean on the edited file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…commits (#931) Three reviews of b289e8f, f274857 and c7e0523 found no must, two shoulds and fifteen nits, some of them the same finding twice. This commit takes the shoulds and the nits that need no ruling. The review's corrections to the earlier messages are below, for the PR body. It moves keys only: every edit to a hashed file is to a comment or docstring. Citations (the shoulds): - The texts for the 9 Nov (late) rulings cited them as microcosm#931 ruling 3, 3a, 3b and 4. In the PR body those numbers belong to the 9 Nov (evening) rulings, and the 9 Nov (late) block is unnumbered, so a reader following one would land on an unrelated ruling. The 24 sites now cite "microcosm#931, ruled 9 Nov (late)", as the changelog does: in gate_phase_codes, graph_evidence, graph_terminal, graph_kernels, graph_national, graph_targets, frs_spine, annuity_trust_column and four test files. "Ruling 6" still means the evening ruling. Phase codes: - The cache hit. Texts said the spine gates' new refusal means a declaration replayed from another process cannot key a spine gate on stale phase code. The check runs only when the kernel runs, and the executor serves a record stored under a node's key without calling run. UKSpineGateKernel's docstring, gate_phase_codes' module docstring, require_current_phase_code's docstring, the spine stale-code test's docstring and the changelog now say what holds: a node with a stale or missing <phase>_code is refused whenever its kernel runs, so no gate evaluates its phase under a key that names other phase code, and the check does not guard a cache hit. - test_each_gate_kernel_checks_its_phase_code_first also holds that no statement before the check returns or yields, outside the functions it defines, so every path through the run reaches the check or refuses first. It also holds that the run binds phase exactly once, from its node's parameter, before the check, so the phase evaluated is the phase checked. A return in a parameter branch before the check, and phase rebound after it, passed before and fail now, in either battery. - A new test, test_the_phase_code_check_refuses_what_its_tables_leave_out, calls the check directly. A phase its battery's table leaves out passes with no <phase>_code and is refused with one, whatever its value, and a battery with no table is refused. No gate node reaches either branch today. - test_no_uk_graph_kernel_hashes_the_phase_codes' docstring said an edit to what one phase computes through re-keys no node whose kernel hashes graph_evidence or graph_terminal. It now says what the test holds: a list edit in gate_phase_codes moves that phase's parameter, which re-keys that phase's node and every node after it, and moves no kernel's hash. - The changelog's phase-codes bullet now gives what a list edit in graph_evidence or graph_terminal used to re-key, chaining included. For a spine phase's list that was every node of the spine and of the in-graph build, and all but the checkpoint of the local line on a bound spine. For a full phase's list it was the preflight, the dense solve and every node after them. Simulated edits to the two files at c7e0523, where they have the same hashers as at 4df4065, re-key 113 of 113, 32 of 33 and 0 of 6 nodes for the first and 9, 9 and 0 for the second (in-graph build, local line on a bound spine, national line). The What moves sentences for the phase codes and for the column now add "and by key chaining every node after one". Target keys: - frs_spine's PENTYPE comment said the nodes that read the column's name key on it without keying on frs_spine. annuity_trust_column's docstring said the same of the target and problem kernels. Both now name the two kernels that do: the dense measures and the national problem. The create kernel and the SPI income stage read the name and hash both modules, and the other four target and problem kernels neither read nor hash it. The docstring also says an edit to frs_spine would otherwise re-solve both lines on a bound spine, not only the national one. - test_every_reader_of_the_annuity_and_trust_column_hashes_its_module now pins frs_spine as the household-type test pins frs_relationships: every key that covers frs_spine is the create kernel's or a spine stage's. Neither runs on the local or the national line on a bound spine, so the test now holds the changelog's claim that an edit to frs_spine re-keys no node of either line there. It also refuses a uk_runtime module reading the column's name as an attribute of any module but annuity_trust_column. measure_simulation reading it through frs_spine, which imports it, passed every target test before. - The changelog's column bullet names both checks, and its gate-battery bullet names the new phase-code checks. Premises test: - The alias scan also reads spi_support's bindings outside every function, since spi_support defines the helpers. So an alias spi_support binds, plain or as a functools.partial, which the clone imports and calls for a second correction, now counts. spi_support's functions are not counted as correction sites, and at the tip the scan finds nothing there. - The comment says the limits. Wrappers are looked for only in the three stage modules. A correction made through another module's function that corrects a total itself, such as spi_band_donors' _exact_incumbent_total or the spi_support functions that call the fallback, is not counted. Nor is one made through a helper looked up by a name built at run time. - The review's other fix was a fixed point over every uk_runtime module. It is not taken, and the test states the limit instead, which the review offered as the alternative. By name, that fixed point takes in 18 functions across 7 modules at the tip, the stage transforms' __call__ methods among them. The count would then have to tell correction sites from their callers, and keying it by module would need import resolution in the test. Not taken, for María: the check does not guard a cache hit. Closing that would need a check before keying, for example the drivers comparing each gate node's declared <phase>_code with gate_phase_codes before run_graph. No production path replays a stored declaration today, since full_build_cli and spine_build build their graphs in-process. Corrections to the earlier messages, for the PR body: - b289e8f, the first What moves bullet. Its per-file counts (65, 8, 5 and 2) overlap. Split disjointly: 65 kernels hash graph_evidence.py; 5 more hash graph_terminal.py (the four export kernels and the rotated holdout); 2 more hash gate_scopes.py (the bound spine and the national gates); none more hash calibration_run.py. That makes 72. - c7e0523, the correction to a0933a5. It should read: an import alias, a module-level alias or partial, and a wrapper that refers to a helper a second time all passed then; all three fail now, as does an alias spi_support binds (this commit). A correction repeated through one reference, in a loop, still passes, as the test's comment says. So does one made through another module's own corrector. What moves: node keys only. No value, declaration or generated file moves, and neither the solve nor the country fingerprint does. - 80 of the 99 UK kernel hashes move: exactly those that hash a file this commit edits. Split disjointly: - 57 hash graph_kernels.py: the spine stage, create, claim and identity kernels, and the spine gates; - 10 more hash graph_evidence.py: the preflight and calibrated gates, the sample, normalize, expand, pool and locations kernels, the geography mapping and gate, and the certification readiness; - 2 more hash annuity_trust_column.py: the measures and the national problem; - 6 more hash graph_targets.py: the compilation, the selection, the problem, the rotated holdout, and the national targets and gates; - 1 more hashes graph_national.py: the national readback; - 4 more hash graph_terminal.py: the export kernels. - frs_spine.py's 20 hashers are all spine kernels counted above. No kernel's set of hashed modules changes, and none hashes gate_phase_codes. The other 19 keep theirs: the bound spine, the dense solve, the calibrated, selected and full identity kernels, the size checkpoint import, draw, refit and search, the geography, property and private rent kernels, and the four atomic geography kernels. No transport module imports a uk_runtime module, and none of the 31 transport and atomic geography kernels the key probe registers moves. No phase code moves. - Node keys: every node of the hermetic and gated spines (79 and 81) and of the in-graph build (113) re-keys. On a bound spine the local line re-keys 32 of 33 nodes and the national line 5 of 6; both keep only the checkpoint, whose kernel hashes none of the seven files. b289e8f already re-keys every node of every UK graph against 4df4065, so the push re-keys nothing more. - source_stages.json and the coverage manifest are current; both --check pass at this commit. uk_spine.json and the H2 fixture hold stage contract hashes, so neither regenerates. Tests (engine-free; code facts and synthetic values): the five touched files and the shared rule and transport rule tests pass (286 tests). Ruff is clean on the 13 edited Python files. On a scratch copy of c7e0523 with this commit laid over it, 17 mutations each do what is expected: - the unmutated copy passes; - run A's Pimport, Pmodule and Pwrapper2 fail the premises test, and so does the control Pbare; - an alias, or a top-level functools.partial, that spi_support binds and the clone calls fails it too; both pass at c7e0523; - the stated limits pass, as the comment says: a correction through spi_band_donors' _exact_incumbent_total, and one through a helper looked up by a built name; - a return before the check, or phase rebound after it, fails the static test alone, in the spine gates and in the full gates; - the check accepting any code on a phase outside its table, or an unknown battery getting past it, fails the new test alone; - measure_simulation reading the column through frs_spine or spi_spine, or the bound spine hashing frs_spine, fails the reader test alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…emises test looks (#931) The re-review of e5d0acc left two nits, both in tests. This commit takes them. It moves nothing: no kernel hash, node key, value, declaration or generated file. - test_every_reader_of_the_annuity_and_trust_column_hashes_its_module looked an import of the name up by the name it binds, since facts.imports is keyed that way. So an aliased import through frs_spine or spi_spine, both of which import the name, passed it, although its docstring and the changelog's column bullet say every module reads the name from annuity_trust_column. The test now checks every from-import of the name, whatever name it binds, resolving a relative import as module_definitions does, which also catches two imports that bind one name. Keys were safe either way: the kernel rule follows frs_spine's import of the name to annuity_trust_column, and at the tip all four readers import it from there. - The premises test's comment said wrappers and aliases are found to a fixed point over four modules, and then that wrappers are looked for only in the three stages' modules, which is what the code does. It now says aliases are found over the four modules and wrappers over the three, together to a fixed point, and so does the comment over the counted names. Tests (engine-free; code facts only): test_uk_graph and test_uk_identity_stability_receipts pass (154 tests), and ruff is clean on both. On a scratch copy of e5d0acc with this commit laid over it, the re-review's three mutations now fail the reader test alone: the name imported under another name through frs_spine, or through spi_spine, in place of measure_simulation's own import, and the same import through frs_spine added beside it. The first two ran with the kernel, stage and name rules and both target-compile edit tests, the third with the whole of test_uk_graph and test_uk_target_compile_modules. The unmutated copy passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…es share a rank (#931) Step 6 of docs/uk-isa-subscriptions-1415.md has said since 960cd0a that the Lifetime ISA payment rank blends the balance and income ranks with a keyed uniform. The code never did: since the stage was added (bf0abdc), draw_lifetime_isa_payments has blended only the weighted rank of the person's Lifetime ISA balance, over every holder aged 18 to 47 rather than within income bands, with a keyed uniform at the same balance_rank_weight as the subscription amounts. Step 6 now says so (María's ruling 4 of 10 Nov). Step 3 also says, in the module docstring's words, that holders tied on a balance share its rank, so the keyed uniform alone orders them: since 0c850b1, _weighted_rank gives a tie group one mid-rank, and both draws blend it with a keyed uniform. Ruling 1 of 10 Nov has stage 53's declaration state ties in a later change. Text only: no code, declaration, generated file or key moves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part of #931. This is the output-moving half of the geography-first UK spine, stacked on #1519, which changes no outputs.
The spine clones every FRS household K times at its start: K=15 on the dense line, K=1 on national. Each clone gets its own small area and its own later draws (wealth, consumption, SPI copies and so on). The K=1 national spine nests in the K=15 dense one wherever draws are keyed: it is the clone-0 slice.
This is a draft, opened early so the commits can be followed as they land. The base is
uk-931-geography-first-prep(#1519). CI runs only on PRs to main, so this PR gets none until #1519 merges.Commit groups
frs_brmafold, the SPI-copy area draw, G7, G8 inside the WAS chain, gates, dense re-basingGroup (0)
After the move, the dense build's per-clone engine blocks are no longer identical copies. policyengine-uk's
shareholding,corporate_land_valueandconsumption_shareholdingdivide by a weighted total over the simulated households, so a block would read its own total instead of the pool's (PolicyEngine/policyengine-uk#2896 asks pe-uk to fix the cause).microcosm.frame.adapters.engine_population, and policyengine-uk's declaration of its 19 whole-population formulas. The shared module does the pool-share correction, a recorder of formula runs, and a guard that judges them. Nothing imports it yet.exactnow certifies corrected blocks plus a clean guard (exact_basis), equal to one pooled engine to float32 rounding. Today's identical copies take a correction of 1 and give the same matrix and metrics, bitwise; national runs one block and records none of the new keys.household_clone_indexcarries more than one clone. National never batches, and today no spine frame carries the column, so every call still takes one simulation.engine_batchesonly when the final frame carries more than one clone. A follow-up commit reads that record from the frame, so a resumed spine reports what a fresh run does, and holds each batch's parameter tree to the one the stage-52 restatement reads.rank_route,engine_parity).Data changes in group (0): only the annual-income factor's float32 step, kept by ruling on 7 Nov. Everything else is inert until G1.
What moves without being an output:
measure_simulationand the two frame modules);The spine graph declaration (
uk_spine.json),source_stages.jsonand the coverage manifest are unchanged.For Max:
engine_populationmodule (pool shares, the recorder,judge,EngineBlock,materialize_in_batches,frame_key_sums);exactand whether the preflight, assembler or microcosm-data mirror should also readexact_basis;WAS and land
was_wealthandwas_lisa, match the donor's household income to the engine's HBAI net income before negative incomes are reset to zero, under one predictor name,hbai_household_net_income. The donor's is WASDVTotinc_bhcR8, net regular income before housing costs less council tax. The LCFS and ETB services stages already read the engine's concept (UK: move the LCFS stage to the Living Costs and Food Survey 2024-25 #1164). Before, both stages readhousehold_net_income, which also takes off stamp duty, the TV licence, and the business rates and corporate stamp duty thatshareholdingspreads over the simulated households.ons.land.corporate_land_value(E43B) andons.land.land_value(E44R) out of the solve as measured diagnostics, inuk/calibration_measure_exclusions.json, tracked to workstream 2 of Epic: UK land value data for land value tax results by constituency #1521.corporate_land_valuehands out the engine's own aggregate by each household's share of corporate-sector wealth, so a released file meets it at any weights.land_valuedoes not hold.ons.land.household_land_value(E44N) stays bound, and its binding note now says households alone.national_contract_registry.json).tools/evaluate_uk_incumbent_surface.py, measures them, as it does every held row.What moves:
ons.land.household_land_value, now theons_landfamily's only bound row, carries three times its loss weight before. No other row's weight changes. Weights move slightly through both effects, and a calibrate-only rerun attributes them.uk_target_fitandcalibration_diagnostics.json.Group (a)
Commits 1a, 1b, 2, 3 and 4 are in, each with its review follow-up, and so are the 8 and 9 Nov follow-ups and the commits that apply the rulings on their reviews. Commits 5 to 7 are in too, and so is the derived binding rule for the transport kernels (354296a to 4df4065). So are the rest of the 9 Nov (late) rulings, for the UK, and the ISA doc correction (b289e8f to 30fe64f). Arm a1 is still to run.
uk_runtime/copy_id_table.py, so a copy id depends only on its source id and its copy path, not on K.kin a field takesid + k × radixon all five id columns. The fields, least significant first:vintage(reserved, 10^8),geographic_support(10^9),spi_support_channel(10^11),high_value_support(10^12),cgt_support_split(10^13),cgt_incidence_clone(10^17) andcgt_residential_split(10^18).cgt_support, adds tests for the dense line, stage-16 nesting and the executor, and corrects texts.tools/verify_uk_identity_stability.py) still decodes the old rule, and 11 of its tests fail there. Don't run it from those two commits. They were pushed together with 1b.weight_tables(id_multiplier=)), though none does yet.tools/evaluate_uk_dataset_size.pyinfers the multiplier from the spine's household ids or the clone-0 household ids. It refuses, by count, a dataset the inferred multiplier does not fit, where it used to miscount silently.microcosm.build.stochastic_assignmentnow binds it, so an edit to the keyed draws re-keys the stages it moves. That is 32 more stages, 37 in all.ceil(K × w / 60), as ruled on 7 Nov. K is the frame's geographic clone count, 1 on every national spine today.ceil(w / 60)bit for bit.clone_count, and the stage-health gate and the identity tool scale their checks by K.national_sampling's legacy compact sampler (sample_uk_national_frame) and its family decoder (uk_source_family_units). No build or tool has called them since 8adc836. It adds a changelog fragment of type removed. No value moves. The full build's population nodes that hash the module re-key, and so does every node downstream ofuk.full.sample.test_uk_graphderives the bindings. From each stage's transform it follows every definition the code names, withinmicrocosm.build, and requires the stage's hash to cover each module it reaches.employer_pension_contributions.rowwise_geography, with a dense test.copy_rule, and gives the identity tool's recount its 1e-9 tolerance at K = 1 as well.main_residence_valueandproperty_wealthbefore it checks stage 25's uprating. So it no longer fails on spines built sinceregional_home_valuesjoined the roster.national_sampling. It declares the modules it samples through, so the root node hashes them.spi_support_channel:inclusion), so a household's draw depends neither on row order nor on the other households. At π = 1 every household is copied without reading a uniform.e7_spi_synthetic_share. Its maximum is the repo's 4σ binomial band around the drawn share. The band moves tostochastic_assignment.binomial_share_tolerance, with a sigma argument, andstage_health.latent_attribute_tolerancestays as an alias.household_clone_index, the prior-mass allocation adds a per-clone stratum. It is inert at K = 1.clone_count. Its eligible counts are rounded,seating_scalereads full or capped, and a band's carrier mean age is withheld below three carrier source households. The gate scales by K.stochastic_assignment, where the latent-attribute band now lives.CGT_SUPPORT_CORRECTION_CANDIDATES), heaviest first, tied weights in row order. The first that can carry the correction takes it, so every root and copy keeps bitwise w / n. Only where none of the 64 can does spi_support's shared fallback run, unchanged.keeps_the_split). On 11 synthetic frames the correction would otherwise have lifted such a household to a weight that counts two copies. This check is a reading of the ruling, not in its text. It was kept on 9 Nov (evening), and 8ea25b9 names it incopy_rule.copy_ruletexts gain one clause: "except where none of the 64 heaviest households left whole can carry the exact-total correction, when the shared fallback can put it on a root or copy, off weight / copies by about one ulp of the total weight".frame.materialize,frame.rules,calibrate.geography_constants,graph.randomnessandgraph.canonicalwhere stages compute through them, and the engine adapter in every stage's hash.uk.spine-gates,uk.full-gates.preflight,uk.full-gates); the target compilation, selection, measures and problem kernels; and the national targets and problem. So are the national gates' evaluators behind the gate registry.uk_runtime/graph_frames.py, which the bound spine hashes in place ofgraph_kernels. So an edit tograph_kernelsno longer re-keys the national line and re-solves it. The size search and refit hash their five extra modules in their own hashes, so the dense solve no longer carries them.copy_rule, as ruled on 9 Nov (evening). The clause after the shared exception reads "of those 64, a selected household left at one copy can carry the correction only where its corrected weight keeps its one copy and the walk's selection, a check the fallback does not make".PolicyEngineUKEngine. Beyond the three hand-bound engine modules, the walk reaches one module,microcosm.frame.materialize, nowgraph_kernels._ENGINE_COMPUTE_MODULES.microcosm.graphexports lazily. The create kernel's call toload_sourceis a listed stop. It sits on the hermetic registry's data-only path, which no build and no test runs.regional_home_valueskeepsSeedSource.NONE, and the comment records why.materialize, and 6 of them callvariable_metadata. On a bound spine, 24 of the local line's 33 nodes re-keyed, fromuk.full.expand.ownedon.<phase>_codeparams:assembled_code,transferred_code,preflight_codeandterminal_code.preflight_code, as they refused a staleterminal_code.parity_referenceandrelease_identityjoined the solve's key, and so didcalibration_rununtil a0933a5.transport/compile_modules.py's reason for leaving the graph runtime off the transport compile list is now true of the code.calibration_run, the national calibration runner, into a newuk_runtime/gate_scopes.py. So, as Target compilation keys do not cover the geography labels in microcosm.calibrate.geography_constants #1492 designed, an edit to the runner no longer re-solves in the in-graph build.stochastic_assignment.stable_identity_uniform). The ISA stage uses it, and its output is bit for bit unchanged.property_components' receipts-walk offset is unchanged, as ruled.stochastic_assignment.keyed_tie_orderadds a uniform keyed on each row's own id as a secondary sort key, so distinct values keep the id order bit for bit. It runs at stage 42's support walk, stage 26's high-value stratum, stage 28's receipts walk, stage 44's rank, and stage 47's BADR and main asset type walks.tie_salt, in the declarations and the strict schema, and_declared_seedsregisters them. Stage 42's split,keeps_the_split's rerun and the identity tool's recompute all pass the declared pair.uk/gates.json's note on the high-value home support gate says the stage makes no draw._weighted_rankorders by (value, id), and every member of a tie group takes the group's mid-rank, (F(x−) + F(x)) / 2 over the total weight. So stage 53 no longer depends on row order where balances tie, and clone 0 no longer ranks lowest among its clones.test_support/microcosm_build/derived_binding_rule.py(RuleTablesandBindingRule), with its controls intests/engine_free/shared/test_derived_binding_rule.py.test_uk_graph.pykeeps its tables and its UK-only parts: the stage table, the engine root and the gate-battery phase walk. A dump of every UK walk the rule computes, with the 99 UK kernel hashes, is the same byte for byte before and after.ledger_targets._assert_household_type_bindings_declarednow passes the check. Listing it and unbindingfrs_relationshipscome with the UK follow-ups.test_transport_binding_rule.pywith the transport's own tables, each entry with a reason.build_transport_registryregisters: the transport package's 37 kernel classes, the shared atomic geography's four,calibrate.ordered_adam@1and the three rules kernels.implementation_hash, which the zero solve inherits. The ordered solve's internals go intocalibrate.ordered_adam@1's adapter half, socalibrate.adam@1's hash, which UK kernels embed, does not move.test_transport_compile_modules.py's Classify the imports the target-compilation kernels' hashes do not yet cover #1497 record goes, as 08b437a did for the UK. Its closure checks and hash tests stay.compile_modules.py's_GRAPH_RUNTIMEnow covers only the kernel protocol and the declarations, and says why.test_nz_transport_declarationscomposes it, the graph has 375 nodes, 18 of them synthetic quantile stand-ins, and all 375 re-key. Two reasons in its tables were wrong (the Frame's andrequire_outputs'), and 4df4065 corrects them.globalornonlocaldeclaration disqualifies a name.AxiomEnginemust be the only engine class the builder constructs.simulate.counterfactual@1hashesframe.exact_money, which only the zero solve's quantized branches run, so an edit to it re-keys the counterfactual's nodes needlessly.frame.materialize, since the rule followsAxiomEnginewhole.for,withor comprehension target that writes through a subscript; a module-level rebinding ofdict,listorset; and an in-place method handed to a call that is not an in-place call on a local, as insorted(rows, key=values.pop). No listed refusal-only check uses any of them.calibrate.adam@1's own hash gaps, whose fix would re-key the UK.uk_runtime/gate_phase_codes.py, which only the stale-code check reads. The spine gates now refuse a stale or missing phase code, as the full gates do (ruled 9 Nov (late)).require_current_phase_code(params, phase, *, battery), takes the battery's name, not its table, so no gate kernel computes through the tables or hashes the module.graph_evidence.pythe same edit moved 65 kernel hashes and re-keyed every node of the spine and of the in-graph build.uk_runtime.diagnosticsre-solves the national line. In the in-graph build an edit to the runner re-keys only the calibrated gates and what follows them, but on a bound spine it re-keys the preflight and the solve too.graph_evidence.py, 5 moregraph_terminal.py, 2 moregate_scopes.pyand none morecalibration_run.py: 72 in all.frs_spineandfrs_relationshipsoff the target kernels' keys (ruled 9 Nov (late)).uk_runtime/annuity_trust_column.py, which holds nothing else. The dense measures and the national problem hash that module in place offrs_spine. The spine root, which writes the column, binds it, and so doeshmrc_spi_income_spine.ledger_targets._assert_household_type_bindings_declaredis listed as a refusal-only check, so the target compilation and the national targets no longer hashfrs_relationships.frs_spine.pynow re-keys no node of the local or the national line, where it re-keyed 12 and 4 of their nodes; forfrs_relationships.pyit was 14 and 5.spi_supportbinds all fail. A correction repeated through one reference in a loop still passes, and so does one through another module's own corrector; the test's comment says so.frs_spineto the create kernel and the spine stages. The premises test also reads the aliasesspi_supportbinds.docs/uk-isa-subscriptions-1415.md, as ruled on 10 Nov. Step 6 said the Lifetime ISA payment rank blends the balance and income ranks. The code has only ever blended the person's Lifetime ISA balance rank with a keyed uniform, over every holder aged 18 to 47. Step 3 also says that tied balances share a rank. Text only.What moves in (a) so far:
*_build_idcolumns carry the new copy ids, so copy rows join across versions by lineage (source id and copy path), not by build ids.clone_count.copy_ruleclause and notes sentence. So its stage contract moves, and so does every node keyed on the country spec's or solve's fingerprint.source_stages.json, the coverage manifest and the H2 fixture's declarations are regenerated.geography.person_location@1keep theirs. Every node of New Zealand's graph re-keys: the packaged graph's 357 and the test composition's 18 synthetic stand-ins. Belgium composes no graph, and nothing of the US's moves. No value, declaration or generated file moves, and no committed file records a transport hash or node key.terminal_codemoves, and no transport kernel hash moves. No value, declaration or generated file moves.Rulings applied
exact.copy_rulesays so;copy_rulenames that one exception (the 8 Nov rule of never correcting a copy cannot always keep the total bitwise exact: in the synthetic nesting test at K = 7 and 13, no whole household can carry the correction);cgt_structurein stages 26 and 46 stays a listed refusal-only check;uk.full.expand, is reworded so it holds at any clone count, at (b)'s next stage-17 regeneration, so the H2 fixture regenerates once.copy_rulenames it;regional_home_valueskeepsSeedSource.NONE;microcosm.graphexports lazily, and the create kernel's H2-only use ofload_sourceis a listed stop;graph_framesmove stays.frs_spineandfrs_relationshipsleave the target kernels' hashes;microcosm.graph.canonicalin the transport compile kernels;calibration_runstays off the preflight's and the solve's key, throughgate_scopes.copy_ruletexts of stages 42 and 26 say the exact-total correction takes tied weights in row order, and stage 53's declaration says tied balances share a rank (itswithin_bandfield allows onlyuniform, so (b) settles the form);integer_record_keys' uniqueness check becomes sort-based, in (b);Proof plan
🤖 Generated with Claude Code