juaristi22 · Nov 4, 2026 · Nov 4, 2026 · Nov 4, 2026 · Nov 4, 2026 · Nov 4, 2026
diff --git a/changelog.d/1490-uk-occupation-carry.added.md b/changelog.d/1490-uk-occupation-carry.added.md
@@ -0,0 +1,3 @@
+The UK spine now carries the FRS's own occupation as policyengine-uk's `soc2020_major_group` (microcosm#1490). `frs_employment` reads the adult table's `SOC2020` code, which gives the SOC 2020 major group as 1000-9000, and maps it to the engine's `SOC2020MajorGroup` names, held once in `occupation_constants`. The FRS asks the employment questions of everyone who has ever worked, so the group is the main job's, or the last job's for an adult not in work; child-table people are `NONE`, and an adult with a blank or unknown code refuses the build.
+
+The column is on the export allow-list, and a terminal gate, `uk_soc2020_major_group_enum_domain`, keeps it inside the engine's enum. The engine has the enum from policyengine-uk 2.173.0.
diff --git a/changelog.d/1490-uk-occupation-spi-redraw.added.md b/changelog.d/1490-uk-occupation-spi-redraw.added.md
@@ -0,0 +1,3 @@
+A new UK spine stage, `spi_employment_shell`, redraws the SOC 2020 major group of the SPI-drawn rows from their own incomes (microcosm#1490). The SPI support copies and income band donors are whole FRS households, so a drawn row kept its twin's occupation, industry and sector, all chosen with the twin's pay. The stage trains on the FRS rows' adults with an occupation and conditions on age, sex, region, employment-status group and employment, self-employment, investment and private pension income. Its classifier is microcosm-fit's new country-agnostic `CategoricalClassifier`: the equal blend of a regularised gradient-boosted classifier and a spline multinomial logit, the learner that held out best on FRS 2024-25. Each drawn row then copies the SIC division and sector of an FRS adult with the drawn group, the same status group and the nearest age and incomes, so occupation and industry stay a pair that occurs. A drawn row whose twin had no occupation and that drew no pay or profit keeps NONE, and FRS rows and undrawn copies are untouched.
+
+The receipt records held-out evidence on five household-grouped folds (log loss against the constant and each learner, the calibration slope, and occupation-share gaps within pay bands and status groups), and a stage-health gate, `uk_stage_spi_employment_shell_fit`, holds them to declared bounds. Because the stage rewrites `employment_sector`, it moves the `ons.public_sector_employment` calibration target's measured value.
diff --git a/changelog.d/1490-uk-occupation-sub-major.added.md b/changelog.d/1490-uk-occupation-sub-major.added.md
@@ -0,0 +1,3 @@
+A new UK spine stage, `aps_occupation`, draws each person's SOC 2020 sub-major group, policyengine-uk's `soc2020_sub_major_group`, within the major group the earlier stages settle (microcosm#1490). The FRS codes the major group only; the donors are the Annual Population Survey's adults with a minor group, April 2024 to March 2025 (UK Data Service SN 9408, a caller-supplied licensed tab pinned by sha256 and passed with `--aps-tab`). Each major group gets its own `CategoricalClassifier` blend over its sub-major groups, on age, sex, region, employment-status group, annual pay and SIC 2007 section, the predictors both surveys record alike. A person with no major group keeps NONE, and the signed NONE default for the column is retired.
+
+Draws are keyed on `person_source_id`, so an SPI copy draws its FRS twin's group, except for the SPI income recipients, which key on their own `person_id`. The receipt records held-out evidence on five person folds per major group (the APS file has no household identifier) and the FRS rows' expected shares against the donors'. A stage-health gate, `uk_stage_aps_occupation_fit`, holds them to declared bounds, and a terminal gate, `uk_soc2020_sub_major_group_enum_domain`, keeps the column inside the engine's enum. microcosm-fit's classifier gains `unseen_categories="ignore"`, which gives a category the fit never saw no indicator, and counts the rows that held one.
diff --git a/docs/uk-full-build-graph.md b/docs/uk-full-build-graph.md
@@ -52,7 +52,8 @@ To include raw spine construction in the same execution, pass `--spine-request /
   "--nts-ticket-tab", "/data/nts/ticket.tab",
   "--lcfs-hh-tab", "/data/lcfs/household.tab",
   "--lcfs-person-tab", "/data/lcfs/person.tab",
-  "--etb-tab", "/data/etb/household.tab"
+  "--etb-tab", "/data/etb/household.tab",
+  "--aps-tab", "/data/aps/apsp_a24m25_eul_pwta22.tab"
 ]
 ```
 
@@ -68,11 +69,11 @@ For smaller outputs, add `--dataset-households 150000` to use the common informe
 
 ## Graph owners and shared contracts
 
-The spine graph is the 48-stage source roster declared by the UK spec, built by `uk_runtime.spine_build` (the spine tool moved into the package; `tools/build_uk_frs_spine.py` is a six-line shim over it). The superseded `frs_hmrc_retained_leaves` and `hmrc_spi_income` stages and the `UK_SPINE_EXCLUSIONS` list that hid them are removed; the active HMRC path is `frs_hmrc_spine_leaves`, `spi_support_channel`, `spi_income_band_donors` and `hmrc_spi_income_spine`, and the FRS HMRC leaf columns come from `uk_runtime.frs_hmrc_source`. The spine's assembled and transferred gate batteries are graph nodes (`uk_runtime.graph_evidence`); stage evidence and fit-weight records are read back from the content store through the shared `stage_evidence` artifact rather than from in-memory collectors; the HMRC replay sidecar is rebuilt from the SPI stage's checkpoint metadata; the spine sidecar records the operation inventory and the graph manifest, and the graph manifest is saved as `spine.graph.json` under the checkpoint root.
+The spine graph is the 54-stage source roster declared by the UK spec, built by `uk_runtime.spine_build` (the spine tool moved into the package; `tools/build_uk_frs_spine.py` is a six-line shim over it). The superseded `frs_hmrc_retained_leaves` and `hmrc_spi_income` stages and the `UK_SPINE_EXCLUSIONS` list that hid them are removed; the active HMRC path is `frs_hmrc_spine_leaves`, `spi_support_channel`, `spi_income_band_donors` and `hmrc_spi_income_spine`, and the FRS HMRC leaf columns come from `uk_runtime.frs_hmrc_source`. The spine's assembled and transferred gate batteries are graph nodes (`uk_runtime.graph_evidence`); stage evidence and fit-weight records are read back from the content store through the shared `stage_evidence` artifact rather than from in-memory collectors; the HMRC replay sidecar is rebuilt from the SPI stage's checkpoint metadata; the spine sidecar records the operation inventory and the graph manifest, and the graph manifest is saved as `spine.graph.json` under the checkpoint root.
 
 | Operation | Graph owner |
 | --- | --- |
-| Raw FRS and donor preparation, enrichment, support channels | The 48 UK spine stage nodes and their declared composite operations |
+| Raw FRS and donor preparation, enrichment, support channels | The 54 UK spine stage nodes and their declared composite operations |
 | Assembled and transferred spine gates | `spine.gates.assembled`, `spine.gates.transferred` |
 | Bound checkpoint admission | `uk.full.spine_checkpoint`, when resuming a saved spine |
 | Pool sample and mass normalization | `uk.full.sample`, `uk.full.normalize` |

diff --git a/packages/microcosm-build/src/microcosm/build/source_manifest.py b/packages/microcosm-build/src/microcosm/build/source_manifest.py
@@ -79,6 +79,7 @@
         "clone_records",
         "clean_nts_travel_tables",
         "clean_was_lisa_donor",
+        "clean_aps_occupation_table",
         # microcosm#1415: ISA subscriptions per adult and the Lifetime ISA anchor.
         "allocate_isa_subscribers",
         "anchor_lifetime_isa_ownership",
@@ -165,6 +166,8 @@
         "impute_energy_subsidy_to_puf_support",
         "impute_cell_means",
         "impute_spi_housing_shell",
+        "impute_spi_employment_shell",
+        "impute_soc2020_sub_major_group",
         "impute_housing_assistance_to_puf_support",
         "impute_other_health_insurance_premiums_to_puf_support",
         "impute_prior_year_income_to_puf_support",
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,3 @@
		The UK spine now carries the FRS's own occupation as policyengine-uk's `soc2020_major_group` (microcosm#1490). `frs_employment` reads the adult table's `SOC2020` code, which gives the SOC 2020 major group as 1000-9000, and maps it to the engine's `SOC2020MajorGroup` names, held once in `occupation_constants`. The FRS asks the employment questions of everyone who has ever worked, so the group is the main job's, or the last job's for an adult not in work; child-table people are `NONE`, and an adult with a blank or unknown code refuses the build.

		The column is on the export allow-list, and a terminal gate, `uk_soc2020_major_group_enum_domain`, keeps it inside the engine's enum. The engine has the enum from policyengine-uk 2.173.0.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,3 @@
		A new UK spine stage, `spi_employment_shell`, redraws the SOC 2020 major group of the SPI-drawn rows from their own incomes (microcosm#1490). The SPI support copies and income band donors are whole FRS households, so a drawn row kept its twin's occupation, industry and sector, all chosen with the twin's pay. The stage trains on the FRS rows' adults with an occupation and conditions on age, sex, region, employment-status group and employment, self-employment, investment and private pension income. Its classifier is microcosm-fit's new country-agnostic `CategoricalClassifier`: the equal blend of a regularised gradient-boosted classifier and a spline multinomial logit, the learner that held out best on FRS 2024-25. Each drawn row then copies the SIC division and sector of an FRS adult with the drawn group, the same status group and the nearest age and incomes, so occupation and industry stay a pair that occurs. A drawn row whose twin had no occupation and that drew no pay or profit keeps NONE, and FRS rows and undrawn copies are untouched.

		The receipt records held-out evidence on five household-grouped folds (log loss against the constant and each learner, the calibration slope, and occupation-share gaps within pay bands and status groups), and a stage-health gate, `uk_stage_spi_employment_shell_fit`, holds them to declared bounds. Because the stage rewrites `employment_sector`, it moves the `ons.public_sector_employment` calibration target's measured value.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,3 @@
		A new UK spine stage, `aps_occupation`, draws each person's SOC 2020 sub-major group, policyengine-uk's `soc2020_sub_major_group`, within the major group the earlier stages settle (microcosm#1490). The FRS codes the major group only; the donors are the Annual Population Survey's adults with a minor group, April 2024 to March 2025 (UK Data Service SN 9408, a caller-supplied licensed tab pinned by sha256 and passed with `--aps-tab`). Each major group gets its own `CategoricalClassifier` blend over its sub-major groups, on age, sex, region, employment-status group, annual pay and SIC 2007 section, the predictors both surveys record alike. A person with no major group keeps NONE, and the signed NONE default for the column is retired.

		Draws are keyed on `person_source_id`, so an SPI copy draws its FRS twin's group, except for the SPI income recipients, which key on their own `person_id`. The receipt records held-out evidence on five person folds per major group (the APS file has no household identifier) and the FRS rows' expected shares against the donors'. A stage-health gate, `uk_stage_aps_occupation_fit`, holds them to declared bounds, and a terminal gate, `uk_soc2020_sub_major_group_enum_domain`, keeps the column inside the engine's enum. microcosm-fit's classifier gains `unseen_categories="ignore"`, which gives a category the fit never saw no indicator, and counts the rows that held one.