Skip to content

refactor(config): hardcode the Cornerstone model schema, delete use_cornerstone_2026_model_schema - #561

Open
briantobin-99 wants to merge 1 commit into
btobin__waterfall-progression-testfrom
btobin__hardcode-cornerstone-schema
Open

refactor(config): hardcode the Cornerstone model schema, delete use_cornerstone_2026_model_schema#561
briantobin-99 wants to merge 1 commit into
btobin__waterfall-progression-testfrom
btobin__hardcode-cornerstone-schema

Conversation

@briantobin-99

Copy link
Copy Markdown
Contributor

cc:
Closes:

What changed? Why?

use_cornerstone_2026_model_schema is deleted. Every config set it True — the Cornerstone 2026 model schema is the only supported schema — so all code paths now assume the flag's True side unconditionally. 40 files, net −719 lines.

Model pipeline.

  • transform/eeio/derived.py: the legacy wrappers collapse to their cornerstone implementations; derive_B_usa_via_vnorm and scale_abq_via_summary.py (orphaned legacy-B helpers) are deleted, along with the unused inflate_A_matrix/inflate_B_matrix/inflate_q_or_y helpers in inflation_helpers_ceda.py.
  • transform/allocation/derived.py: map_fbs_sectors_to_model_schema always maps into Cornerstone_2025 (the electricity-disaggregation retargeting hook is unchanged); the E collapse reindexes to the Cornerstone columns directly, and the legacy BEA→CEDA groupby collapse is gone.

Allocator chain. The MECS mapping selectors, get_allocation_sectors(), and load_bea_use_table() are pinned to the Cornerstone frame — the side that generated the pinned *_CEDA allocation FBA parquets on GCS, which carry Cornerstone-frame activity codes (33131B, 335220, disaggregated 562*). The use-table alignment helpers for CEDA-vocabulary sectors are unchanged.

Diagnostics. EF diagnostics always align old (CEDA v7) vs new (Cornerstone) EF vectors, always emit comparison_type and the sector_mapping_notes tab.

Guards and shims. require_cornerstone_config (publish) and the _force_cornerstone_model_schema context manager (nowcast) are deleted with their call sites — they only asserted or forced the flag.

Config. The field is removed from USAConfig and the key stripped from all config YAMLs. Behavior-neutral: every YAML set it True, and USAConfig's strict validation requires removing the key in the same change.

Testing

  • Unit suite, black, ruff, and mypy green.
  • The v0.3 snapshot integration tests (test_usa.py, 9 tests) pass against the pinned snapshots.
  • The allocation FBA builders were re-run end-to-end and reproduce the pinned GCS parquets bit-identically: BEA_CEDA_foam_allocation_2023, BEA_CEDA_refrigeration_allocation_2023, and EIA_MECS_Energy_Allocation_CEDA_2018 (2,835 rows; rebuilt under usa_ghg_data_year: 2023, the vintage the pin was generated with — the EPA allocation tables used by the MECS builders do not yet support 2024).

Snapshot integration run for this branch: https://github.com/cornerstone-data/bedrock/actions/runs/30674351415

…ornerstone_2026_model_schema

The Cornerstone 2026 model schema is the only supported schema; every config
set the flag True. All code paths now assume the flag's True side
unconditionally:

- eeio/derived.py: legacy wrappers collapse to their cornerstone
  implementations; derive_B_usa_via_vnorm and scale_abq_via_summary.py
  (orphaned legacy-B helpers) are deleted, along with the unused
  inflate_A_matrix/inflate_B_matrix/inflate_q_or_y helpers in
  inflation_helpers_ceda.py.
- allocation/derived.py: map_fbs_sectors_to_model_schema always maps into
  Cornerstone_2025; the E collapse reindexes directly (the BEA->CEDA
  groupby collapse for the legacy schema is gone).
- The allocator chain (MECS mapping selectors, get_allocation_sectors,
  load_bea_use_table) is pinned to the Cornerstone frame — the side that
  generated the pinned *_CEDA allocation FBA parquets on GCS, which carry
  Cornerstone-frame activity codes (33131B, 335220, disaggregated 562*).
  The use-table alignment helpers for CEDA-vocabulary sectors are kept
  verbatim. Rebuilding all three allocation FBAs (foam, refrigeration,
  MECS energy) reproduces the pinned parquets bit-identically.
- EF diagnostics always align old (CEDA v7) vs new (Cornerstone) vectors
  and always emit the sector_mapping_notes tab.
- publish loses require_cornerstone_config; nowcast loses its
  _force_cornerstone_model_schema shim.
- The field is removed from USAConfig and stripped from all config YAMLs
  (behavior-neutral: every YAML set it True).

Verified: black/ruff/mypy clean, unit suite green, the v0.3 snapshot
integration tests (test_usa.py, 9 tests) pass against pinned snapshots
c60bdf4, and the three allocation FBA rebuilds match their pinned
parquets exactly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@briantobin-99
briantobin-99 requested a review from bl-young August 1, 2026 00:07
@briantobin-99
briantobin-99 marked this pull request as ready for review August 1, 2026 00:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant