Skip to content

Add waterfall-progression regression test (incremental bucket flags) - #559

Open
briantobin-99 wants to merge 3 commits into
btobin__introduce-io-adjustments-bucketfrom
btobin__waterfall-progression-test
Open

Add waterfall-progression regression test (incremental bucket flags)#559
briantobin-99 wants to merge 3 commits into
btobin__introduce-io-adjustments-bucketfrom
btobin__waterfall-progression-test

Conversation

@briantobin-99

@briantobin-99 briantobin-99 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

cc:
Closes:

What changed? Why?

Adds an integration test guaranteeing the release-waterfall progression keeps working, and fixes a B year-scaling regression on the legacy-footing configs.

Test. Toggling the bucket flags incrementally (v0 baseline → use_cornerstone_ghg_model+ implement_waste_disaggregation+ apply_io_year_adjustments + margins → usa_ghg_data_year: 2024, via the v03_waterfall_* configs) must reproduce each step's q-weighted average total emission factor — the release waterfall convention:

wavg N = Σᵢ Nᵢ qᵢ / Σᵢ qᵢ, with N = 1ᵀ B L (total-requirements L) and each step weighted by its own run's gross-output vector q.

  • New bedrock/utils/validation/waterfall_progression.py — computes wavg N for one config, deriving N exactly as the EF diagnostics derive N_new, with a CLI and a --v0-baseline mode recomputing the baseline from the frozen v0 snapshots.
  • New test_waterfall_progression.py (eeio_integration-marked, so it runs on the twice-weekday scheduled integration workflow) — runs each step in a fresh subprocess so functools.cache state cannot leak between configs, and asserts each step's level against pinned expected values whose per-sector N vectors match the N_and_diffs tabs of the pinned diagnostics sheets in release_v0_v03_ceda_groups (the sheets behind the v0.3 assessment) to ~5e-10 across all 405 sectors. Tolerance is 1e-4 kgCO2e/USD against step bars of ~1e-2, and v03_waterfall_final must telescope to the last step.

Fix. derive_cornerstone_B_non_finetuned keys the legacy-footing B year treatment on the bucket vocabulary: when use_ghg_year_x_in_B is false, B is scaled to usa_io_data_year (scale_cornerstone_B) and inflated to model_base_year (inflate_cornerstone_B_matrix_with_industry_pi); when true, B stays plain-vnorm. This restores the pre-#482 semantics for legacy-footing configs, which #482 had inadvertently dropped by re-keying the treatment onto the experimental use_scaled_x_and_scaled_Vnorm_for_B flag. The v0.3 path is unaffected.

Testing

All six waterfall tests pass, and running with USA_CONFIG_FILE exported into the environment (the scheduled-workflow condition, where earlier suite tests in the same process set it) produces identical values. Unit suite, black, ruff, and mypy green.

@briantobin-99
briantobin-99 force-pushed the btobin__introduce-io-adjustments-bucket branch from a45bf8d to eb6f072 Compare July 30, 2026 23:56
@briantobin-99
briantobin-99 force-pushed the btobin__waterfall-progression-test branch from ac0ff0e to 3d34b65 Compare July 30, 2026 23:56
@bl-young

Copy link
Copy Markdown
Member

I like this idea (sorry I know its still in draft), but would it make sense to build this test around the weighted average N waterfalls instead of the BLy waterfalls that we shifted away from?

briantobin-99 and others added 2 commits July 31, 2026 09:51
…gnostics sheets

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@briantobin-99
briantobin-99 force-pushed the btobin__waterfall-progression-test branch from 0afa703 to 9cd0eb6 Compare July 31, 2026 16:52
Replace the ΣBLy step metric with the q-weighted average total emission
factor (wavg N = Σ N_i q_i / Σ q_i, N = 1ᵀ B L with total-requirements L),
each step weighted by its own run's gross-output vector q — the release
waterfall convention. N is derived exactly as the EF diagnostics derive
N_new; the per-sector vectors match the pinned sheets' N_and_diffs tabs to
~5e-10 across all 405 sectors on every step, and the expected levels are
pinned from those runs. FINAL telescopes to G3 unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@briantobin-99
briantobin-99 requested a review from bl-young July 31, 2026 23:58
@briantobin-99
briantobin-99 marked this pull request as ready for review July 31, 2026 23:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants