Add waterfall-progression regression test (incremental bucket flags) - #559
Open
briantobin-99 wants to merge 3 commits into
Open
Add waterfall-progression regression test (incremental bucket flags)#559briantobin-99 wants to merge 3 commits into
briantobin-99 wants to merge 3 commits into
Conversation
briantobin-99
force-pushed
the
btobin__introduce-io-adjustments-bucket
branch
from
July 30, 2026 23:56
a45bf8d to
eb6f072
Compare
briantobin-99
force-pushed
the
btobin__waterfall-progression-test
branch
from
July 30, 2026 23:56
ac0ff0e to
3d34b65
Compare
Member
|
I like this idea (sorry I know its still in draft), but would it make sense to build this test around the weighted average N waterfalls instead of the BLy waterfalls that we shifted away from? |
…gnostics sheets Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
briantobin-99
force-pushed
the
btobin__waterfall-progression-test
branch
from
July 31, 2026 16:52
0afa703 to
9cd0eb6
Compare
Replace the ΣBLy step metric with the q-weighted average total emission factor (wavg N = Σ N_i q_i / Σ q_i, N = 1ᵀ B L with total-requirements L), each step weighted by its own run's gross-output vector q — the release waterfall convention. N is derived exactly as the EF diagnostics derive N_new; the per-sector vectors match the pinned sheets' N_and_diffs tabs to ~5e-10 across all 405 sectors on every step, and the expected levels are pinned from those runs. FINAL telescopes to G3 unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
briantobin-99
marked this pull request as ready for review
July 31, 2026 23:58
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cc:
Closes:
What changed? Why?
Adds an integration test guaranteeing the release-waterfall progression keeps working, and fixes a B year-scaling regression on the legacy-footing configs.
Test. Toggling the bucket flags incrementally (v0 baseline →
use_cornerstone_ghg_model→+ implement_waste_disaggregation→+ apply_io_year_adjustments+ margins →usa_ghg_data_year: 2024, via thev03_waterfall_*configs) must reproduce each step's q-weighted average total emission factor — the release waterfall convention:wavg N = Σᵢ Nᵢ qᵢ / Σᵢ qᵢ, with N = 1ᵀ B L (total-requirements L) and each step weighted by its own run's gross-output vector q.
bedrock/utils/validation/waterfall_progression.py— computes wavg N for one config, deriving N exactly as the EF diagnostics deriveN_new, with a CLI and a--v0-baselinemode recomputing the baseline from the frozen v0 snapshots.test_waterfall_progression.py(eeio_integration-marked, so it runs on the twice-weekday scheduled integration workflow) — runs each step in a fresh subprocess sofunctools.cachestate cannot leak between configs, and asserts each step's level against pinned expected values whose per-sector N vectors match theN_and_diffstabs of the pinned diagnostics sheets inrelease_v0_v03_ceda_groups(the sheets behind the v0.3 assessment) to ~5e-10 across all 405 sectors. Tolerance is 1e-4 kgCO2e/USD against step bars of ~1e-2, andv03_waterfall_finalmust telescope to the last step.Fix.
derive_cornerstone_B_non_finetunedkeys the legacy-footing B year treatment on the bucket vocabulary: whenuse_ghg_year_x_in_Bis false, B is scaled tousa_io_data_year(scale_cornerstone_B) and inflated tomodel_base_year(inflate_cornerstone_B_matrix_with_industry_pi); when true, B stays plain-vnorm. This restores the pre-#482 semantics for legacy-footing configs, which #482 had inadvertently dropped by re-keying the treatment onto the experimentaluse_scaled_x_and_scaled_Vnorm_for_Bflag. The v0.3 path is unaffected.Testing
All six waterfall tests pass, and running with
USA_CONFIG_FILEexported into the environment (the scheduled-workflow condition, where earlier suite tests in the same process set it) produces identical values. Unit suite, black, ruff, and mypy green.