Repository navigation
Conversation
…l each (#1343) Tests only. On the pinned feed (Chronicle 51aa40f, facts 965a29ac...), the compiled 2024 targets label three values two ways, and this test fails for each: - us:statutes/26/62#adjusted_gross_income = under_1: 2 targets (the week-47 filing-season EITC rows) read "No adjusted gross income and deficit"; - us:statutes/26/62#adjusted_gross_income = all: 47 targets (Historic Table 2 and Table 1.1) read "All returns"; - age = 85_plus: 1 target (national Census PEP) reads "Age 85 and over". Like the other pinned-feed tests, it skips where the feed file is absent, as in CI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l each (#1343) The pin moves from Chronicle 51aa40f to 9b91e8b: 51aa40f plus three label-only commits (branch feed/us-51aa40f-agi-age-labels), not Chronicle main. - AGI under_1: the week-47 package's "No adjusted gross income and deficit" becomes "No adjusted gross income" (chronicle#377's 6d74e42, without its build-bundle check). - AGI all: Historic Table 2 and Table 1.1's "All returns" becomes "All returns, total". Table 1.1 now states its IRS row header explicitly, so its row-header check still reads "All returns" in cell A10. - age 85_plus: national Census PEP's "Age 85 and over" becomes "Age 85 plus". Measured (experiments/us-feed-agi-age-labels/compare.json): the feed keeps its 40,178 aggregate_fact_keys and every value, key and constraint; 816 rows in six record sets change only in label fields (66 rows) and layout.record_set_spec_hash. Compiled at 2024 as the release does, the 33,250 targets keep their names, order and values; 50 change only in the hierarchy value_label and metadata ledger_layout_groupby_value_label (2 under_1, 47 all, 1 85_plus). Registry dde00e855af8 -> b8d571e91c22; national_state 6,102 targets, 3fad14f85468 -> 3aa2a0830e7d. Pin: facts 965a29ac... -> e477e04a..., scope f0901020... -> 8fedefb7... (only source_commit), 40,178 rows. Parity resources regenerated: only the feed name (consumer_facts_us_9b91e8b.jsonl) and digest change, 32 compiled / 52 reviewed. Test pins, the re-pin doc and a changelog fragment updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#1343) Tests only. The label test added for #1343 compiles the pinned feed, so it skips in CI, and a later re-pin built on a Chronicle commit without the three label commits would bring the old labels back unnoticed. A re-pin must regenerate target_parity_feed_families.json from its feed (CI checks that the inventory restates the pinned digest), so the inventory is where CI can see the labels: - test_feed_inventory_lists_only_the_reviewed_label_variants requires the committed inventory to list, as dimension_value_label_variants, exactly the two values the feed still labels two ways (age_20 and age_21), and none of adjusted gross income under_1 or all or age 85_plus; - TestFeedLabelVariants checks the generator's listing on synthetic rows, reading both dimension_value_labels and layout.groupby_value_label. Both fail now: the inventory has no dimension_value_label_variants (KeyError) and the generator has no _dimension_value_label_variants (AttributeError). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tools/build_us_target_parity_manifest.py now writes, into target_parity_feed_families.json, dimension_value_label_variants: every (dimension, value id) the feed labels more than one way, with its labels, read from dimension_value_labels and layout.groupby_value_label as the ledger compiler reads them. Regenerated from consumer_facts_us_9b91e8b.jsonl (sha256 e477e04a...), it lists only age = age_20 and age_21; the manifest and the family counts are unchanged. A re-pin must regenerate the inventory from its feed, because CI checks that the inventory restates the pinned digest. So a re-pin built on a Chronicle commit without the three #1343 label commits now lists those values again and fails test_feed_inventory_lists_only_the_reviewed_label_variants in CI, where the compiled-target label test skips for want of the feed. The re-pin doc says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…onicle a9026cc main's microcosm#1189 moved the US feed pin from 51aa40f to cf9ece2 (51aa40f plus the digit-text guard match and the BEA B4923C package), while this branch pinned 9b91e8b (51aa40f plus the three #1343 label commits). The seven files derived from the pin conflicted, and neither side's text is right: the feed has to come from a Chronicle commit that holds both changes. That commit is a9026cc (Chronicle branch feed/us-cf9ece2-agi-age-labels, tag microcosm-us-feed-1343-v1): cf9ece2 with the three label commits cherry-picked, without conflicts. Each conflict is resolved to it: - chronicle_feed_scope.json: main's 640 pairs, source_commit a9026cc. - chronicle_feed.json: source_commit, build text, facts_sha256 4d89e4a4... and scope_sha256 04460f39...; fact_row_count stays 40,181. - build_us_target_parity_manifest.py: DEFAULT_FEED_NAME consumer_facts_us_a9026cc.jsonl, keeping this branch's label-variant listing. - target_parity_manifest.json and target_parity_feed_families.json: regenerated from the a9026cc feed. Only the feed name and digest change against main, plus this branch's dimension_value_label_variants (age_20 and age_21 only); still 32 compiled families, 53 reviewed exclusions and 82 feed families. - test_us_chronicle_feed.py: _CHRONICLE_COMMIT and _FACTS_SHA256. - docs/us-chronicle-feed-repin.md: the #1343 section rewritten for the cf9ece2 base, above main's #454 section, and the pin table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-run experiments/us-feed-agi-age-labels/compare_feeds.py with main's pinned feed as the baseline: consumer_facts_us_cf9ece2.jsonl (rebuilt at cf9ece2 with main's scope file; it reproduced 054e5684... byte for byte) against consumer_facts_us_a9026cc.jsonl (4d89e4a4...). The changes are the ones the label commits made on the 51aa40f feed: the same 40,181 keys, 816 rows changed only in labels and record_set_spec_hash (66 label rows), and of the 33,250 compiled targets, 50 change only their value label (2 under_1, 47 all, 1 85_plus). The registry version goes from dde00e855af8 to b8d571e91c22 and the national_state surface (6,102 targets) from 3fad14f85468 to 3aa2a0830e7d. Only the report's feed names, digests and row counts change. The changelog fragment names the new pin and its base. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_the_committed_feed_pin_banks_the_anchor (#1189) read the #454 re-pin report and required its new feed digest to equal the pin. After the merge the pin is a9026cc's feed, so the check failed in CI (run 40527755862, engine-free) although the feed still carries the three B4923C facts. The check now follows a list of committed reports of later re-pins that kept every row, starting from the #454 feed: each must start where the previous feed ended, keep every key and the row count, change only label fields and record_set_spec_hash, and leave the three B4923C record sets alone; the last must end at the pinned digest. The only such report is #1343's compare.json (cf9ece2 feed to a9026cc feed). A later re-pin that adds or changes rows still fails here until it brings its own check. Before: 1 failed, 2 passed in TestAnchorFeedCheck. After: the whole file, 148 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1343) The docstring still called the old feed 51aa40f; since main's pin moved, the feed this PR replaces was built at Chronicle cf9ece2. Docstring only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1343
Summary
The pinned US Chronicle feed labels three dimension values two ways, so the compiled US targets show one value id under two names. This PR re-pins the feed to a Chronicle commit that gives each value the one wording #1343 chose, and changes nothing else.
us:statutes/26/62#adjusted_gross_income=under_1us:statutes/26/62#adjusted_gross_income=allage=85_plusUpdate 2026-10-27: merged with main and moved onto #1189's feed commit
#1189 merged first and moved main's pin from
51aa40ftocf9ece2(51aa40fplus the digit-text guard match and the BEA B4923C package; tagmicrocosm-us-feed-454-v1). This branch had pinned9b91e8b(51aa40fplus the three label commits), so the two pins each lacked the other's changes, and seven files derived from the pin conflicted. This update follows the plan this description set out for that order:feed/us-cf9ece2-agi-age-labels, starts atcf9ece2and carries the three label commits, cherry-picked in order. They applied without conflicts, and each patch is identical to its original. Its last commit,a9026cc, is taggedmicrocosm-us-feed-1343-v1.11ba36758: merges main (7d45c0c33) with a merge commit. Each of the seven conflicts is resolved toa9026cc, not to either side's text: the scope file takes main's 640 pairs with the newsource_commit, the feed is rebuilt, the parity files are regenerated, and the pins move (the commit message lists each file).529944125: rerunscompare_feeds.pywith main'scf9ece2feed as the baseline and updates the changelog fragment.7fe24b972: fixes a test that Produce ESI employer premiums scaled to BEA B4923C from restored ASEC policyholder fields (#454) #1189 added and that the new pin broke in CI (run 40527755862,engine-free).test_the_committed_feed_pin_banks_the_anchorread ESI employer premium contributions as a first-class, validated resource component #454's re-pin report and required its feed digest to equal the pin.compare.json. Each report must start where the previous feed ended, keep every key, change only labels andrecord_set_spec_hash, and leave the three B4923C record sets alone. The last report must end at the pinned digest.8f1973735: a docstring intest_us_fiscal_targets.pynames the feed this PR replaces (built atcf9ece2) instead of only51aa40f. Docstring only.51aa40f: 816 rows and 50 targets, labels only.feed/us-51aa40f-agi-age-labels(9b91e8b) is no longer used.The Chronicle commit
a9026cc8f80307af0c4527fdb3779fab32ffad9fonfeed/us-cf9ece2-agi-age-labels, tagmicrocosm-us-feed-1343-v1: main's current pincf9ece2plus three label-only commits, not Chronicle main.2431ba8(fromedb7cf1): the week-47 YAML change of Give each dimension value one Chronicle-declared label, keep the package's own wording beside it (consumer_fact.v5) chronicle#377 (6d74e42), and nothing else from that PR (not itsbuild-bundlecheck).d1d28dc(from59d9d06):soi-historic-table-2andsoi-table-1-1relabel theirallrows.expected_row_header, so its row-header check compared the label itself with the IRS text in cell A10. The row now statesexpected_row_header: All returns, so the check still reads the same cell for the same text.record_set_spec_hash, the checked-in consumer fact sample built from Table 1.1's first rows, and that sample's pinned digest.cf9ece2changes none of these, and they pass unchanged at the new base.a9026cc(from9b91e8b):census-pep-2024-national-age-sexrelabels its85_plusrow.Every package's checks of the publisher's text are unchanged: the week-47 column header, Historic Table 2's
AGI_STUB0, Table 1.1's row header, and Census PEP's row header 85 and single-year age sequence.soi-historic-table-2-state-broad-2022keeps "All returns" forstate=all(chronicle#267). That is thestatedimension, which no other feed package labels, and its 2,397 targets carry one label.Checks at
a9026cc:chronicle validate-package: valid, no warnings, for each changed package at each scope year (week 47 at 2024, Historic Table 2 at 2022, Table 1.1 at 2022 and 2023, national PEP at 2024), run one at a time.chronicle build-suite: valid for the same five runs, with lineage coverage 1.0 and 0 acceptance errors. The only warnings come from running without Axiom:no_concept_alignments(week 47, national PEP) andconcept_alignment_validation_skipped(Historic Table 2, Table 1.1).test_chronicle_soi_targets,_source_cells,_source_package,_suite,_harness,_consumer_contract,_core,_facts_only,test_issue_265_non_uk_labels,test_etl_soi);test_chronicle_artifacts,_mirror,_database,_pe_source_plan,_soi_state_agi_2023,test_policyengine_chronicle_imports,_schema,test_us_poverty_target_coverage).tests/test_chronicle_bundle.py, which builds the merged bundle, was not run locally; see "Unverified".What changes in this PR (against main)
us/chronicle_feed.json:source_commit,buildtext,facts_sha256andscope_sha256.fact_row_countstays 40,181.us/chronicle_feed_scope.json: onlysource_commit; the 640 pairs are main's.target_parity_feed_families.jsonandtarget_parity_manifest.json, regenerated withtools/build_us_target_parity_manifest.py:feed_sha256change;dimension_value_label_variants(onlyage_20andage_21on this feed; see "Review" below);tools/build_us_target_parity_manifest.py:DEFAULT_FEED_NAME, and the listing of label variants._CHRONICLE_COMMITand_FACTS_SHA256intest_us_chronicle_feed.py, plus its test that the committed inventory lists only the reviewed label variants;test_us_fiscal_targets.py, thenational_statesurface version intest_pinned_feed_national_state_surface_restores_the_fences(the version hashes the hierarchy labels), andtest_pinned_feed_targets_give_each_reviewed_value_one_label;test_release_target_parity.py;test_us_esi_premiums.py: the B4923C anchor check follows the label-only re-pin's report to the pin (see the update above).docs/us-chronicle-feed-repin.md: a new section above ESI employer premium contributions as a first-class, validated resource component #454's, and the pin table.changelog.d/1343-us-feed-one-label-per-value.fixed.md.experiments/us-feed-agi-age-labels/compare_feeds.pyand its reportcompare.json.Digests
cf9ece2e7be11158c21fdc4dd38b44d92db552f6a9026cc8f80307af0c4527fdb3779fab32ffad9fconsumer_facts_us_cf9ece2.jsonl, 40,181 rows, 169,055,585 bytesconsumer_facts_us_a9026cc.jsonl, 40,181 rows, 169,056,411 bytesfacts_sha256054e5684005a7b143dcb84eabac634135ce9b2b33bb57b540602e474a4ba7c574d89e4a498ffd73a407821d9ce2c45837072901e0a29acf3206160d1fafd7c14scope_sha256f4a218fc3be534f3e447f25bf1ec4064d9f3db6b72a55874ed5db4823c92223904460f39e5c2b79011724ce33cac4ccf3b0f42812bc7179b6b532773483fe04edde00e855af8b8d571e91c22national_statesurface (6,102 targets)3fad14f854683aa2a0830e7dReproducibility:
tools/build_us_chronicle_feed.py --skip-artifactatcf9ece2, with main's scope file, reproduced main's054e5684…byte for byte, so the comparison below starts from main's pinned feed.Only labels change
uv run python experiments/us-feed-agi-age-labels/compare_feeds.py <cf9ece2 feed> <a9026cc feed> compare.json, compiled at 2024 as #940 did (packaged CD crosswalk, age targets, Medicaid substitutions):Feed rows, joined on
aggregate_fact_key:The same 40,181 keys are in both feeds, including the three B4923C facts. 816 rows change, and no field other than the ones below changes on any row.
dimension_value_labels[…adjusted_gross_income][under_1]eitc_by_agi.countand.amountdimension_value_labels[…adjusted_gross_income][all]ty2022.historic_table_2.us(55),ty2022.table_1_1(4),ty2023.table_1_1(4)dimension_value_labels[age][85_plus]census_pep.cy2024.national_resident_population_agelayout.groupby_value_labellayout.record_set_spec_hash(covers row labels)Label variants in the feed. Before: the three values, plus
age=age_20andage_21("Age 20" against "Female 20 years", and the same for 21, from Census population projections and ACS B01001). After: onlyage_20andage_21, which no compiled target selects.Compiled targets, compared field by field on
TargetSpec.to_dict():Both feeds give 33,250 targets, with the same names in the same order and the same values. 50 targets change, in exactly two fields each:
hierarchy.dimensions[…].value_label;metadata["ledger_layout_groupby_value_label"].Label variants in the compiled targets. Before: those three values. After: none, so no dimension value in the compiled targets has two labels.
The registry and surface versions move as listed above. The surface keeps its 6,102 targets.
compare.jsonlists every changed feed label, every record-set hash change and all 100 changed target fields. Against the 21 October comparison (51aa40fto9b91e8b), only the feed names, digests and row counts in the report differ.Tests
2026-10-27, with
MICROCOSM_US_CHRONICLE_FACTSand the default feed path set to thea9026ccfeed (at529944125):test_release_target_parity.py,test_us_chronicle_feed.py,test_us_fiscal_targets.py): 239 passed, 0 skipped;test_us_acs_local_release_tool,_acs_local_sparse_cd,_fiscal_refresh_builder(including Produce ESI employer premiums scaled to BEA B4923C from restored ASEC policyholder fields (#454) #1189's B4923C anchor checks),_release_gate_preflight,_soi_congressional_district_vintageand_soi_state_agi_bands: 19 passed, 0 skipped;ruff check .passes, andtools/ci_test_plan.py verifyreportsok.Without the feed, as in CI (at
7fe24b972):test_us_esi_premiums.py, the fourth test file this PR changes (in7fe24b972), which needs no feed: 148 passed;test_us_medicare_irmaa_holdout,_fiscal_refresh_builder,_release_gate_preflight,_soi_congressional_district_vintage,_soi_state_agi_bands,_spec_bundle,_acs_local_release_tool,_acs_local_sparse_cd): 637 passed, 15 skipped. The skipped tests are the ones that need the feed.CI: all ten jobs pass on
7fe24b972(run 40532463928):lint,wheels,select-countries,engine-free,engine-usandengine-ukon Python 3.13 and 3.14, andintegration-uk. Run 40527755862 on529944125failed inengine-freeon the B4923C anchor test, which7fe24b972fixes.Red, then green (2026-10-21, on the
51aa40fbase):5769515: tests only.test_pinned_feed_targets_give_each_reviewed_value_one_labelcompiles the pinned feed as the release does, and requires every target that selects each value to carry the chosen label. On the51aa40ffeed all three cases fail: 2 targets read "No adjusted gross income and deficit", 47 read "All returns", and 1 reads "Age 85 and over".09b950e: the re-pin to9b91e8b.Like the other pinned-feed tests, that test skips where the feed file is absent, as in CI. The committed-inventory test below runs in CI without the feed.
Interplay with #1192
#1192 also re-pins the feed, from the pre-#940 base
c5e5bf8with the W-2 package (f98acf4), and conflicts with main. If this PR merges first, #1192 should:a9026cc;4d89e4a4…;under_1,allor85_plus;Unverified
a9026cc(run 40523279822), because Chronicle'sci.ymlruns on every push.test_build_bundle_writes_merged_consumer_contract(1 failed, 1,209 passed). That test pins the merged bundle's row counts to an older bundle, and it failed the same way atcf9ece2(run 38173127570), with the same counts: 340,969 against the pinned 339,948.a9026cc.272c987, 25 October) still has all three old labels in these packages. Give each dimension value one Chronicle-declared label, keep the package's own wording beside it (consumer_fact.v5) chronicle#377, still open, sets the same three wordings in its label register (chronicle/dimension_values/canonical_labels.jsonatab38667). Until it merges, a re-pin onto main (Read restamped Chronicle facts at their data year before aging #1030) must carry these labels or lose them.Review (2026-10-21, of the
51aa40f-based version)No bug found in this PR or in the Chronicle branch
feed/us-51aa40f-agi-age-labels:tools/build_us_chronicle_feed.pyreproduced that version'sfacts_sha256e477e04a…andscope_sha2568fedefb7…byte for byte.compare_feeds.pyconfirmed that only labels change: 50 national targets change onlyvalue_labelandledger_layout_groupby_value_label.expected_row_headerchecks the same IRS cell, A10.value_id;Added by the review: a check CI runs without the feed file. The pinned-feed label test skips in CI.
target_parity_feed_families.json, every dimension value the feed labels more than one way. On this feed that's onlyage_20andage_21.under_1,allor85_plusgain a second label again, or if any other value gains one without review.388699c9KeyError,AttributeError)2880b263🤖 Generated with Claude Code