Add bundle_gas (Group A + Group B Strep) and the ABCs Strep sources

[MadiBal]

Adds bundle_gas, plus the two ABCs Strep sources it needs. Supersedes #187.

Stacked on #188. Base is gas-respiratory, so this diff excludes the epic_resp_infections refresh. Merge #188 first and GitHub retargets this to main automatically.

@DanWeinberger — two shape questions for you in Open questions below, plus a data-quality oddity. @anshap32 — the ABCs findings in What checking Group A turned up are the substantive part.

---

What's here

bundle_gas — 13 long-format parquets in dist/, one per contributing source standard file. Every parquet shares the same shape:

Column	Notes
geography	state name, or "United States" for the national total
geography_fips	matching FIPS code ("00" national), kept so these join to FIPS-keyed data
time	ISO YYYY-mm-dd period end
measure	which measure the row reports
value	the plotting column

plus each source's dimension columns (age, sex, race_ethnicity, onset, population_group) and suppressed for Epic.

Parquet	Source	Rows
epic_gas.parquet	epic_resp_infections	39,312
nnds_stss.parquet	nnds	24,856
abcs_gas{,_counts,_syndromes,_resistance,_emm}.parquet	abcs_gas	644 / 84 / 107 / 206 / 1,128
abcs_gbs{,_counts,_syndromes,_resistance,_serotypes,_alph}.parquet	abcs_gbs	812 / 140 / 243 / 719 / 1,260 / 286

abcs_gbs (new) — ABCs Group B Streptococcus, CDC dataset 95m5-agj4, national, annual 1997–2024. ABCs publishes one dataset per pathogen and only Group A was being read in. GBS is organized differently, so the ingest isn't a copy: infant onset timing is central (early-onset 0–6 days vs late-onset 7–89 days), a population_group column replaces sex, and serotypes plus ALPH typing stand in for emm types.

abcs_gas — brought onto main-line development from the stale gas branch and substantially rewritten (below). Note data/abcs is pneumococcus, a different pathogen despite the similar name.

---

What checking Group A turned up

Re-running the existing abcs_gas ingest against current CDC data does not just add a year. Three real problems:

1. CDC restructured the 2026 release, changing meaning. emm types was one topic whose viewby held the type; now each type is its own topic (emm 1, emm 12, …), viewby holds that type's isolate count, and value is a proportion despite units reading "Percent". The old parsing keyed on the vanished topic and produced an empty emm file. Separately, syndromes went from percent-of-cases (1997+) to rate per 100,000 (2001+), so those columns are now abcs_gas_rate_syndrome_* — carrying rates under a pct_ name would mislead silently.

   Verified against the old release: 1997 emm 1 arrives as viewby=99, value=0.225; per-type counts sum to 440; 99/440 = 0.225 = the 22.5 percent previously reported on 440 isolates.

2. A 100× scale error. Every "Percent" value is now a proportion — 2023 clindamycin arrives as 0.264 where the old release said 26.4. Caught by diffing against the old committed file; resistance and emm now both reproduce the old values exactly.

3. The ≥65 age band was being silently dropped. Under the C locale dcf_process runs in, that non-ASCII literal can't be translated for comparison, so all 28 rows vanished. Labels are now stripped to ASCII before matching.

Each of these now fails loudly: the script stop()s if "Percent" values exceed 1.5, if the emm or syndrome topics stop matching, or if no 65+ rows survive — rather than writing an empty or 100×-wrong file.

Also picked up: GAS 2024 data (the repo's raw file was from May 2025), linezolid resistance, and a survivals count.

Counts are split into their own data_counts.csv.gz in both sources. Folded in, they left columns ~85–93% NA — sparse enough that vroom's type guessing infers logical and blanks out real values. This was observed, not theoretical: 140 real GBS values silently became NA before the split.

---

Open questions

1. Parquet shape — @DanWeinberger. These are fully long (one row per measure). Looking at bundle_childhood_immunizations/overall_rates_by_source.parquet and bundle_youth_wellbeing/yrbss_state_age_demographics.parquet, those are noticeably less tall, so it's unclear whether the Epic and NNDSS outputs should be long, semi-wide, or just a parquet copy of the standard file. Happy to reshape once there's a house rule.

2. NNDSS long vs wide, and negative values — @DanWeinberger. NNDSS publishes STSS as a cumulative year-to-date count that resets each MMWR year (national 2024 runs 5 → 647 across weeks 1–52), so build.R de-accumulates it into stss_cases_weekly and keeps stss_cases_cumulative alongside. Two questions: should those be two measure levels or two columns, and what should happen to the 27 negative weekly increments? They come from NNDSS revising earlier weeks downward. Currently left as reported rather than clamped to zero, with build.R logging the count — clamping is a one-line change.

3. 13 parquets is a lot. One per source standard file, following the earlier "keep the upstream split" decision. Reasonable place to push back if consolidation is preferred.

---

Verification

• dcf::dcf_process clean on all three projects; bundle rebuild is deterministic (re-running produces no diff).
• emm percents sum to 100.00 ± 0.02 in all 28 years, and per-type counts reconcile to the typed total. GBS serotypes sum to 99.7–100.2, ALPH to 99.8–100.2; resistance within 0–100.
• All 11 standard files parse with zero vroom problems.
• dcf_check: 0 undocumented columns on abcs_gas (was 36 — the gas-branch measure_info used generic keys matching no actual column name), abcs_gbs, and epic_resp_infections.
• Bundle measure_info.json documents all 78 columns across 13 parquets — 0 undocumented, 0 falling back to STANDARD_VARS defaults — and all 97 measure levels resolve against the contributing sources. Includes a _bundle block mapping each parquet to its source file.
• geography ↔ geography_fips is 1:1 in every parquet, national always "00".
• docs/ and resources/ regenerated (14 bundles, 46 sources); every bundle_gas column renders with a name and a description or value list.

Caveats worth carrying forward

• ABCs covers a catchment of roughly 35 million people, not the whole US — rates are observed within the catchment and "national" counts are CDC's population-weighted extrapolations.
• value mixes units within a parquet; always filter or facet by measure first.
• Group A and Group B are different diseases with different at-risk populations — don't pool them.
• Three different strep-toxic-shock series now exist (NNDSS counts, ABCs rate per 100,000 from 2001, and the older ABCs percent-of-cases) and are not interchangeable.

🤖 Generated with Claude Code