feat(dml): DMLDiD panel=False repeated cross sections - Chang (2020) Case 2 (DML PR-B2) (#794)

* feat(dml): DMLDiD panel=False repeated cross sections - Chang (2020) Case 2 (DML PR-B2)

Ships Chang (2020) Case 2 (repeated cross sections) as an RCS lane inside
DMLDiD, selected by a new `panel: bool = True` constructor parameter
(mirrors CallawaySantAnna). Declared RCS only: one row per unit, unique
row IDs required; no allow_unbalanced_panel RC-routing.

Score family (diff_diff/_dr_scores.py):
- chang_rcs_score: Eq 3.2 uncentered summand
  (D - g)/(p lam (1-lam)(1-g)) * ((T - lam) y - l2_hat), with the single
  control-only (T - lam) Y regression (Chang's I_kz^c) as the outcome
  nuisance and global within-cell p_hat = mean(D), lam_hat = mean(T).
- chang_rcs_lambda_slope: sample-analogue G2_lambda from the closed-form
  d/d-lambda of psi_2 (the paper prints no estimator - documented
  implementation decision, REGISTRY Note).
- chang_rcs_score_augmented: Theorem 2 psi_bar = summand - D theta/p
  + G2_lambda (T - lam); SE = sqrt(mean(psi_bar^2)/n_cell). The
  lambda-correction is mandatory (review warning quoted in docstring)
  and pinned by a regression test that recomputes the no-lambda SE.
- Shared fail-closed validator (strict-binary D AND T, strictly interior
  p_hat/lam_hat, ps in [0,1)).

Estimator lane (diff_diff/dml_did.py):
- panel=False validation: unique unit IDs, covariates required,
  stationary-sampling UserWarning (Assumption 2.3) emitted only after
  the declared-RCS structure validates.
- _precompute_rcs: per-row cohorts, is_panel False, canonical_size =
  n_obs; agg_cohort_masses deliberately unset (float-keyed lookups
  collide on >2^53 int64 labels; the bincount fallback is numerically
  identical under unique row IDs).
- _compute_dml_rcs_gt: pooled two-period cells on level outcomes,
  FOUR-group guard (any empty treated/control x period group ->
  zero_treated_control), D x T 4-class stratified folds (singleton
  stratum -> cross_fit_degenerate), propensity clip-never-drop,
  lam_hat extremeness warning, per-observation IF payload
  psi_bar/n_cell, diagnostics gain lam_hat and g2_lambda.
- RCS aggregation weights are the FIXED cohort row masses via per-cell
  agg_weight (CS-RCS convention, WIF-consistent SEs);
  aggregate('total') fails closed on RCS fits.
- Results/reports design-aware: summary Design line + obs labels,
  BusinessReport cites Assumption 2.3 + 3.2(h) on RCS, practitioner
  snippet carries panel=False, target-parameter text names the
  cohort-mass weighting, DiagnosticReport skips the Goodman-Bacon
  check on RCS fits (auto-refit path only; precomputed passthrough
  honored).

Validation (no Case 2 parity oracle exists - DoubleMLDIDCSBinary
implements the Sant'Anna-Zhao 4-regression score and omits the
lambda-correction):
- benchmarks/doubleml/chang_rcs_characterization.py: CHARACTERIZATION
  spike (not parity) with pasted transcript; Part 1 documents nonzero
  Chang-vs-DoubleML gaps on shared folds, Part 2 golden literals pin
  public DMLDiD(panel=False) == hand Eq 3.2/Thm 2 pipeline at 0.0.
- Equation-level fixtures at 1e-15/1e-12, finite-difference checks of
  G2_lambda and d/dp psi_2, oracle-nuisance recovery (true propensity
  + true l20), hand-pipeline replay at rtol 1e-14, DR in BOTH nuisance
  directions, ATT recovery on a fixed RCS DGP (theta_0 = 3),
  lambda-correction regression guard, native golden reproduction with
  live-fit gap pins, slow-lane Monte Carlo coverage, payload/idx
  contracts, degenerate handling with assert_nan_inference, >2^53
  int64 cohort-label aggregation + bootstrap replay, variance-
  conventions dml_did_rcs row (table regenerated).

Docs: REGISTRY DMLDiD section covers both cases (Case 2 equations,
8 new Notes, Case 2 assumption citations 3.2(a)/3.2(h) + Assumption
2.3 bullet, characterization reference block); dml_did.rst full
rework (Case 2 methodology, declared-RCS restrictions, loud
no-survey-weights note); chang-2020-review.md checklist flips;
llms*.txt guides; README/index one-liners; choosing_estimator,
migration-4.0, survey-roadmap, practitioner_decision_tree,
REPORTING.md, variance-conventions prose, references.rst,
doc-deps.yaml, CHANGELOG. ROADMAP DML section removed (Case 3 stays
in DEFERRED.md); new TODO row tracks Chang section-4 RCS DGP
replication.

* docs(todo): track Case 2 lambda-slope double-computation as a perf follow-up (#794 review P3)

Author: igerber

CHANGELOG.md

@@ -20,6 +20,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   HonestDiD / PreTrendsPower / DiagnosticReport / BusinessReport / MMM totals
   integration. See the REGISTRY "DMLDiD" section for equations and documented
   implementation choices.
+- **`DMLDiD(panel=False)` — Chang (2020) Case 2 repeated cross sections**:
+  the same staggered per-(g,t)-cell architecture on DECLARED cross-sectional
+  data (one observation per row, row-unique unit IDs): pooled two-period
+  cells on level outcomes, the single control-only `(T - λ̂)·Y` outcome
+  nuisance (`chang_rcs_score`), D×T-stratified folds, and the λ-corrected
+  Theorem 2 variance (`chang_rcs_score_augmented` with the explicit
+  `Ĝ₂λ(T − λ̂)` term). Per-observation influence functions; aggregation
+  weights are fixed cohort row masses (WIF-consistent SEs);
+  `aggregate('total')` fails closed on RCS fits; no survey weights (weighted
+  RCS belongs to `CallawaySantAnna(panel=False, survey_design=...)`).
+  Validated by equation-level fixtures, oracle closed forms,
+  derivative-identity checks, double robustness in both directions, and a
+  committed `DoubleMLDIDCSBinary` characterization spike (no parity oracle
+  exists — DoubleML's RCS score differs and omits the λ term).
 
 ## [3.10.0] - 2026-08-22
 

DEFERRED.md

@@ -156,4 +156,4 @@ decisions (refactor waivers, perf trade-offs, test-infrastructure calls) are rec
 | **`ImputationDiD` SE vcov is already rank-guarded upstream.** Excluded from the structural rank-guard sweep: the lead/effect vcov comes from `solve_ols(..., return_vcov=True, rank_deficient_action=...)` at the OLS fit (`imputation.py:~2316`), which already drops rank-deficient columns. The only raw inverse (`solve(V_gamma, gamma)`, `imputation.py:~2530`) is the pretrends **Wald F-test statistic** with a safe `NaN` fallback — a test statistic, not a sandwich bread — so there is no garbage-SE exposure. No structural rank-guard needed. | `imputation.py` | structural-rank-guard / 2026-06-28 |
 | **TWFE HC2/HC2-BM full-dummy dedup: drift-prone duplication already resolved by the shared builder; full delegation waived.** The former Actionable row (origin: follow-up review, citing pre-#655 line numbers) asked to extract a shared dummy-construction helper or delegate TWFE's HC2/HC2-BM path to DiD's `fixed_effects=` branch. The shared-helper half SHIPPED in #655: both sites now delegate dummy construction, drop-first convention, FE column naming, and the duplicate-term backstop to the single `build_fe_dummy_blocks` (`utils.py`) + `validate_design_term_names` implementation (`twfe.py::fit` full-dummy branch; `estimators.py::DifferenceInDifferences.fit` `fixed_effects=` branch) — the FE-naming / survey-behavior drift risk the row targeted is gone. What remains per site is ~4 lines of genuinely estimator-specific design-matrix assembly (TWFE stacks `const`/`ATT`/covariates; DiD stacks its formula terms), which is not drift-prone duplication. The remaining full-delegation option — routing `TWFE.fit` through DiD machinery with TWFE-specific cluster-default threading — would touch TWFE's user-visible result surface (coefficient-dict keys, cluster-label conventions, warning text) for near-zero residual benefit; waived on cost/benefit. | `twfe.py::fit`, `estimators.py::DifferenceInDifferences.fit`, `utils.py::build_fe_dummy_blocks` | #655 / 2026-07-10 |
 | **Survey TSL SE intentionally counts genuine-subpopulation zero-weight PSUs (matches R, NOT a bug).** Recorded as the REGISTRY § "Subpopulation Analysis" TSL-meat Note (Lumley 2004 §3.4 full-design domain convention; R `survey::svyrecvar(subset())` parity); regression-locked by `tests/test_survey.py::TestZeroWeightPsuConventionWaiver`. | `survey.py` (`_compute_stratified_psu_meat`) | PR-B / 2026-06-30 |
-| DMLDiD survey/cluster support: Chang (2020) assumes i.i.d. sampling, so `DMLDiD.fit()` accepts no `survey_design=`/`cluster=` (bare TypeError). The per-unit augmented-score influence function IS unit-level clustering (REGISTRY DMLDiD M-080 Note), so the 4.0 auto-cluster default flip is inert; what is missing is the COARSER-than-unit CR1 surface CS exposes and any design-based (survey) variance. The CS mixins were verified safe with survey keys absent (every read is `.get` with panel fallback), so the extension is additive: survey keys in `_precompute` + kit label threading (`_BOOTSTRAP_LABEL` reaches the survey-bootstrap <2-PSU warning) + a clustered-score variance derivation consistent with the cross-fitting. Needs a methodology decision on clustered cross-fitting (cluster-level folds vs unit folds with clustered scores). | `diff_diff/dml_did.py` | DML PR-B1 | Low |
+| DMLDiD survey/cluster support: Chang (2020) assumes i.i.d. sampling, so `DMLDiD.fit()` accepts no `survey_design=`/`cluster=` (bare TypeError). The per-unit augmented-score influence function IS unit-level clustering (REGISTRY DMLDiD M-080 Note), so the 4.0 auto-cluster default flip is inert; what is missing is the COARSER-than-unit CR1 surface CS exposes and any design-based (survey) variance. The CS mixins were verified safe with survey keys absent (every read is `.get` with panel fallback), so the extension is additive: survey keys in `_precompute` AND `_precompute_rcs` (both design lanes) + kit label threading (`_BOOTSTRAP_LABEL` reaches the survey-bootstrap <2-PSU warning) + a clustered-score variance derivation consistent with the cross-fitting — per-UNIT influence functions on the panel lane, per-OBSERVATION on the declared-RCS lane (where clustering would group rows into design clusters). Needs a methodology decision on clustered cross-fitting (cluster-level folds vs unit folds with clustered scores). Note RCS data is typically survey data (BRFSS/ACS/CPS), so the RCS lane raises the priority of the survey half. | `diff_diff/dml_did.py` | DML PR-B1 | Low |

README.md

@@ -121,7 +121,7 @@ Full guide: `diff_diff.get_llm_guide("practitioner")`.
 - [LPDiD](https://diff-diff.readthedocs.io/en/stable/api/lpdid.html) - Dube, Girardi, Jorda & Taylor (2025) Local Projections DiD: per-horizon long-difference event study on clean controls (no negative weighting), variance- or equally-weighted ATT, for absorbing or non-absorbing (reversible) treatment
 - [ChangesInChanges](https://diff-diff.readthedocs.io/en/stable/api/changes_in_changes.html) - Athey & Imbens (2006) nonlinear/distributional DiD for the 2x2 design: full counterfactual distribution and quantile treatment effects via CDF transformation, plus the QDiD comparison estimator via `method="qdid"`; bootstrap inference; R qte parity. Alias `CiC`
 - [LWDiD](https://diff-diff.readthedocs.io/en/stable/api/lwdid.html) - Lee & Wooldridge (2025, 2026) rolling-transformation DiD: unit-specific demean/detrend converts panel to cross-section, staggered adoption, `estimation_method` in `reg`/`ipw`/`dr`/`psm` (the papers' RA/IPW/IPWRA plus propensity-score matching), exact small-N inference on the classical collapsed regression
-- [DMLDiD](https://diff-diff.readthedocs.io/en/stable/api/dml_did.html) - Chang (2020) double/debiased machine learning DiD: staggered ATT(g,t) with cross-fitted ML nuisance learners (DML2) and Neyman-orthogonal scores, for flexible/high-dimensional covariate adjustment under conditional parallel trends
+- [DMLDiD](https://diff-diff.readthedocs.io/en/stable/api/dml_did.html) - Chang (2020) double/debiased machine learning DiD: staggered ATT(g,t) with cross-fitted ML nuisance learners (DML2) and Neyman-orthogonal scores, for flexible/high-dimensional covariate adjustment under conditional parallel trends; panel or declared repeated cross sections (`panel=False`)
 - [BaconDecomposition](https://diff-diff.readthedocs.io/en/stable/api/bacon.html) - Goodman-Bacon (2021) decomposition for diagnosing TWFE bias in staggered settings
 
 ## Diagnostics & Sensitivity

ROADMAP.md

@@ -105,16 +105,6 @@ Unified framework encompassing synthetic control and regression approaches via l
 
 **Reference**: Athey et al. (2021), *Journal of the American Statistical Association*.
 
-### Double / Debiased ML for DiD — repeated cross sections
-
-The panel lane shipped as `DMLDiD` (Chang 2020 Case 1, staggered ATT(g,t)).
-Remaining scope: Chang's Case 2 repeated-cross-section score (Equation 3.2)
-with the lambda-corrected variance (a proposed `chang_rcs_score` helper
-alongside the existing panel scores in `diff_diff/_dr_scores.py`), and its
-staggered RCS lane.
-
-**Reference**: Chang, N.-C. (2020), *The Econometrics Journal* 23(2), 177-191.
-
 ### Alternative Inference Methods
 
 - **Randomization inference**: exact p-values for small samples.

TODO.md

@@ -74,6 +74,7 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
 | Reuse the demeaner's factorized codes in `absorbed_fe_rank`/`absorbed_fe_cr1_k_increment` instead of re-factorizing: at 186k rows the rank helper adds ~1.9 ms per absorbed fit (7.7% of the fastest Rust-served TWFE fit) and the K_reference increment ~3.2 ms per clustered-hc1 absorbed fit (~13%; see `docs/performance-plan.md` "Component-aware absorbed-FE rank"), and the helpers and `demean_by_groups` factorize the same group columns. Threading the codes through the call sites halves the factorize work; the `connected_components` call itself is ~1.1 ms. Deliberately not done in the correctness PRs. | `diff_diff/utils.py` | #variance-inventory | Quick | Low |
 | `EfficientDiD` conditional path: the largest remaining O(n) stage is the sieve/nuisance construction outside the tiled pass (~9s at 10k). (The `_ridge_solve_weights` Python-prep shave landed 2026-07-07 — the `omega_stack[rest]` fancy-index copy and tail scatter are skipped when no row is zero-masked, byte-identical outputs; the `zero_mask` abs scan itself remains, needed for correctness.) | `efficient_did_covariates.py` | CS-scaling | Mid | Low |
 | `_rq_fit` LP assembly is dense (`A_eq = [X, I, -I]` with dense identity blocks, rebuilt per cell fit): a `scipy.sparse` construction would cut memory and likely HiGHS time for large cells / bootstrap-heavy covariate CiC/QDiD fits. CAVEAT before doing it: a different matrix representation can change HiGHS's vertex selection at degenerate/tied QR optima - end-to-end covariate goldens are tie-selection-gated (fine), but the `qr_cases` tight coefficient matches may shift to the equal-loss branch; re-run the parity suite and re-calibrate if needed. | `diff_diff/changes_in_changes.py::_rq_fit` | covariates PR | Quick | Low |
+| `_compute_dml_rcs_gt` computes the Case 2 λ-slope `Ĝ₂λ` twice per successful cell — once inside `chang_rcs_score_augmented()` and again for the `g2_lambda` diagnostic — duplicating the input validation and an O(n_cell) pass (nuisance fitting still dominates). Add a private augmented-score helper that accepts a precomputed slope, or return `(psi_bar, g2_lambda)` from an internal variant, keeping the public API unchanged | `diff_diff/_dr_scores.py`, `diff_diff/dml_did.py` | #794 | Quick | Low |
 | Evaluate flipping `DIFF_DIFF_SOLVE_OLS_FASTPATH` default-ON after an opt-in soak (the 2026-07 certified normal-equations Cholesky fast path, both backends). A flip needs: golden/parity-suite recapture at the tol-bounded posture (fitted ~1e-8 abs / SE ~1e-6 rel — the default today is byte-pinned in several benchmark conventions), certification-rate telemetry across real workloads (any decline is silent-correct but forfeits the speedup), and the staged default-flip protocol used for `df_convention` (v4-class change). Lifecycle tracked in docs/v4-deprecations.yaml (M-008). | `diff_diff/linalg.py::_resolve_solve_ols_fastpath`, `rust/src/linalg.rs::solve_ols_chol` | CS-scaling | Mid | Low |
 
 ### Testing / docs
@@ -82,7 +83,8 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
 |-------|----------|--------|--------|----------|
 | `CallawaySantAnnaResults.summary(alpha=...)` (and sibling staggered summaries relaying the parent renderer) relabels the confidence-interval header at the requested alpha while always printing the FIT-TIME `overall_conf_int` — silent mislabeling on any non-fit alpha (bootstrap percentile intervals cannot be reconstructed from the SE at all). DMLDiDResults now rejects non-fit alpha (the EventStudyResults.summary precedent, results_base.py:643-660); apply the same guard (or a real recomputation on analytical fits) to the CS-family summaries | `diff_diff/staggered_results.py` | DML PR-B1 review | Quick | Medium |
 | `plot_event_study`'s pointwise CI reconstruction (`_event_study.py:302-304`) gates on `np.isfinite(std_err)` but not `> 0` — a zero-SE period draws a finite zero-width interval while its stored inference is all-NaN (the CS-surface twin of the `plot_group_effects` alternate-alpha gate fixed in DML PR-B1; pre-existing main behavior, cross-surface-twins audit) — apply the same `se > 0` NaN-gate | `diff_diff/visualization/_event_study.py` | DML PR-B1 review | Quick | Medium |
-| DMLDiD tutorial notebook (CONTRIBUTING's new-estimator checklist requires a tutorial; deferred to its own PR per the numbers-locked notebook protocol — prototype in scripts, lock numbers, assemble + execute once, register in `docs/tutorials/index.rst` with a toctree short label + group card): staggered DGP with nonlinear covariate confounding, learner comparison (linear/ridge/sieve/sklearn object), post-fit aggregation + HonestDiD via the event-study container, seed/reproducibility note | `docs/tutorials/`, `docs/tutorials/index.rst` | DML PR-B1 | Mid | Medium |
+| DMLDiD tutorial notebook (CONTRIBUTING's new-estimator checklist requires a tutorial; deferred to its own PR per the numbers-locked notebook protocol — prototype in scripts, lock numbers, assemble + execute once, register in `docs/tutorials/index.rst` with a toctree short label + group card): staggered DGP with nonlinear covariate confounding, learner comparison (linear/ridge/sieve/sklearn object), post-fit aggregation + HonestDiD via the event-study container, seed/reproducibility note, and a `panel=False` repeated-cross-section example | `docs/tutorials/`, `docs/tutorials/index.rst` | DML PR-B1 | Mid | Medium |
+| Replicate Chang (2020) §4's own RCS simulation DGPs (pp. 17-21, "fully specified" per the paper review) as recovery/coverage fixtures for the `DMLDiD(panel=False)` lane — the shipped tests use a library-authored RCS design (documented in the REGISTRY checklist caveat); needs the paper PDF to extract the parameterization | `tests/test_methodology_dml_did.py`, `docs/methodology/papers/chang-2020-review.md` | DML PR-B2 | Mid | Low |
 | Optional scheduled end-to-end execution gate for the MMM tutorials (29/30): a cron-only workflow (or extension of `mmm-interop.yml`) that executes both notebooks in isolated exact-pin environments, so a stale/invalid committed posterior cannot stay green indefinitely - today the hybrid posture (deliberate: notebooks execute locally with committed outputs; CI smoke-tests the exporters without sampling; drift tests pin source + committed-output needles) leaves the MCMC claims un-re-executed in CI | `.github/workflows/mmm-interop.yml`, `docs/tutorials/29_mmm_calibration_pymc.ipynb`, `docs/tutorials/30_mmm_calibration_meridian.ipynb` | mmm-interop | Mid | Low |
 | Committed `fixest::feols` event-study golden for TWFE `event_study=True` (within + pooled specs, unbalanced + covariate panels, matched CR1 cluster convention, per-period effects + vcov block) - the in-suite gates are shared-core cross-checks (TWFE-within == MPD-absorb, pooled == MPD bit-exact), so a defect common to the shared core would pass; the live-R harness (`benchmarks/R/benchmark_multiperiod.R`, `feols(y ~ treated * time_f \| unit)`) validated the within design in `docs/benchmarks.rst` but is not a committed regression test - follow the `fixest_did_twfe_golden.json` committed-golden pattern (pytest.skip when absent) | `tests/test_fixest_did_twfe_parity.py`, `benchmarks/R/` | 3(a) R2 | Mid | Medium |
 | Type-blind `n_bootstrap` acceptance in already-validated estimators - HAD bool (`isinstance(..., int)` passes `True`, runs as 1 replicate), dCDH bool+float (its bare `< 0` check passes both `True` and `2.5`), TROP float (`2.5` passes the `>= 2` floor), SyntheticDiD float under all three variance methods + bool/negative under jackknife (its floor check is skipped there) - align these local checks with the `utils.validate_n_bootstrap` type guard (M-081 kept them out of the sweep: it scoped to previously-UNvalidated estimators only) | `diff_diff/had.py`, `diff_diff/chaisemartin_dhaultfoeuille.py`, `diff_diff/trop.py`, `diff_diff/synthetic_did.py` | 2(d) PR-B | Quick | Low |