fix(survey): unit-level survey_metadata recompute reports raw weight scale

CHANGELOG.md

@@ -7,6 +7,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Fixed
+- **`survey_metadata` raw-scale provenance on the unit-level recompute**
+  (CallawaySantAnna panel + repeated-cross-section lanes,
+  `TripleDifference`/`StaggeredTripleDifference` staggered engine,
+  `ContinuousDiD` analytical branch, `EfficientDiD`): the recompute passed
+  the RESOLVED (mean-1 rescaled) weights as `compute_survey_metadata`'s
+  raw weights, so `sum_weights`/`weight_range` reported the normalized
+  scale instead of the user's original weight scale. They now report the
+  raw scale, matching every other estimator (DMLDiD got the pattern in
+  its survey PR). For previously-successful fits whose survey design does
+  not alias a mutated role column, this is metadata-provenance only:
+  estimates, SEs, p-values, CIs, `df_survey`, `n_strata`, `n_psu` are
+  byte-identical, and `effective_n`/`design_effect` are scale-invariant
+  (unchanged within floating-point round-off). Additionally,
+  `ContinuousDiD`'s zero-dose-unit filter now re-resolves the survey
+  design from pristine input rows: a design column aliasing a mutated
+  role column (e.g. `weights` naming the dose column) previously
+  zero-weighted every never-treated unit on filtered fits (failing with
+  "No valid (g,t) cells"); such fits now estimate under the user's
+  original weights, consistent with the unfiltered path.
+
 ## [3.11.0] - 2026-08-29
 
 ### Added

TODO.md

@@ -37,7 +37,7 @@ Related tracking surfaces:
 | `EventStudyResults` inference-provenance fields: the container records no `vcov_type`/`cluster_name`/`n_clusters`/`df_convention`/Conley metadata, so a serialized surface cannot distinguish unit auto-clustering from explicit clustering, survey, Conley, or the one-way carve-out (3(a) R9 review). Adding them is a cross-producer M-092 schema amendment (six builders, to_dict/summary rendering, surface-suite pins) - follow the pre-cut amendment convention (optional fields appended last, ledger note same-diff) rather than bolting onto one producer | `diff_diff/results_base.py` | 3(a) R9 | Mid | Low |
 | Opt-in singleton-group pruning for TwoWayFixedEffects (static + event-study mode; reghdfe parity): singleton units/periods are currently RETAINED class-wide - the within-demeaned row is zero so points are unchanged, but N/G/residual-df count it and CR1/finite-sample SEs shift (~0.41019 -> 0.40962 measured; REGISTRY "Deviation from R" Note, R5 review) - reghdfe iteratively drops singletons by default while fixest retains them (diff-diff matches fixest); an opt-in knob needs iterative unit+period pruning with consistent cluster/survey/replicate/Conley array subsetting and a default-flip decision protocol (moves published SEs) | `diff_diff/twfe.py`, `diff_diff/estimators.py`, `diff_diff/utils.py` | 3(a) R5 | Mid | Low |
 | Cohort-timing validation input for the simultaneous-adoption event-study family (TWFE `event_study=True` + MultiPeriodDiD through 3.9): an optional `first_treat=`/`cohort=` column so simultaneous adoption becomes checkable under the contract-valid time-invariant `D_i` indicator - today the staggered-adoption advisory derives timing from within-unit 0->1 transitions, so it can only fire on off-contract time-varying `D_it` input, and with valid `D_i` adoption timing is not observable in the inputs at all (REGISTRY "staggered-adoption detection limit" Notes, both sections); design questions: validate-only vs steering error, and interplay with the M-011 removal | `diff_diff/twfe.py`, `diff_diff/estimators.py` | 3(a) R2 | Mid | Medium |
-| Unit-level `survey_metadata` recompute passes NORMALIZED weights as `compute_survey_metadata`'s `raw_weights` in three pre-existing families — CS (`staggered.py:2412`), the staggered DDD engine (`_staggered_triple_diff_engine.py:299`), and ContinuousDiD (`continuous_did.py:1107`, where the variable is even named `raw_w_unit`) — so `sum_weights`/`weight_range` report the mean-1 rescaled values (scale-invariant fields are unaffected; estimates/inference untouched). Same class as the DMLDiD fix in the DML survey PR (raw per-obs weights collapsed per unit, groupby-first): apply the identical pattern per family with raw-scale metadata pins on non-unit-scale weights | `diff_diff/staggered.py`, `diff_diff/_staggered_triple_diff_engine.py`, `diff_diff/continuous_did.py` | DML survey PR review | Quick | Medium |
+| `ContinuousDiD` `survey_metadata` granularity diverges across inference branches: the analytical branch recomputes at UNIT level (raw unit-weight sum/range since the raw-scale provenance fix) while `n_bootstrap>0` keeps the OBS-level metadata from `_resolve_survey_for_fit` (raw per-obs sum over the dose-filtered frame) — both raw-scale, but `sum_weights` differs by panel length between branches on the same data (characterization pin: `TestContinuousDiDRawScale::test_bootstrap_metadata_stays_obs_level_raw`). Unify on one granularity (unit-level, matching the analytic branch and CS/EfficientDiD convention) | `diff_diff/continuous_did.py` | survey-meta-raw | Quick | Low |
 | Bootstrapped ES REPLAY containers (CS, DMLDiD and EfficientDiD) publish the analytical `survey_metadata.df_survey` provenance scalar beside percentile inference on survey fits — a cross-estimator provenance residual (shipped CS behaves identically; the per-row df/inference channels are correctly NaN, the scalar metadata field is the residual; DMLDiD joined with its 3.11 survey support). Evaluate clearing/gating it on all three estimators together (cross-surface twins) | `diff_diff/staggered_results.py`, `diff_diff/dml_did_results.py`, `diff_diff/efficient_did_results.py`, `diff_diff/results_base.py` | EDiD-replay review | Quick | Low |
 | DMLDiD replicate-weight survey designs (currently fail closed with `NotImplementedError` pointing here): the IF-reweighting route (`compute_replicate_if_variance` on the augmented scores) is BETTER justified for DMLDiD than for CS — Neyman orthogonality makes the score first-order insensitive to nuisance perturbation — but needs per-cell replicate SE plumbing, the aggregate `_se_from_psi` replicate branch already activates on kit keys, and its own rejection/df test matrix (QR-rank df, `n_valid - 1` tightening) | `diff_diff/dml_did.py` | DML survey PR | Mid | Low |
 | ImputationDiD/TwoStageDiD `aggregate()` recompute levels on bootstrapped fits fail closed ('simple' relays since the M-027 per-level convergence; M-021/M-022); ImputationDiD's per-target psi machinery makes seeded replay tractable (the panel-backed kit retains everything the psi precompute reads), TwoStageDiD's per-level GMM scores are function-locals and would need retention | `diff_diff/imputation_results.py`, `diff_diff/two_stage_results.py`, `diff_diff/aggregation.py` | 2(b) PR-3b | Mid | Low |

diff_diff/_staggered_triple_diff_engine.py

@@ -295,8 +295,20 @@ def _fit_staggered_core(
             if resolved_survey is not None and survey_metadata is not None:
                 resolved_survey_unit = precomputed.get("resolved_survey_unit")
                 if resolved_survey_unit is not None:
-                    unit_w = resolved_survey_unit.weights
-                    survey_metadata = compute_survey_metadata(resolved_survey_unit, unit_w)
+                    from diff_diff.survey import _extract_unit_survey_weights
+
+                    # Raw (pre-normalization) unit weights for metadata
+                    # provenance: compute_survey_metadata expects the ORIGINAL
+                    # scale (resolve() rescales pweights to mean 1, so the
+                    # resolved weights would misreport sum_weights/
+                    # weight_range; scale-invariant fields are unaffected
+                    # either way). Read from ``data``: ``df`` has first_treat
+                    # overwritten by this point.
+                    assert survey_design is not None
+                    raw_unit_w = _extract_unit_survey_weights(
+                        data, unit, survey_design, precomputed["all_units"]
+                    )
+                    survey_metadata = compute_survey_metadata(resolved_survey_unit, raw_unit_w)
 
             # Survey df for t-distribution critical values
             df_survey = precomputed.get("df_survey")

diff_diff/continuous_did.py

@@ -506,6 +506,18 @@ def fit(
             if col not in df.columns:
                 raise ValueError(f"Column '{col}' not found in data.")
 
+        # Snapshot the raw survey-weight column BEFORE any df mutation
+        # (never-treated dose zeroing, first_treat inf->0, to_numeric): a
+        # design whose weight column aliases a mutable role column (e.g.
+        # weights == dose) must still surface the user's ORIGINAL values in
+        # survey_metadata. Per-unit via groupby-first so the later
+        # dose-filter (which drops whole units) cannot desync alignment.
+        raw_unit_w_meta: Optional[pd.Series] = None
+        if survey_design is not None and survey_design.weights is not None:
+            # `is not None`, not truthiness: resolve() treats any non-None
+            # string — an empty-string column name included — as a column.
+            raw_unit_w_meta = data.groupby(unit)[survey_design.weights].first()
+
         # Covariate-path guards (conditional parallel trends).
         if cov_cols:
             if survey_design is not None:
@@ -770,11 +782,20 @@ def fit(
                 )
             lowest_dose = d_L
 
-        # Re-resolve survey design on filtered df if rows were dropped
-        # (survey arrays must align with df, not the original data)
+        # Re-resolve survey design on the filtered rows if rows were dropped
+        # (survey arrays must align with df, not the original data). Resolve
+        # from PRISTINE ``data`` rows, not the mutated working frame: the
+        # unfiltered path resolves from ``data``, and df's role-column
+        # coercions (never-treated dose zeroing, first_treat inf->0,
+        # to_numeric) must not leak into a design whose column aliases a
+        # mutated role column — resolving on df previously zero-weighted
+        # every never-treated unit when ``weights == dose``. The dose filter
+        # drops whole units and preserves row order, so the pristine
+        # unit-mask selection is row-for-row identical to df.
         if resolved_survey is not None and len(df) < len(data):
+            _kept_row_mask = data[unit].isin(set(df[unit].unique())).to_numpy()
             resolved_survey, survey_weights, survey_weight_type, survey_metadata = (
-                _resolve_survey_for_fit(survey_design, df, "analytical")
+                _resolve_survey_for_fit(survey_design, data[_kept_row_mask], "analytical")
             )
 
         # 2. Precompute structures
@@ -1103,7 +1124,22 @@ def fit(
                 if _unit_resolved is not None:
                     from diff_diff.survey import compute_survey_metadata
 
-                    raw_w_unit = _unit_resolved.weights
+                    # Raw (pre-normalization) unit weights for metadata
+                    # provenance: compute_survey_metadata expects the
+                    # ORIGINAL scale (resolve() rescales pweights to mean 1,
+                    # so _unit_resolved.weights would misreport sum_weights/
+                    # weight_range; scale-invariant fields are unaffected
+                    # either way). ``raw_unit_w_meta`` was snapshotted from
+                    # pristine ``data`` before the df mutations; reindexing
+                    # to ``all_units`` (the dose-filtered unit order that
+                    # built unit_resolved) keeps alignment — survey weights
+                    # are unit-constant (validated at resolve time).
+                    assert survey_design is not None
+                    raw_w_unit = (
+                        raw_unit_w_meta.reindex(precomp["all_units"]).to_numpy(dtype=np.float64)
+                        if raw_unit_w_meta is not None
+                        else np.ones(precomp["n_units"], dtype=np.float64)
+                    )
                     survey_metadata = compute_survey_metadata(_unit_resolved, raw_w_unit)
 
                 # Propagate replicate df override to survey_metadata for display

diff_diff/dml_did.py

@@ -1886,7 +1886,7 @@ def fit(
             assert survey_design is not None
             raw_obs_weights = (
                 data[survey_design.weights].values.astype(np.float64)
-                if survey_design.weights
+                if survey_design.weights is not None
                 else np.ones(len(data), dtype=np.float64)
             )
 

diff_diff/efficient_did.py

@@ -1397,6 +1397,22 @@ def _finalize_cell(g: Any, att_gt: float, eif_vals: np.ndarray) -> Dict[str, Any
             apply_bootstrap_group_overrides(group_effects, bootstrap_results, self.alpha)
 
         # ----- Build results -----
+        # Raw (pre-normalization) unit weights for the metadata recompute:
+        # compute_survey_metadata expects the ORIGINAL scale (resolve()
+        # rescales pweights to mean 1, so the resolved unit weights would
+        # misreport sum_weights/weight_range; scale-invariant fields are
+        # unaffected either way).
+        raw_unit_w_meta: Optional[np.ndarray] = None
+        if self._unit_resolved_survey is not None:
+            assert survey_design is not None
+            # `is not None`, not truthiness: resolve() treats any non-None
+            # string — an empty-string column name included — as a column.
+            raw_obs_w_meta = (
+                data[survey_design.weights].values.astype(np.float64)
+                if survey_design.weights is not None
+                else np.ones(len(data), dtype=np.float64)
+            )
+            raw_unit_w_meta = raw_obs_w_meta[self._unit_first_panel_row]
         self.results_ = EfficientDiDResults(
             group_time_effects=group_time_effects,
             overall_att=overall_att,
@@ -1452,7 +1468,7 @@ def _finalize_cell(g: Any, att_gt: float, eif_vals: np.ndarray) -> Dict[str, Any
             kernel_bandwidth=self.kernel_bandwidth,
             omega_ridge=self.omega_ridge,
             survey_metadata=(
-                self._recompute_unit_survey_metadata(survey_metadata)
+                self._recompute_unit_survey_metadata(survey_metadata, raw_unit_w_meta)
                 if survey_metadata is not None
                 else None
             ),
@@ -1483,14 +1499,22 @@ def _finalize_cell(g: Any, att_gt: float, eif_vals: np.ndarray) -> Dict[str, Any
         self.is_fitted_ = True
         return self.results_
 
-    def _recompute_unit_survey_metadata(self, panel_metadata):
-        """Recompute survey metadata from unit-level design if available."""
+    def _recompute_unit_survey_metadata(self, panel_metadata, raw_unit_weights=None):
+        """Recompute survey metadata from unit-level design if available.
+
+        ``raw_unit_weights`` carries the ORIGINAL-scale (pre-normalization)
+        unit weights and MUST be passed whenever ``_unit_resolved_survey``
+        is set — never fall back to ``_unit_resolved_survey.weights``, which
+        resolve() rescaled to mean 1 and would misreport sum_weights/
+        weight_range (scale-invariant fields are unaffected either way).
+        """
         if self._unit_resolved_survey is not None:
             from diff_diff.survey import compute_survey_metadata
 
+            assert raw_unit_weights is not None
             meta = compute_survey_metadata(
                 self._unit_resolved_survey,
-                self._unit_resolved_survey.weights,
+                raw_unit_weights,
             )
             # Propagate effective replicate df if available
             # (but not the df=0 sentinel — keep metadata as None for undefined df)

diff_diff/estimators.py

@@ -709,7 +709,7 @@ def fit(
             if resolved_survey.psu is not None and survey_metadata is not None:
                 raw_w = (
                     data[survey_design.weights].values.astype(np.float64)
-                    if survey_design.weights
+                    if survey_design.weights is not None
                     else np.ones(len(data), dtype=np.float64)
                 )
                 survey_metadata = compute_survey_metadata(resolved_survey, raw_w)
@@ -1889,7 +1889,7 @@ def _fit_event_study_core(
             if resolved_survey.psu is not None and survey_metadata is not None:
                 raw_w = (
                     data[survey_design.weights].values.astype(np.float64)
-                    if survey_design.weights
+                    if survey_design.weights is not None
                     else np.ones(len(data), dtype=np.float64)
                 )
                 survey_metadata = compute_survey_metadata(resolved_survey, raw_w)

diff_diff/imputation.py

@@ -619,7 +619,7 @@ def fit(
                 assert survey_design is not None
                 raw_w = (
                     data[survey_design.weights].values.astype(np.float64)
-                    if survey_design.weights
+                    if survey_design.weights is not None
                     else np.ones(len(data), dtype=np.float64)
                 )
                 survey_metadata = compute_survey_metadata(resolved_survey, raw_w)

diff_diff/lpdid.py

@@ -1501,7 +1501,7 @@ def fit(
                 resolved_panel = _inject_cluster_as_psu(resolved_panel, data[cluster].to_numpy())
             raw_weights = (
                 data[survey_design.weights].to_numpy(dtype=float)
-                if survey_design.weights
+                if survey_design.weights is not None
                 else np.ones(len(data), dtype=float)
             )
             survey_metadata = compute_survey_metadata(resolved_panel, raw_weights)

diff_diff/spillover.py

@@ -3035,7 +3035,10 @@ def fit(
                 # Matches the post-injection resolved_survey_fit length.
                 raw_w_for_meta = (
                     np.asarray(data[survey_design.weights].values, dtype=np.float64)
-                    if (survey_design is not None and getattr(survey_design, "weights", None))
+                    if (
+                        survey_design is not None
+                        and getattr(survey_design, "weights", None) is not None
+                    )
                     else np.ones(len(data), dtype=np.float64)
                 )
                 survey_metadata = _csm(resolved_survey_fit, raw_w_for_meta)