Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,58 @@
# Changelog

## Unreleased

### LP-DiD: non-absorbing rules (**estimates change** on panels with gaps, missing treatment values, or units treated from their first row)
- Switches are counted between consecutive observed treatment values. They were summed
from the observed first differences, as `lpdid` does, so a switch in an absent period or
behind a missing value was not seen: a unit observed untreated and then treated could
serve as a not-yet-treated control (`control = :notyet`; 13 of the 260 rows at h = 0 on
the unbalanced test panel), its next switch counted as the first (`firsttreat`), and an
absorbing treatment that switched off inside a gap was not refused.
- A control is a row with ΔD = 0. The first-treatment estimator (`firsttreat` with
`control = :notyet` or `:nevertreated`) kept the rows where a unit treated from its first
row switches off, with a regressor of −1, as `lpdid` does (on a noise-free panel with a
true effect of 2 the estimate was 1.05).
- `compat = :lpdid` reproduces the command where it differs from these rules and from the
reweighting of 0.2.1; `reweight = :lpdid` is now short for `reweight = true,
compat = :lpdid`. On balanced panels without missing values whose units all start
untreated, nothing changes.
- `ldiff(lags(y, 1))` as an LP-DiD response is read at its own periods (it went through
the row of t + h, the nested-shift bug that 0.2.1 fixed elsewhere).
- When no horizon has both treated rows and clean controls, the error says so (it
suggested dropping the time effects).

### Other fixes (black-box review of 2026-11-18)
- `CDOLS()`: the checks crashed (`BoundsError`) with a path term `leads(z)` written before
the shock, and rejected the path of the outcome with an unrelated message.
- A lead of a path term cannot be the shock, by name or as the first regressor: a clear
error instead of "No value arguments present".
- `firststage(r; vcov = Homoskedastic())` returned NaN: it is now the classical variance of
the first stage, whose F equals the Cragg–Donald F with one endogenous regressor.
- `firststage` with a variance that is exactly zero (an aggregate instrument times a
time-invariant exposure, clustered by unit, on a balanced panel: the scores sum to zero
within every unit) reported an F of order 1e33; it is NaN, with the warning.
- `synthetic` on a weighted fit says that weighted fits store no synthetic series.
- Docs: with `nonabsorbing = L`, the `pmd` base needs k + L − 1 clean periods before t (the
page said the base window only); `nonabsorbing = 0` is 1.
- Tests: the runner only takes the items of this repository's `test/` directory.

### The three behaviours that 0.2.1 reported and left as they were
- A shock that lies in the span of the other regressors makes the horizon `:singular` (an
error when it happens at every horizon), whatever the order of the formula. Before,
the pivoting decided which column of the collinear set was omitted: `x + x2` with
`x2 = 2x` gave the shock `x` a coefficient, `x2 + x` an error. A collinear set without
the shock still loses one column, with a note; a lag added by `LagAugmented` still gives
way to the formula's columns.
- `Clustered(:unit, :time; ssc = :mss)` with a negative factor (n ≤ N + K) gives a `NaN`
variance, with a warning. Before, the default PSD repair clipped the sign-flipped
variance into a finite or zero standard error. `psd = :raw` keeps the clipping, which is
the authors' order of operations, and the warning says that the numbers have no meaning.
- With `AlternatingProjections(tol = …)` looser than the default, the columns that are
mostly absorbed are transformed again at the default tolerance before the
absorbed-column screen. On a slowly mixing panel an aggregate regressor with time
effects could otherwise get a coefficient (1.95 on the test panel at `tol = 1e-6`).

## 0.2.1 (2026-11-18): LP-DiD, negative horizons, lead paths

### LP-DiD (Dube, Girardi, Jordà & Taylor 2025)
Expand Down
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ One module and no submodules. Every struct lives in `src/types.jl`. Include orde
- LP-DiD (`lpdid.jl`): the formula is `ldiff(y) ~ firstdiff(D) + fe(time)`; `_did_resolve` builds the treatment histories once, `_cleanflag(s, pidx, h)` the sample of horizon h (rules evaluated on all the data, a condition on an unobserved value is false), `_response` the long difference from the base (`pmd`). `reweight` is a weighted fit (`_rwweights`; `:lpdid` adds `_rwextra` rows to a period's controls) or, with covariates or further FE, regression adjustment (`_fit_ra`: only the treatment coefficient, bread 1, the influence function as the score, no singleton dropping). `HorizonFit.did` (`DiDInfo`) holds treated and total row counts by group, which `averageeffect` and `pretrendtest` combine with the score sums across horizons. Written from the paper; never from the GPL ado.
- `pooled = true` stores one fit at the far end of the window (`r.horizons` is that single horizon, `r.pooled` the window).
- Re-absorb every FE on each horizon's sample and inside each SPJ half. Screen each half for absorbed columns as `_ols` does (`_absorbed_cols`): rescaling a round-off column turns it into an arbitrary coefficient. A failing half makes the horizon `:spj_half_singular`, and with no `:ok` horizon `panellp` throws.
- A shock absorbed by the FE (e.g. an aggregate shock with `fe(time)`) is never a zero coefficient. It is an error when it happens at every horizon. At some horizons only, those fits are `:singular` with NaN coefficients.
- A shock absorbed by the FE (e.g. an aggregate shock with `fe(time)`) is never a zero coefficient. It is an error when it happens at every horizon. At some horizons only, those fits are `:singular` with NaN coefficients. The same holds for a shock in the span of the other regressors (`_shockspanned`): `:singular` whatever the formula's order; only lags added by `LagAugmented` give way to it. With a `tol` looser than the default, `_recheck_absorbed!` re-absorbs nearly absorbed columns at the default tolerance before the screen.
- Problems at some horizons (empty, singular, omitted regressors, non-convergence) are reported once per `panellp` call (`_fitnotes`) and listed by `show(r)`. Never use `maxlog`, which silences later calls for the rest of the session.
- Undefined variances (a single cluster or period, no residual df) are NaN with a warning, never an exception. Negative variances give NaN standard errors.
- Estimator-specific specification checks run before estimation (`_validate(estimator, design, formula)` in `estimators.jl`). `CDOLS` enforces Ugarte-Ruiz's rules, structurally (`_collags` dates every column; the shock's lag is found numerically by `_haslag`): a long-difference response, each shock dated right after the base period with its first lag among the regressors, outcomes only lagged beyond the response's start and differenced when the response differences them, and no FE nested in the units (also per horizon sample, in `_fit_horizon(::CDOLS)`). `validate = false` skips them.
Expand All @@ -115,7 +115,7 @@ One module and no submodules. Every struct lives in `src/types.jl`. Include orde
- `CappedLags`: the A&S code's lag rule;
- `TLAHR(weights = :raw)`: the A&S code's synthetic regression (raw exposure, unweighted periods);
- `timelags = :position`: xtlp/xtscc DK;
- `LPDiD(reweight = :lpdid)`: lpdid 1.1.0's count of a period's controls in the reweighted regression;
- `LPDiD(compat = :lpdid)`: lpdid 1.1.0's rules where the defaults differ (a period's controls in the reweighted regression, switches hidden in gaps, switch-off rows under the first-treatment estimator); `reweight = :lpdid` is short for `reweight = true, compat = :lpdid`;
- `Homoskedastic(ssc = :none)`: ivreg2 without options (Ilzetzki 2024).

The map from each reference command to the matching Julia options is in `crosscheck/README.md`.
Expand Down
15 changes: 11 additions & 4 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -264,10 +264,17 @@ BSD-3 code, CC BY 4.0 data, local only).
warning
- [x] A NaN (or `nothing`) id was a unit; −0.0 and 0.0 were two levels; `maxiter` was
exceeded by one sweep; stale documentation
- [ ] Left as they are (judgment calls, see the CHANGELOG): the order dependence of which
collinear column is dropped; `ssc = :mss` with a negative factor under a PSD repair; the
absorbed-column screen under a loose `tol`
- [ ] The fourth lens (black-box use and documentation) was stopped before it reported
- [x] The three judgment calls, left as they were in 0.2.1 and changed afterwards (see the
CHANGELOG, Unreleased): a shock in a collinear set is `:singular` whatever the formula's
order (`_shockspanned`); `ssc = :mss` with a negative factor gives a NaN variance
(clipped only with `psd = :raw`); the absorbed-column screen re-absorbs nearly absorbed
columns at the default tolerance when `tol` is looser (`_recheck_absorbed!`)
- [x] The fourth lens (black-box use and documentation) ran on 2026-11-18 (fuzzing against
dummy-variable oracles, invariances, every docs code block, about 150 error-path probes):
eight findings, fixed (see the CHANGELOG, Unreleased). The two that matter are in LP-DiD's
non-absorbing rules and were inherited from `lpdid`: switches hidden in gaps, and
switch-off rows under the first-treatment estimator; the command's behaviour is now
`compat = :lpdid`

- [x] Phase 7 gate (2026-11-18): `Pkg.test()` on Julia 1.13.3 and 1.10.13 (45,623 tests, the
local MSS and Ilzetzki items included), format, fixtures, docs build, the p7 Stata
Expand Down
11 changes: 6 additions & 5 deletions crosscheck/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,15 +240,18 @@ max(df_a − its levels, 1).
`fx_lpdid` panels: `crosscheck/stata/p7_lpdid.do` → `test/fixtures/stata/p7_lpdid.csv`
(1424 rows: 47 option sets × 2 panels, with the event-study, pooled, aggregate and
pre-trend tables), compared in `test/test_crosscheck_p7.jl`. The formula is
`ldiff(y) ~ firstdiff(d) + fe(t)` with `estimator = LPDiD(…)`, `horizon = -5:8`.
`ldiff(y) ~ firstdiff(d) + fe(t)` with `estimator = LPDiD(…, compat = :lpdid)`,
`horizon = -5:8`. `compat = :lpdid` selects the command's rules where the package's
defaults differ (`docs/src/audit/lpdid.md`): it matters on `fx_lpdid_unbal` only; on
`fx_lpdid` the test also checks that the defaults give the same numbers.

| `lpdid y, unit(id) time(t) treat(d) pre_window(5) post_window(8) …` | `panellp` | `vcov` spec | Agreement |
|---|---|---|---|
| (baseline); `cluster(st)`; `[pweight = w]` | `LPDiD()`; `clusters = (:st,)`; `weights = :w` | `Clustered(:unit)`; `Clustered(:st)` | b 5e-15, se 5e-11 |
| `ylags(2) controls(x)`, `dylags(2)`, `controls(x xb)`, `absorb(reg)` | `lags(y, 1:2) + x`, `lags(firstdiff(y), 1:2)`, `x + xb`, `fe(reg)` | `Clustered(:unit)` | as above |
| `pmd(max)`, `pmd(3)`; `nevertreated`; `nocomp` | `LPDiD(pmd = :max \| 3)`; `control = :nevertreated`; `sample = :common` | as above | as above |
| `pooled`, `post_pooled(2 6) pre_pooled(3 5)` | `pooled = true` with `horizon = 0:8 \| -5:-2`, `2:6 \| -5:-3` | as above | 1e-15 |
| `rw` (also with weights, `nocomp`, `pmd(max) nevertreated`, `pooled`) | `LPDiD(reweight = :lpdid)` | as above | 5e-15 |
| `rw` (also with weights, `nocomp`, `pmd(max) nevertreated`, `pooled`) | `LPDiD(reweight = true, compat = :lpdid)` | as above | 5e-15 |
| a hand-written `reghdfe [pweight]` with the factor from the regression rows (`d_rw_hand`) | `LPDiD(reweight = true)` | as above | 4e-15 |
| `rw` with `controls(x)`, `ylags(1)`, `dylags(2)`, `absorb(reg)`, `[pweight = w]`, `cluster(st)`, `pooled` (regression adjustment through `listreg`) | `LPDiD(reweight = true)` with the same terms | `Clustered(…; ssc = :none)`, normal bands | 2e-15 |
| `nonabsorbing(5 \| 3)`, `(5, notyet)`, `(5, firsttreat)`, `(, firsttreat notyet)`, `(5, firsttreat notyet) pmd(2)`, with `nevertreated`, `untreated_before`, `pmd(2 \| max)`, `rw`, `nocomp`, `rw dylags(1)`; treatment `d2` | `LPDiD(nonabsorbing = L \| true, control = :notyet \| :nevertreated, firsttreat, untreated_before, …)` | `Clustered(:unit)` | 1e-14 |
Expand All @@ -259,9 +262,7 @@ pre-trend tables), compared in `test/test_crosscheck_p7.jl`. The formula is
Row `pre{k}` of `e(results)` is h = −k and `tau{h}` is h. `pre1` is h = −1, which `lpdid`
reports as (0, missing) and `panellp` as a zero coefficient with a zero standard error. A
pre-treatment difference spanned by the lagged outcomes is zero by construction in both.
`lpdid`'s `rw` counts the controls of a period differently from the regression's rows on
unbalanced panels (`docs/src/audit/lpdid.md`): `reweight = :lpdid` reproduces it. The wild
bootstrap is not compared (not implemented).
The wild bootstrap is not compared (not implemented).

**General additions**: `crosscheck/stata/p7_general.do` → `p7_general.csv` (112 rows).

Expand Down
39 changes: 28 additions & 11 deletions docs/src/audit/lpdid.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,9 @@ The [LP-DiD page](../methods/lpdid.md) maps every option to `panellp`.

## Agreement

Event-study coefficients, standard errors, confidence bounds and numbers of observations
agree to round-off (coefficients 1e-14, standard errors 5e-11) for every option set:
With `compat = :lpdid` on the unbalanced panel (on the balanced one the default rules
are the command's), event-study coefficients, standard errors, confidence bounds and
numbers of observations agree to round-off (coefficients 1e-14, standard errors 5e-11) for every option set:
covariates, lagged outcomes, extra fixed effects, weights, clusters, `pmd`,
`nevertreated`, `nocomp`, the non-absorbing and one-off rules with `notyet`, `firsttreat`
and `untreated_before`, `rw` by weighted regression and by regression adjustment; so do
Expand All @@ -36,29 +37,45 @@ the pooled estimates, `aggregate_average` and `pretrend_test`.
equals that average to round-off (a test), and equals a hand-written `reghdfe` with the
weights from the regression rows (`d_rw_hand` in the do-file). On balanced panels, and
with regression adjustment (which uses the regression rows), there is no difference.
`LPDiD(reweight = :lpdid)` reproduces the command.
2. **An absorbing treatment is not checked.** Without `nonabsorbing()`, a treatment that
`compat = :lpdid` reproduces the command.
2. **A switch hidden in a gap is not seen.** The not-yet-treated and first-treatment rules
count switches by summing the observed ``\lvert\Delta D\rvert``, and ``\Delta D`` is
undefined next to an absent period or a missing treatment value. A unit observed
untreated, then (after a gap) treated, has no counted switch: under `notyet` its
treated rows serve as not-yet-treated controls, and under `firsttreat` its next switch
counts as the first. On `fx_lpdid_unbal`, 13 of the 260 rows of
`nonabsorbing(5, notyet)` at ``h = 0`` are such controls. (The command prints a warning
that gaps can hide switches.) `LPDiD` counts changes between consecutive observed
values; `compat = :lpdid` reproduces the command.
3. **The first-treatment estimator keeps switch-off rows.** With `firsttreat` and `notyet`
or `nevertreated`, a unit that is treated from its first row and later exits has one
observed switch, so its exit row passes the first-treatment rule, and, not being a
control (``\Delta D \ne 0``), escapes the control-group restriction: it enters the
regression with a regressor of −1. On `fx_lpdid_unbal` there are 8 such rows at
``h = 0``; on a small noise-free panel with a true effect of 2 the estimate is 1.05.
`LPDiD` leaves those rows out; `compat = :lpdid` reproduces the command.
4. **An absorbing treatment is not checked.** Without `nonabsorbing()`, a treatment that
switches off stays in the sample with a regressor of −1. `LPDiD()` refuses it and
points to `nonabsorbing`.
3. **`nonabsorbing(0)` is `nonabsorbing(1)`.** The same here (the rule needs
5. **`nonabsorbing(0)` is `nonabsorbing(1)`.** The same here (the rule needs
``\Delta D_{t-1}``), stated in the table of clean controls.
4. **`nocomp` with a one-sided window** still requires the pre-treatment horizon 2 (and,
6. **`nocomp` with a one-sided window** still requires the pre-treatment horizon 2 (and,
with only a pre window, horizon 0), outcomes included. `sample = :common` uses the
requested horizons: pass `horizon = -2:H` for the command's sample.
5. **The pooled regressions under `nocomp`** require the clean-control flags of both
7. **The pooled regressions under `nocomp`** require the clean-control flags of both
pooled windows. `pooled = true` with `sample = :common` uses the horizons of its own
window.
6. **Mixed conventions in the output.** The event-study standard errors carry `reghdfe`'s
8. **Mixed conventions in the output.** The event-study standard errors carry `reghdfe`'s
small-sample factor and t(G − 1) critical values, the regression-adjustment ones no
factor and normal critical values, and `aggregate_average` no factor with t(M − 1).
`averageeffect` and `pretrendtest` follow the command; for the event study choose with
`Clustered(…; ssc = …)`.
7. **The pre-treatment row `pre1` is always (0, missing)**, also under `pmd()`, where the
9. **The pre-treatment row `pre1` is always (0, missing)**, also under `pmd()`, where the
difference between ``y_{t-1}`` and the pre-treatment mean is not zero by construction.
`panellp` estimates ``h = -1`` like any other horizon then.
8. **The reported `t` and `p` are rounded** (2 and 4 decimals); compare coefficients,
10. **The reported `t` and `p` are rounded** (2 and 4 decimals); compare coefficients,
standard errors and bounds.
9. A pre-treatment difference that the lagged outcomes span (`ylags(k)`: horizons up to
11. A pre-treatment difference that the lagged outcomes span (`ylags(k)`: horizons up to
``-k``; `dylags(k)`: up to ``-(k+1)``) is zero by construction: `reghdfe` returns
round-off, regression adjustment (0, 0). Here the coefficient is round-off as well, and
`pretrendtest` leaves those horizons out, as the command does.
Expand Down
4 changes: 3 additions & 1 deletion docs/src/deviations.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,9 @@ reference. The audit pages document each reference in detail.

| topic | `lpdid` | `panellp` (`LPDiD`) | switch |
|---|---|---|---|
| reweighting factor of the weighted regression (`rw`) | the treated share of the period among clean rows with the outcome, including rows whose ``\Delta D`` is undefined and that the regression drops | the treated share among the rows of the regression (the equally weighted average of the cohorts' comparisons) | `reweight = :lpdid` |
| reweighting factor of the weighted regression (`rw`) | the treated share of the period among clean rows with the outcome, including rows whose ``\Delta D`` is undefined and that the regression drops | the treated share among the rows of the regression (the equally weighted average of the cohorts' comparisons) | `compat = :lpdid` |
| switches, for `notyet`, `firsttreat` and the check of an absorbing treatment | summed from the observed first differences: a switch in a gap or behind a missing value is not seen | changes between consecutive observed treatment values | `compat = :lpdid` |
| first treatment with `notyet` or `nevertreated` | keeps the switch-off rows of a unit treated from its first row, with a regressor of −1 | a control has ``\Delta D = 0``: those rows are left out | `compat = :lpdid` |
| a treatment that switches off, without `nonabsorbing()` | kept, with a regressor of −1 | an error | `nonabsorbing = L` |
| `nocomp` with a one-sided window | also requires the pre-treatment horizon 2 (or horizon 0) | the requested horizons (`sample = :common`) | `horizon = -2:H` |
| pooled regressions under `nocomp` | the flags of both pooled windows | the horizons of the window | — |
Expand Down
Loading
Loading