Skip to content

Commit c97e642

Browse files
dmealingclaude
andcommitted
docs(#241): design — D1 FK-cascade auto-fix (closes #226 residual gap)
Replace #226's refuse with a spike-validated cascade (temp-name FK rewrite + referrers-first DROP + parents-first RENAME under defer_foreign_keys) for acyclic graphs incl. self-ref; refuse only multi-table cycles. Thread the actual schema into emit so the affected-set/ordering uses actual∪expected FKs, which also closes #226's residual under-refuse gap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 0d2f468 commit c97e642

1 file changed

Lines changed: 132 additions & 0 deletions

File tree

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
# D1 FK-cascade auto-fix — rebuild referenced tables on remote D1 (#241)
2+
3+
- **Date:** 2026-07-28
4+
- **Status:** Design approved; ready for implementation plan
5+
- **Issue:** [#241](https://github.com/metaobjectsdev/metaobjects/issues/241) — auto-migrate referenced-table rebuilds via FK cascade (follow-up to [#226](https://github.com/metaobjectsdev/metaobjects/issues/226))
6+
- **Surface:** `@metaobjectsdev/migrate-ts` (D1 emitter) + a small `@metaobjectsdev/cli` call-site change. TS-only; no other-port impact.
7+
8+
## Problem
9+
10+
On remote Cloudflare D1, the SQLite table-rebuild recipe cannot rebuild a table that
11+
another table's foreign key references: D1 runs the migration inside an implicit
12+
transaction where `PRAGMA foreign_keys = OFF` is a no-op, so `DROP TABLE <referenced>`
13+
fails with `FOREIGN KEY constraint failed`. [#226](https://github.com/metaobjectsdev/metaobjects/issues/226)
14+
made the D1 emitter **refuse** that case at generation time (safe, but the user must
15+
hand-write the migration). This issue replaces the refusal with a **cascade** that
16+
generates a correct, appliable migration — and closes the residual under-refuse gap
17+
#226 documented.
18+
19+
## Prototype evidence
20+
21+
Throwaway libSQL prototypes (single connection, the emitted batch wrapped in one
22+
explicit transaction with `foreign_keys = ON` — a faithful model of remote D1's
23+
implicit transaction) validated the cascade **10/10** across single-parent,
24+
transitive (`g→c→p`), multi-child, and self-referential graphs: each commits with row
25+
data intact and foreign keys re-enforced. The same prototypes established that
26+
`PRAGMA defer_foreign_keys = ON` alone cannot rescue a parent rebuild, and that
27+
multi-table cycles have no reliable atomic sequence (they must be refused).
28+
29+
## The cascade algorithm
30+
31+
When a migration would rebuild an FK-referenced table (and the FK graph over the
32+
affected set is acyclic — self-loops excepted), the D1 emitter emits one pass:
33+
34+
```sql
35+
PRAGMA defer_foreign_keys = ON; -- belt for self-referential / lazy checks
36+
-- for each table t in the affected set A:
37+
CREATE TABLE __f_<t> ( <t's final schema> ); -- FK cols reference __f_<ref> when ref ∈ A, else <ref>'s real name
38+
-- for each t in A:
39+
INSERT INTO __f_<t> ( <carry cols> ) SELECT <mapped cols> FROM <t>;
40+
-- DROP old tables, REFERRERS-FIRST (reverse-topological):
41+
DROP TABLE <t>;
42+
-- RENAME temps, PARENTS-FIRST (topological) — RENAME rewrites __f_ FK refs to final names:
43+
ALTER TABLE __f_<t> RENAME TO <t>;
44+
```
45+
46+
The two ordering rules are load-bearing: referrers-first `DROP` avoids dropping a
47+
still-referenced table; parents-first `RENAME` lets SQLite rewrite each temp-name FK to
48+
its final target. Forward-references in the `CREATE` statements are fine — SQLite
49+
resolves FK *target* tables lazily.
50+
51+
The D1 safety pass strips the (now unnecessary) explicit transaction control, relying
52+
on D1's implicit transaction for atomicity; the useless `PRAGMA foreign_keys = OFF/ON`
53+
bracket is not emitted on the cascade path.
54+
55+
## Affected set, ordering, cycles
56+
57+
- **Affected set `A` = (tables the diff rebuilds) ∪ (all tables that transitively
58+
reference a table in `A`)**, where "reference" is the **union of the actual and
59+
expected foreign-key graphs**. Using the union (not expected-only) is what closes
60+
#226's residual gap: a migration that rebuilds a parent *and* drops a child's FK to
61+
it in the same run — the child references the parent in the *actual* schema, so it
62+
must be rebuilt (and dropped) before the parent, which only the actual graph reveals.
63+
- A table pulled into `A` only as a referrer (not itself changed) is rebuilt to its
64+
**expected descriptor** (identical to its current schema, so a re-diff converges),
65+
including its columns, indexes, and foreign keys.
66+
- **Ordering** is a topological sort of the union FK graph restricted to `A`.
67+
- **Multi-table cycles** (`A→B→…→A`, length ≥ 2) have no reliable atomic FKs-on
68+
sequence and are **refused** (the topological sort detects them). Self-loops
69+
(self-referential FKs) are excluded from the cycle check — the temp-name mechanism
70+
handles them. The existing `D1ReferencedTableRebuildError` is reused/renamed for the
71+
cycle-refusal message (naming the cycle and the hand-write workaround).
72+
73+
## Threading the actual schema
74+
75+
`EmitOptions` gains `actualSchema?: SchemaSnapshot`. Both `meta migrate` emit call
76+
sites already hold it: the live-introspection path passes `actual` (the introspected
77+
snapshot given to `diff`), and the offline path passes the prior committed `snapshot`.
78+
`renderD1` uses it (unioned with `expectedSchema`) to build the FK graph. When absent
79+
(e.g. a caller that doesn't supply it), the emitter falls back to the expected graph
80+
and **refuses** any referenced rebuild it cannot prove safe — never emits an unproven
81+
cascade.
82+
83+
## Where it lives
84+
85+
- New `src/emit/d1-cascade.ts`: the reverse FK-graph builder, topological sort + cycle
86+
detection, and the cascade emitter. It reuses `src/emit/sqlite.ts` primitives
87+
(`renderCreateTable`, the column carry/remap logic from `renderRecreate`,
88+
`renderCreateIndex`), extracted read-only without changing SQLite behavior.
89+
- `src/emit/d1.ts` (`renderD1`): when `recreatedTables` contains a referenced table,
90+
dispatch to the cascade for an acyclic graph, or throw the cycle-refusal for a cycle.
91+
The no-referenced-rebuild common path stays exactly as today — **byte-identical**.
92+
- `src/emit/index.ts` (`EmitOptions` + `emit`) and the two `cli` emit call sites: add
93+
and pass `actualSchema`.
94+
- No change to `emit/postgres.ts` or the local-SQLite path.
95+
96+
## Down migrations
97+
98+
As today, the recreate/cascade down migration is best-effort — the emitter emits a
99+
`-- WARNING` block (data-destructive reversals cannot be auto-generated). Unchanged
100+
from the existing SQLite recreate behavior.
101+
102+
## Testing
103+
104+
Extend the libSQL real-engine gate (`test/integration/`, one-transaction model = remote
105+
D1), each scenario asserting: the cascade applies cleanly, row data is intact, foreign
106+
keys are re-enforced afterward, and a re-diff is **EMPTY** (the migrate-engine
107+
doctrine):
108+
109+
- single parent + populated child;
110+
- transitive chain `g→c→p`;
111+
- multiple children of one parent;
112+
- self-referential table;
113+
- **the #226 residual gap**: one migration that rebuilds a parent AND drops a child's
114+
FK to it — applies cleanly (this is the case that regresses if the actual graph
115+
isn't used);
116+
- **multi-table cycle** `A↔B` → refused at generation time (unit-level);
117+
- a no-referenced-rebuild migration → D1 output byte-identical to today.
118+
119+
Plus unit tests for the FK-graph builder, topological ordering, and cycle detection.
120+
121+
## Non-goals
122+
123+
- Multi-table foreign-key cycles on D1 (refused, documented).
124+
- Any change to the local SQLite or Postgres emitters.
125+
- Changing `applyPending` / replay semantics.
126+
127+
## Docs
128+
129+
Remove #226's "Known limitation of the current refusal" note from
130+
`docs/features/migrations-and-drift.md` (the gap is closed) and update the D1
131+
limitation subsection to describe the cascade (with cycles as the remaining
132+
hand-write case). CHANGELOG entry. Close #241.

0 commit comments

Comments
 (0)