Skip to content

Commit e570060

Browse files
committed
docs(#192): design — Flyway output adapter for the shared migrate engine
ADR-0015 removed the Java meta:migrate --flyway mojo and designated a Flyway-prefix output adapter on the shared TS engine as its replacement; that adapter was never built, so a Spring-Boot + Exposed + Flyway consumer has no supported path from metadata to migrations. This is a regression, not a new feature. (The issue's other half -- the meta init scaffold misdirecting those adopters -- already shipped in PR #263, so this covers the adapter only.) Design: a third output adapter beside the homegrown and D1/Wrangler ones. The diff/emit engine is untouched -- it already produces {up, down} and an adapter only chooses the envelope, exactly ADR-0015 section 3's model. Decisions: - Integer scan-and-increment versioning (V<N+1>__), matching the removed mojo's prior art. Scanner must match V only (or the generated U__ files double-bump the counter) and take the leading integer of a dotted version (V10.5__ -> V11). - Undo emitted as U<N>__, Flyway's own documented convention. Verified against Flyway docs: undo is a paid edition feature and Community IGNORES U__ files rather than failing, so this is safe on Community and live on Teams. Rejected a .down/ sidecar -- Flyway filesystem locations can scan recursively. - Underscore slug sanitization (Flyway renders underscores as spaces), which deliberately differs from the D1 adapter's hyphens. - Format is orthogonal to dialect (a Flyway shop is still postgres/sqlite), so a --format flag plus a metaobjects.config.ts key, flag winning. - Detect-and-refuse for --apply / apply-pending / --rollback / --dialect d1: Flyway owns apply and history, so each refuses at generation time naming the Flyway command instead -- the #226/#241/#258 posture. Gated by the real-engine requirement every migrate change in this repo carries: emit -> apply to a real engine -> introspect -> re-diff must be EMPTY.
1 parent 87d775d commit e570060

1 file changed

Lines changed: 185 additions & 0 deletions

File tree

Lines changed: 185 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,185 @@
1+
# Design — Flyway output adapter for the shared migrate engine (#192)
2+
3+
**Date:** 2026-08-04
4+
**Issue:** [#192](https://github.com/metaobjectsdev/metaobjects/issues/192)
5+
**ADR of record:** [ADR-0015](../../../spec/decisions/ADR-0015-single-shared-migrate-engine.md) §3
6+
**Scope:** `migrate-ts` + `cli` (npm only). No metamodel change, no vocabulary change, no
7+
conformance-fixture change, no other port touched — schema is TS-owned (ADR-0015).
8+
9+
## 1. Problem
10+
11+
MetaObjects once shipped Flyway-named migration generation for JVM consumers: the Java
12+
`meta:migrate --flyway` Maven mojo (added in `4ca10dd2`, shipped in Java plugin 7.0.0). ADR-0015
13+
consolidated all migration onto the shared TS engine and **removed** that mojo (`77a5c46a`),
14+
designating a "Flyway-prefix output adapter" on the shared engine as its replacement.
15+
16+
That adapter was never built. `server/typescript/packages/migrate-ts/src` has zero `flyway`
17+
references. `meta migrate` emits only its own `<ts>-<slug>/up.sql` + `down.sql` layout plus a private
18+
ledger table — neither of which a Flyway `flyway_schema_history` boot can consume.
19+
20+
Net effect: a Spring-Boot + Exposed + **Flyway** consumer has **no supported path from metadata to
21+
migrations**, and must hand-author `V<n>__*.sql` to match codegen — the exact practice the project's
22+
own doctrine forbids ("pattern-derivable from metadata = codegen, never hand-code").
23+
24+
This is a **regression**, not a new feature: the capability existed, was removed on the promise of a
25+
replacement, and the replacement never landed.
26+
27+
### Already fixed — not in scope
28+
29+
The issue also asked that the `meta init` scaffold stop telling Flyway consumers to "apply schema
30+
only through `meta migrate`", advice impossible to follow on that stack. **That half shipped in PR
31+
#263** (schema-apply doctrine is now per-stack). This design covers the adapter only.
32+
33+
## 2. Architecture
34+
35+
A **third output adapter**, sibling to the two that already exist:
36+
37+
| Adapter | Layout | Selected by |
38+
|---|---|---|
39+
| homegrown (`write-migration.ts`) | `<ts>-<slug>/up.sql` + `down.sql` | default |
40+
| D1/Wrangler (`write-migration-d1.ts`) | `<seq>_<slug>.sql` + `.down/<same>` | `--dialect d1` |
41+
| **Flyway (new)** | `V<N>__<slug>.sql` + `U<N>__<slug>.sql` | `--format flyway` |
42+
43+
The diff/emit engine is **untouched**. It already produces `{ up, down }`; an adapter only decides
44+
the envelope. This is exactly ADR-0015 §3's model ("the engine generates the up+down SQL once;
45+
pluggable output-format adapters name/lay it out per target"), so the change introduces no new
46+
architectural concept — it fills a slot the ADR already specified.
47+
48+
**Format is orthogonal to dialect.** D1 gets its layout free from `--dialect d1` because D1 *is* a
49+
dialect. Flyway is not: a Flyway shop is still on postgres or sqlite. Hence a separate axis.
50+
51+
## 3. Decisions
52+
53+
### D1 — Integer scan-and-increment versioning
54+
55+
Scan the target dir for the highest `V<N>__`, emit `V<N+1>__`. This is what the removed Java mojo did
56+
(`4ca10dd2`: "scan `flywayDir` for the highest `V<N>__`, increment, emit `V<N+1>__<slug>.sql`") and
57+
what the issue's prior art specifies. It also reuses the D1 adapter's `nextSequence` shape.
58+
59+
Rejected: timestamp versions (`V20260804160212__`). Collision-free across branches without
60+
coordination, but diverges from the documented prior art for no benefit this project has evidence of
61+
needing.
62+
63+
Two scanner requirements that are easy to get wrong:
64+
65+
- **Match `V` only.** A regex of `^[VU](\d+)` would let the generated undo files double-bump the
66+
counter, so every run would skip a version.
67+
- **Take the leading integer of a dotted version.** Flyway permits `V1.1__`, `V2.0.1__`. An existing
68+
`V10.5__` must yield `V11__`, not `V1__`.
69+
70+
### D2 — Undo emitted as `U<N>__`, Flyway's own convention
71+
72+
The engine computes down SQL for free. Flyway's documented undo naming is the versioned migration's
73+
name with the leading `V` replaced by `U` (`V3__add_view.sql` → `U3__add_view.sql`).
74+
75+
**Verified externally:** undo is a commercial feature — Flyway **Community ignores `U__` files**
76+
rather than failing on them (sources in §8). So emitting them is safe on Community and becomes live
77+
the moment a shop is on Teams/Enterprise. ADR-0015 §2 already noted this Community/paid split.
78+
79+
Rejected: a `.down/` sidecar dir like D1's. Flyway filesystem locations can scan recursively, so a
80+
subdirectory inside the migrations dir risks being picked up as migrations. Rejected: dropping down
81+
SQL entirely — it discards something already computed, for no gain.
82+
83+
### D3 — Underscore slug sanitization (differs from D1)
84+
85+
Flyway renders the description with underscores as spaces, so `V4__add_program_view.sql` is
86+
idiomatic. The D1 adapter sanitizes to hyphens; the Flyway adapter must not copy that.
87+
88+
### D4 — `--format` flag plus a config key
89+
90+
`--format flyway|default` on `meta migrate`, also settable once in `metaobjects.config.ts`. Flag
91+
overrides config; default is `default`, so existing behavior is untouched.
92+
93+
Rationale: a JVM shop sets it once and forgets, while a one-off generation in another format stays
94+
possible. Flag-only would mean repeating it forever with a forgotten flag silently writing the wrong
95+
layout; config-only would remove the one-off escape.
96+
97+
### D5 — Output directory
98+
99+
`--out-dir` when given, else Flyway's convention `src/main/resources/db/migration`. This mirrors the
100+
existing per-adapter default-fallback pattern: the D1 path already falls back to wrangler's
101+
`migrations_dir` (or `migrations`) when `config.outDir` is still `MIGRATE_DEFAULT_OUT_DIR`.
102+
103+
## 4. Components
104+
105+
1. **`migrate-ts/src/write-migration-flyway.ts`** (new)
106+
`writeMigrationFlyway({ up, down }, { dir, slug }): Promise<{ upPath, downPath, version }>`.
107+
Creates `dir` if missing, computes the next version, writes both files with a trailing newline.
108+
Structurally mirrors `write-migration-d1.ts`.
109+
2. **`cli/src/lib/config.ts`** — add `format: "default" | "flyway"` to `ResolvedMigrateConfig`,
110+
resolved flag > `metaobjects.config.ts` > `"default"`.
111+
3. **`cli/src/commands/migrate.ts`** — dispatch on `config.format` at **both** `writeMigration` call
112+
sites (the live-DB/diff path and the offline `runOfflineGenerate` path), plus the refusal matrix
113+
in §6.
114+
4. **`codegen-ts/src/metaobjects-config.ts`** — typed `migrate.format` config key.
115+
116+
## 5. Data flow
117+
118+
Unchanged through load → diff → emit. `meta migrate baseline --from-db` seeds the snapshot exactly as
119+
today and is format-independent. Only the final write differs:
120+
121+
```
122+
metadata ──► expected schema ─┐
123+
├─► diff ──► emit {up, down} ──► adapter ──► files
124+
snapshot / --from-db ─────────┘ │
125+
default ────────────────┤
126+
d1 ────────────────┤
127+
flyway ────────────────┘
128+
```
129+
130+
## 6. Error handling — detect-and-refuse
131+
132+
Flyway owns apply and history. Generating a migration is ours; applying it is not. These refuse at
133+
generation time with a message naming the Flyway command instead, matching the posture established
134+
by #226/#241 (D1 FK cascade) and #258 (PK move):
135+
136+
| Combination | Refusal reason |
137+
|---|---|
138+
| `--format flyway --apply` | Applying behind Flyway desyncs `flyway_schema_history` → use `flyway migrate` |
139+
| `--format flyway apply-pending` | Same — replaying committed migrations is Flyway's job |
140+
| `--format flyway --rollback` | Our ledger does not exist on a Flyway-managed DB → `flyway undo` (Teams) or roll forward |
141+
| `--format flyway --dialect d1` | D1 has its own Wrangler layout and wrangler transport; the combination is meaningless |
142+
143+
Non-fatal behaviors:
144+
145+
- A missing or empty target dir starts at `V1`.
146+
- Non-migration files, repeatable migrations (`R__`), and `U__` files are ignored by the scanner.
147+
- Engine gaps the diff does not model (triggers, cross-column CHECKs, function-valued defaults, GIN
148+
array indexes) remain the author's to hand-add. The adapter changes nothing here: it degrades by
149+
emitting what the engine computed, never by silently dropping it.
150+
151+
## 7. Testing
152+
153+
**Unit — `write-migration-flyway`:** empty dir → `V1`; mixed non-migration files ignored; dotted
154+
versions (`V10.5__`) → `V11`; `U__` files do **not** bump the counter; slug sanitized to underscores;
155+
trailing newline on both files.
156+
157+
**CLI:** the four refusals in §6, each asserting exit code and message; `--format` precedence
158+
(flag > config > default); dir resolution (`--out-dir` vs the Flyway convention default).
159+
160+
**Real-engine gate (required).** Every migrate change in this repo carries it, because a green unit
161+
suite has historically missed this exact class: **emit → apply to a real engine → introspect →
162+
re-diff must be EMPTY.** For this adapter that means generating `V1__`/`V2__` against a real
163+
sqlite/postgres, applying them in order, and asserting the re-diff is empty.
164+
165+
**No-churn:** default-format and D1-format output stay byte-identical. Existing migrate tests must
166+
pass unchanged.
167+
168+
## 8. Sources
169+
170+
- Flyway migration naming and undo semantics —
171+
[Flyway migrations concepts](https://github.com/flyway/flywaydb.org/blob/gh-pages/documentation/concepts/migrations.md)
172+
- Undo is a paid edition feature; Community ignores `U__` —
173+
[Rolling Back Migrations with Flyway](https://www.baeldung.com/flyway-roll-back)
174+
- Prior art in-repo: `git show 4ca10dd2` (Flyway option added), `git show 77a5c46a` (mojo removed),
175+
`docs/superpowers/specs/2026-05-25-codegen-kotlin-design.md` §5.1 (superseded design)
176+
177+
## 9. Out of scope
178+
179+
- The other ADR-0015 adapters (two-file `.up.sql`/`.down.sql`, single-file-with-divider for
180+
dbmate/goose, Liquibase formatted-SQL). This design deliberately builds only the reference adapter;
181+
the others become mechanical once the `--format` axis exists.
182+
- Restoring any Java-side migrate goal. Schema stays TS-owned per ADR-0015; JVM consumers run the
183+
Node `meta` CLI for migration generation.
184+
- Closing the engine's modeling gaps (triggers, cross-column CHECKs, function-valued defaults, GIN
185+
array indexes). Orthogonal to the output envelope.

0 commit comments

Comments
 (0)