|
| 1 | +# Design — Flyway output adapter for the shared migrate engine (#192) |
| 2 | + |
| 3 | +**Date:** 2026-08-04 |
| 4 | +**Issue:** [#192](https://github.com/metaobjectsdev/metaobjects/issues/192) |
| 5 | +**ADR of record:** [ADR-0015](../../../spec/decisions/ADR-0015-single-shared-migrate-engine.md) §3 |
| 6 | +**Scope:** `migrate-ts` + `cli` (npm only). No metamodel change, no vocabulary change, no |
| 7 | +conformance-fixture change, no other port touched — schema is TS-owned (ADR-0015). |
| 8 | + |
| 9 | +## 1. Problem |
| 10 | + |
| 11 | +MetaObjects once shipped Flyway-named migration generation for JVM consumers: the Java |
| 12 | +`meta:migrate --flyway` Maven mojo (added in `4ca10dd2`, shipped in Java plugin 7.0.0). ADR-0015 |
| 13 | +consolidated all migration onto the shared TS engine and **removed** that mojo (`77a5c46a`), |
| 14 | +designating a "Flyway-prefix output adapter" on the shared engine as its replacement. |
| 15 | + |
| 16 | +That adapter was never built. `server/typescript/packages/migrate-ts/src` has zero `flyway` |
| 17 | +references. `meta migrate` emits only its own `<ts>-<slug>/up.sql` + `down.sql` layout plus a private |
| 18 | +ledger table — neither of which a Flyway `flyway_schema_history` boot can consume. |
| 19 | + |
| 20 | +Net effect: a Spring-Boot + Exposed + **Flyway** consumer has **no supported path from metadata to |
| 21 | +migrations**, and must hand-author `V<n>__*.sql` to match codegen — the exact practice the project's |
| 22 | +own doctrine forbids ("pattern-derivable from metadata = codegen, never hand-code"). |
| 23 | + |
| 24 | +This is a **regression**, not a new feature: the capability existed, was removed on the promise of a |
| 25 | +replacement, and the replacement never landed. |
| 26 | + |
| 27 | +### Already fixed — not in scope |
| 28 | + |
| 29 | +The issue also asked that the `meta init` scaffold stop telling Flyway consumers to "apply schema |
| 30 | +only through `meta migrate`", advice impossible to follow on that stack. **That half shipped in PR |
| 31 | +#263** (schema-apply doctrine is now per-stack). This design covers the adapter only. |
| 32 | + |
| 33 | +## 2. Architecture |
| 34 | + |
| 35 | +A **third output adapter**, sibling to the two that already exist: |
| 36 | + |
| 37 | +| Adapter | Layout | Selected by | |
| 38 | +|---|---|---| |
| 39 | +| homegrown (`write-migration.ts`) | `<ts>-<slug>/up.sql` + `down.sql` | default | |
| 40 | +| D1/Wrangler (`write-migration-d1.ts`) | `<seq>_<slug>.sql` + `.down/<same>` | `--dialect d1` | |
| 41 | +| **Flyway (new)** | `V<N>__<slug>.sql` + `U<N>__<slug>.sql` | `--format flyway` | |
| 42 | + |
| 43 | +The diff/emit engine is **untouched**. It already produces `{ up, down }`; an adapter only decides |
| 44 | +the envelope. This is exactly ADR-0015 §3's model ("the engine generates the up+down SQL once; |
| 45 | +pluggable output-format adapters name/lay it out per target"), so the change introduces no new |
| 46 | +architectural concept — it fills a slot the ADR already specified. |
| 47 | + |
| 48 | +**Format is orthogonal to dialect.** D1 gets its layout free from `--dialect d1` because D1 *is* a |
| 49 | +dialect. Flyway is not: a Flyway shop is still on postgres or sqlite. Hence a separate axis. |
| 50 | + |
| 51 | +## 3. Decisions |
| 52 | + |
| 53 | +### D1 — Integer scan-and-increment versioning |
| 54 | + |
| 55 | +Scan the target dir for the highest `V<N>__`, emit `V<N+1>__`. This is what the removed Java mojo did |
| 56 | +(`4ca10dd2`: "scan `flywayDir` for the highest `V<N>__`, increment, emit `V<N+1>__<slug>.sql`") and |
| 57 | +what the issue's prior art specifies. It also reuses the D1 adapter's `nextSequence` shape. |
| 58 | + |
| 59 | +Rejected: timestamp versions (`V20260804160212__`). Collision-free across branches without |
| 60 | +coordination, but diverges from the documented prior art for no benefit this project has evidence of |
| 61 | +needing. |
| 62 | + |
| 63 | +Two scanner requirements that are easy to get wrong: |
| 64 | + |
| 65 | +- **Match `V` only.** A regex of `^[VU](\d+)` would let the generated undo files double-bump the |
| 66 | + counter, so every run would skip a version. |
| 67 | +- **Take the leading integer of a dotted version.** Flyway permits `V1.1__`, `V2.0.1__`. An existing |
| 68 | + `V10.5__` must yield `V11__`, not `V1__`. |
| 69 | + |
| 70 | +### D2 — Undo emitted as `U<N>__`, Flyway's own convention |
| 71 | + |
| 72 | +The engine computes down SQL for free. Flyway's documented undo naming is the versioned migration's |
| 73 | +name with the leading `V` replaced by `U` (`V3__add_view.sql` → `U3__add_view.sql`). |
| 74 | + |
| 75 | +**Verified externally:** undo is a commercial feature — Flyway **Community ignores `U__` files** |
| 76 | +rather than failing on them (sources in §8). So emitting them is safe on Community and becomes live |
| 77 | +the moment a shop is on Teams/Enterprise. ADR-0015 §2 already noted this Community/paid split. |
| 78 | + |
| 79 | +Rejected: a `.down/` sidecar dir like D1's. Flyway filesystem locations can scan recursively, so a |
| 80 | +subdirectory inside the migrations dir risks being picked up as migrations. Rejected: dropping down |
| 81 | +SQL entirely — it discards something already computed, for no gain. |
| 82 | + |
| 83 | +### D3 — Underscore slug sanitization (differs from D1) |
| 84 | + |
| 85 | +Flyway renders the description with underscores as spaces, so `V4__add_program_view.sql` is |
| 86 | +idiomatic. The D1 adapter sanitizes to hyphens; the Flyway adapter must not copy that. |
| 87 | + |
| 88 | +### D4 — `--format` flag plus a config key |
| 89 | + |
| 90 | +`--format flyway|default` on `meta migrate`, also settable once in `metaobjects.config.ts`. Flag |
| 91 | +overrides config; default is `default`, so existing behavior is untouched. |
| 92 | + |
| 93 | +Rationale: a JVM shop sets it once and forgets, while a one-off generation in another format stays |
| 94 | +possible. Flag-only would mean repeating it forever with a forgotten flag silently writing the wrong |
| 95 | +layout; config-only would remove the one-off escape. |
| 96 | + |
| 97 | +### D5 — Output directory |
| 98 | + |
| 99 | +`--out-dir` when given, else Flyway's convention `src/main/resources/db/migration`. This mirrors the |
| 100 | +existing per-adapter default-fallback pattern: the D1 path already falls back to wrangler's |
| 101 | +`migrations_dir` (or `migrations`) when `config.outDir` is still `MIGRATE_DEFAULT_OUT_DIR`. |
| 102 | + |
| 103 | +## 4. Components |
| 104 | + |
| 105 | +1. **`migrate-ts/src/write-migration-flyway.ts`** (new) |
| 106 | + `writeMigrationFlyway({ up, down }, { dir, slug }): Promise<{ upPath, downPath, version }>`. |
| 107 | + Creates `dir` if missing, computes the next version, writes both files with a trailing newline. |
| 108 | + Structurally mirrors `write-migration-d1.ts`. |
| 109 | +2. **`cli/src/lib/config.ts`** — add `format: "default" | "flyway"` to `ResolvedMigrateConfig`, |
| 110 | + resolved flag > `metaobjects.config.ts` > `"default"`. |
| 111 | +3. **`cli/src/commands/migrate.ts`** — dispatch on `config.format` at **both** `writeMigration` call |
| 112 | + sites (the live-DB/diff path and the offline `runOfflineGenerate` path), plus the refusal matrix |
| 113 | + in §6. |
| 114 | +4. **`codegen-ts/src/metaobjects-config.ts`** — typed `migrate.format` config key. |
| 115 | + |
| 116 | +## 5. Data flow |
| 117 | + |
| 118 | +Unchanged through load → diff → emit. `meta migrate baseline --from-db` seeds the snapshot exactly as |
| 119 | +today and is format-independent. Only the final write differs: |
| 120 | + |
| 121 | +``` |
| 122 | +metadata ──► expected schema ─┐ |
| 123 | + ├─► diff ──► emit {up, down} ──► adapter ──► files |
| 124 | +snapshot / --from-db ─────────┘ │ |
| 125 | + default ────────────────┤ |
| 126 | + d1 ────────────────┤ |
| 127 | + flyway ────────────────┘ |
| 128 | +``` |
| 129 | + |
| 130 | +## 6. Error handling — detect-and-refuse |
| 131 | + |
| 132 | +Flyway owns apply and history. Generating a migration is ours; applying it is not. These refuse at |
| 133 | +generation time with a message naming the Flyway command instead, matching the posture established |
| 134 | +by #226/#241 (D1 FK cascade) and #258 (PK move): |
| 135 | + |
| 136 | +| Combination | Refusal reason | |
| 137 | +|---|---| |
| 138 | +| `--format flyway --apply` | Applying behind Flyway desyncs `flyway_schema_history` → use `flyway migrate` | |
| 139 | +| `--format flyway apply-pending` | Same — replaying committed migrations is Flyway's job | |
| 140 | +| `--format flyway --rollback` | Our ledger does not exist on a Flyway-managed DB → `flyway undo` (Teams) or roll forward | |
| 141 | +| `--format flyway --dialect d1` | D1 has its own Wrangler layout and wrangler transport; the combination is meaningless | |
| 142 | + |
| 143 | +Non-fatal behaviors: |
| 144 | + |
| 145 | +- A missing or empty target dir starts at `V1`. |
| 146 | +- Non-migration files, repeatable migrations (`R__`), and `U__` files are ignored by the scanner. |
| 147 | +- Engine gaps the diff does not model (triggers, cross-column CHECKs, function-valued defaults, GIN |
| 148 | + array indexes) remain the author's to hand-add. The adapter changes nothing here: it degrades by |
| 149 | + emitting what the engine computed, never by silently dropping it. |
| 150 | + |
| 151 | +## 7. Testing |
| 152 | + |
| 153 | +**Unit — `write-migration-flyway`:** empty dir → `V1`; mixed non-migration files ignored; dotted |
| 154 | +versions (`V10.5__`) → `V11`; `U__` files do **not** bump the counter; slug sanitized to underscores; |
| 155 | +trailing newline on both files. |
| 156 | + |
| 157 | +**CLI:** the four refusals in §6, each asserting exit code and message; `--format` precedence |
| 158 | +(flag > config > default); dir resolution (`--out-dir` vs the Flyway convention default). |
| 159 | + |
| 160 | +**Real-engine gate (required).** Every migrate change in this repo carries it, because a green unit |
| 161 | +suite has historically missed this exact class: **emit → apply to a real engine → introspect → |
| 162 | +re-diff must be EMPTY.** For this adapter that means generating `V1__`/`V2__` against a real |
| 163 | +sqlite/postgres, applying them in order, and asserting the re-diff is empty. |
| 164 | + |
| 165 | +**No-churn:** default-format and D1-format output stay byte-identical. Existing migrate tests must |
| 166 | +pass unchanged. |
| 167 | + |
| 168 | +## 8. Sources |
| 169 | + |
| 170 | +- Flyway migration naming and undo semantics — |
| 171 | + [Flyway migrations concepts](https://github.com/flyway/flywaydb.org/blob/gh-pages/documentation/concepts/migrations.md) |
| 172 | +- Undo is a paid edition feature; Community ignores `U__` — |
| 173 | + [Rolling Back Migrations with Flyway](https://www.baeldung.com/flyway-roll-back) |
| 174 | +- Prior art in-repo: `git show 4ca10dd2` (Flyway option added), `git show 77a5c46a` (mojo removed), |
| 175 | + `docs/superpowers/specs/2026-05-25-codegen-kotlin-design.md` §5.1 (superseded design) |
| 176 | + |
| 177 | +## 9. Out of scope |
| 178 | + |
| 179 | +- The other ADR-0015 adapters (two-file `.up.sql`/`.down.sql`, single-file-with-divider for |
| 180 | + dbmate/goose, Liquibase formatted-SQL). This design deliberately builds only the reference adapter; |
| 181 | + the others become mechanical once the `--format` axis exists. |
| 182 | +- Restoring any Java-side migrate goal. Schema stays TS-owned per ADR-0015; JVM consumers run the |
| 183 | + Node `meta` CLI for migration generation. |
| 184 | +- Closing the engine's modeling gaps (triggers, cross-column CHECKs, function-valued defaults, GIN |
| 185 | + array indexes). Orthogonal to the output envelope. |
0 commit comments