Skip to content

Commit 272ee9d

Browse files
dmealingclaude
andcommitted
fix(#258): migrate refuses a primary-key move instead of silently dropping the PK
Adopting a database whose PRIMARY KEY differs from the metadata identity had no expressible migration: the diff/emit has no primary-key change kind, so a moved PK (e.g. live PRIMARY KEY (user_id), metadata identity id) degraded into an add-column + drop-column — the old PK column and its constraint were dropped, the new column was never made PK, and the table was left with NO primary key, so every referencing foreign key failed at apply. Detect-and-refuse (the #226->#241 arc precedent): migration generation now compares the introspected primary key to the metadata identity and throws PrimaryKeyChangeError with a clear message instead of emitting the un-appliable SQL. Runs after rename detection, so a PK column that was merely RENAMED (the engine preserves the PK through RENAME COLUMN) is not mistaken for a move. Gated by a new DiffArgs.refusePrimaryKeyChange flag set only by the two migration-generation paths (online meta migrate --db and offline planOffline); the read-only drift/verify path is unchanged, so meta verify still reports drift. Auto-migrating a PK move (add/drop-primary-key change kinds) is a later follow-up. npm-only (migrate-ts + cli). Gated by unit tests (refuse on a move, not on an unchanged PK, not on a renamed PK column, and off by default) plus a real-Postgres integration test proving introspection reads the live PK and the refusal fires on the genuine reproduction. Existing meta gen / meta migrate output is byte-identical. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HLoJkFSyoticveo5ehMUAr
1 parent 494ebbb commit 272ee9d

7 files changed

Lines changed: 313 additions & 1 deletion

File tree

‎server/typescript/packages/cli/src/commands/migrate.ts‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ import {
2525
readSnapshot,
2626
writeSnapshot,
2727
BlockedChangesError,
28+
PrimaryKeyChangeError,
2829
renderD1,
2930
writeMigrationD1,
3031
introspectD1,
@@ -389,6 +390,10 @@ export async function migrateCommand(
389390
actual,
390391
dialect: kysely.dialect,
391392
allow: tokensToAllowOptions(config.allow),
393+
// #258 — adopting a live DB whose PRIMARY KEY differs from the metadata identity
394+
// has no expressible migration; refuse loudly instead of emitting SQL that drops
395+
// the constraint and breaks referencing FKs at apply.
396+
refusePrimaryKeyChange: true,
392397
// #208 §7 — declared-@unmanaged objects are external: exclude them from the
393398
// actual side so migrate proposes neither create nor drop for them.
394399
unmanagedNames: collectUnmanagedNames(metadata),
@@ -398,6 +403,13 @@ export async function migrateCommand(
398403
},
399404
});
400405
} catch (err) {
406+
// #258 — a primary-key move has no expressible migration; refuse loudly.
407+
if (err instanceof PrimaryKeyChangeError) {
408+
log.error(`migrate: ${err.message}`);
409+
emitStructuredError(`migrate: ${err.message}`, "align the primary key manually, or reconcile the metadata identity to match the live table", fmt);
410+
await kysely.close();
411+
return 1;
412+
}
401413
// diff() throws when onAmbiguous returns "abort" — surface as exit 1
402414
// with the collected ambiguity list.
403415
if ((err as Error).message.includes("aborted by onAmbiguous")) {
@@ -809,6 +821,12 @@ export async function runOfflineGenerate(
809821
},
810822
});
811823
} catch (err) {
824+
// #258 — a primary-key move has no expressible migration; refuse loudly.
825+
if (err instanceof PrimaryKeyChangeError) {
826+
log.error(`migrate: ${err.message}`);
827+
emitStructuredError(`migrate: ${err.message}`, "align the primary key manually, or reconcile the metadata identity to match the live table", fmt);
828+
return 1;
829+
}
812830
if ((err as Error).message.includes("aborted by onAmbiguous")) {
813831
log.error(`migrate: ambiguous rename/drop detected; re-run with --on-ambiguous rename|drop-add`);
814832
return 1;

‎server/typescript/packages/migrate-ts/src/diff/index.ts‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ import type {
88
import type { SqlType } from "../sql-type.js";
99
import { sqlTypeEquals } from "../sql-type.js";
1010
import { applyStatus } from "./status.js";
11+
import { PrimaryKeyChangeError } from "../errors.js";
1112
import { detectColumnRenames, detectTableRenames } from "./rename-heuristic.js";
1213
import { viewSqlEquals } from "../view-sql-compare.js";
1314
import { viewReplaceIsLegal } from "../view-column-types.js";
@@ -58,6 +59,16 @@ export interface DiffArgs {
5859
unmanagedNames?: string[];
5960
/** Dialect; CHECK-constraint evolution on existing tables is emitted for postgres only. */
6061
dialect?: Dialect;
62+
/**
63+
* #258 — refuse (throw {@link PrimaryKeyChangeError}) when an existing table's live
64+
* PRIMARY KEY differs from the metadata identity. There is no primary-key change kind
65+
* in the emitter, so such a move would silently degrade into add-column + drop-column
66+
* and leave the table with no PK, breaking referencing FKs at apply time. Set by the
67+
* migration-generation path (snapshot/plan.ts); left unset by the read-only drift/verify
68+
* path so `meta verify` keeps reporting drift rather than throwing. Off by default —
69+
* existing callers are byte-identical.
70+
*/
71+
refusePrimaryKeyChange?: boolean;
6172
}
6273

6374
const ALLOWED: ChangeStatus = { state: "allowed" };
@@ -266,10 +277,51 @@ export async function diff(
266277
delete (c as Aug)._columns;
267278
}
268279

280+
// #258: refuse a primary-key MOVE at generation time. There is no primary-key change
281+
// kind, so a table whose live PK differs from the metadata identity would degrade into
282+
// an add-column + drop-column and lose the constraint (breaking referencing FKs at
283+
// apply). Runs after rename detection so a PK column that was merely RENAMED (PK
284+
// preserved by the engine) is not mistaken for a move. Gated by refusePrimaryKeyChange
285+
// so only migration generation refuses; the read-only drift/verify path is unchanged.
286+
if (args.refusePrimaryKeyChange === true) {
287+
for (const [id, expectedTable] of expectedTables) {
288+
const actualTable = actualTables.get(id);
289+
if (actualTable === undefined) continue; // create-table: PK is inline, not a move
290+
assertPrimaryKeyUnchanged(expectedTable, actualTable, changes);
291+
}
292+
}
293+
269294
applyStatus(changes, args.allow ?? {});
270295
return { changes, blocked: changes.filter((c) => c.status.state === "blocked") };
271296
}
272297

298+
/**
299+
* #258 — throw {@link PrimaryKeyChangeError} when a table's live PRIMARY KEY differs from
300+
* the metadata identity. Live PK column names are first mapped through any detected
301+
* `rename-column` for this table, so a renamed PK column (the engine preserves the PK
302+
* through a `RENAME COLUMN`) is not treated as a move. A genuine move — a PK column added
303+
* or dropped, or the key repointed to different columns — has no expressible migration and
304+
* is refused.
305+
*/
306+
function assertPrimaryKeyUnchanged(
307+
expected: TableDescriptor,
308+
actual: TableDescriptor,
309+
changes: Change[],
310+
): void {
311+
const wantId = tableIdentity(expected);
312+
const renamed = new Map<string, string>();
313+
for (const c of changes) {
314+
if (c.kind === "rename-column" && tableIdentity({ name: c.table, ...schemaSpread(c.schema) }) === wantId) {
315+
renamed.set(c.from, c.to);
316+
}
317+
}
318+
const livePk = actual.primaryKey.map((col) => renamed.get(col) ?? col);
319+
const wantPk = expected.primaryKey;
320+
const unchanged = livePk.length === wantPk.length && livePk.every((col, i) => col === wantPk[i]);
321+
if (unchanged) return;
322+
throw new PrimaryKeyChangeError(expected.name, actual.primaryKey, expected.primaryKey, expected.schema);
323+
}
324+
273325
function isDiffArgs(x: DiffArgs | SchemaSnapshot): x is DiffArgs {
274326
return "expected" in x && "actual" in x;
275327
}

‎server/typescript/packages/migrate-ts/src/errors.ts‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,3 +93,37 @@ export class BlockedChangesError extends Error {
9393
this.enableHints = hints;
9494
}
9595
}
96+
97+
/**
98+
* #258 — a table whose live PRIMARY KEY differs from the metadata identity cannot be
99+
* migrated: the diff/emit has no primary-key change kind, so the difference degrades
100+
* silently into an add-column + drop-column (the old PK column is dropped, the new one
101+
* is never made PK), leaving the table with no primary key and breaking every foreign
102+
* key that references it at apply time. Migration generation detects the move and throws
103+
* this instead of emitting un-appliable SQL (detect-and-refuse; the #226→#241 arc for D1
104+
* FK cascades is the precedent). A pure column RENAME is NOT a key move — the engine
105+
* preserves the PK through a `RENAME COLUMN` — and does not trigger this.
106+
*/
107+
export class PrimaryKeyChangeError extends Error {
108+
override readonly name = "PrimaryKeyChangeError";
109+
readonly table: string;
110+
readonly livePrimaryKey: string[];
111+
readonly expectedPrimaryKey: string[];
112+
readonly schema?: string;
113+
114+
constructor(table: string, livePrimaryKey: string[], expectedPrimaryKey: string[], schema?: string) {
115+
const qualified = schema !== undefined ? `${schema}.${table}` : table;
116+
const fmt = (cols: string[]) => (cols.length > 0 ? `PRIMARY KEY (${cols.join(", ")})` : "no primary key");
117+
super(
118+
`primary key of "${qualified}" differs from the live database: live ${fmt(livePrimaryKey)} vs ` +
119+
`metadata ${fmt(expectedPrimaryKey)}. migrate cannot express a primary-key change (there is no ` +
120+
`add/drop-primary-key change kind), so this would silently drop the constraint and break every ` +
121+
`foreign key that references this table. Align the primary key manually — or reconcile the metadata ` +
122+
`identity to match the live table — before migrating.`,
123+
);
124+
this.table = table;
125+
this.livePrimaryKey = livePrimaryKey;
126+
this.expectedPrimaryKey = expectedPrimaryKey;
127+
if (schema !== undefined) this.schema = schema;
128+
}
129+
}

‎server/typescript/packages/migrate-ts/src/index.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ export { planOffline, baselineFromMetadata } from "./snapshot/plan.js";
3131
export type { PlanOfflineArgs, PlanOfflineResult } from "./snapshot/plan.js";
3232

3333
// Errors
34-
export { BlockedChangesError, SetNullNotNullableError } from "./errors.js";
34+
export { BlockedChangesError, SetNullNotNullableError, PrimaryKeyChangeError } from "./errors.js";
3535

3636
// SqlType helpers (rarely needed but useful for advanced consumers)
3737
export { isWidening, sqlTypeEquals } from "./sql-type.js";

‎server/typescript/packages/migrate-ts/src/snapshot/plan.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,10 @@ export async function planOffline(args: PlanOfflineArgs): Promise<PlanOfflineRes
3838
expected: nextSnapshot,
3939
actual: args.snapshot,
4040
dialect: args.dialect,
41+
// #258 — migration generation refuses a primary-key MOVE (there is no primary-key
42+
// change kind to express it; it would otherwise silently drop the constraint). The
43+
// read-only verify/drift path does NOT set this, so `meta verify` still reports drift.
44+
refusePrimaryKeyChange: true,
4145
// #208 §7 — exclude declared-@unmanaged objects from the actual (snapshot) side too,
4246
// so the OFFLINE generate path never proposes DROP for an external table that a
4347
// `baseline --from-db` captured into the snapshot (parity with the online/verify paths).
Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
import { describe, test, expect } from "bun:test";
2+
import { diff } from "../src/diff/index.js";
3+
import { PrimaryKeyChangeError } from "../src/errors.js";
4+
import type { SchemaSnapshot, TableDescriptor, ColumnDescriptor } from "../src/types.js";
5+
6+
// #258 — migrate has no primary-key change kind, so a table whose live PRIMARY KEY
7+
// differs from the metadata identity degrades silently into add-column + drop-column
8+
// (the old PK column is dropped, the new one is never made PK), leaving the table
9+
// with no PK and breaking every referencing FK at apply time. The migration-generation
10+
// diff must DETECT the move and REFUSE loudly rather than emit un-appliable SQL.
11+
12+
function table(name: string, columns: ColumnDescriptor[], primaryKey: string[]): TableDescriptor {
13+
return { name, columns, indexes: [], foreignKeys: [], primaryKey, checks: [] };
14+
}
15+
const col = (name: string, sqlType: ColumnDescriptor["sqlType"], identity?: "increment" | "uuid"): ColumnDescriptor =>
16+
({ name, sqlType, nullable: false, ...(identity ? { identity } : {}) });
17+
18+
const UUID = { kind: "uuid" } as const;
19+
const BIGINT = { kind: "integer", bits: 64 } as const;
20+
21+
// live table has PRIMARY KEY (user_id bigint); metadata identity is id (uuid).
22+
const liveUserIdPk = (): SchemaSnapshot => ({
23+
tables: [table("user_profiles", [col("user_id", BIGINT)], ["user_id"])],
24+
views: [],
25+
});
26+
const metadataIdPk = (): SchemaSnapshot => ({
27+
tables: [table("user_profiles", [col("id", UUID, "uuid")], ["id"])],
28+
views: [],
29+
});
30+
31+
describe("diff — #258 primary-key move detect-and-refuse", () => {
32+
test("REFUSES when the live PK differs from the metadata identity (add/drop, not a rename)", async () => {
33+
await expect(
34+
diff({ expected: metadataIdPk(), actual: liveUserIdPk(), refusePrimaryKeyChange: true }),
35+
).rejects.toBeInstanceOf(PrimaryKeyChangeError);
36+
});
37+
38+
test("the refusal names the table and both primary keys", async () => {
39+
try {
40+
await diff({ expected: metadataIdPk(), actual: liveUserIdPk(), refusePrimaryKeyChange: true });
41+
throw new Error("expected diff to refuse");
42+
} catch (e) {
43+
expect(e).toBeInstanceOf(PrimaryKeyChangeError);
44+
const err = e as PrimaryKeyChangeError;
45+
expect(err.table).toBe("user_profiles");
46+
expect(err.livePrimaryKey).toEqual(["user_id"]);
47+
expect(err.expectedPrimaryKey).toEqual(["id"]);
48+
expect(err.message).toContain("user_profiles");
49+
}
50+
});
51+
52+
test("does NOT refuse when the primary key is unchanged", async () => {
53+
const same = (): SchemaSnapshot => ({
54+
tables: [table("user_profiles", [col("id", UUID, "uuid")], ["id"])],
55+
views: [],
56+
});
57+
const r = await diff({ expected: same(), actual: same(), refusePrimaryKeyChange: true });
58+
expect(r.changes).toEqual([]);
59+
});
60+
61+
test("does NOT refuse a PK-column RENAME (rename resolved → PK preserved by the engine)", async () => {
62+
const actual: SchemaSnapshot = { tables: [table("t", [col("user_id", BIGINT)], ["user_id"])], views: [] };
63+
const expected: SchemaSnapshot = { tables: [table("t", [col("userId", BIGINT)], ["userId"])], views: [] };
64+
const r = await diff({
65+
expected,
66+
actual,
67+
refusePrimaryKeyChange: true,
68+
allow: { dropColumn: true },
69+
onAmbiguous: async () => "rename",
70+
});
71+
// The renamed PK column is a rename-column change; the PK is preserved, so no refusal.
72+
expect(r.changes.some((c) => c.kind === "rename-column")).toBe(true);
73+
});
74+
75+
test("without the flag, diff does not throw — the verify/drift path is unchanged", async () => {
76+
const r = await diff({ expected: metadataIdPk(), actual: liveUserIdPk(), allow: { dropColumn: true } });
77+
expect(r).toBeDefined();
78+
// The un-fixed behavior: silent add-column + drop-column, PK move lost.
79+
expect(r.changes.some((c) => c.kind === "add-column")).toBe(true);
80+
expect(r.changes.some((c) => c.kind === "drop-column")).toBe(true);
81+
});
82+
});
Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
/**
2+
* Real-Postgres gate for #258 — "adopting a database whose PRIMARY KEY differs
3+
* from the metadata identity."
4+
*
5+
* Root cause: the diff/emit has no primary-key change kind, so a table whose live
6+
* PK moved (e.g. live `PRIMARY KEY (user_id)`, metadata identity `id`) degrades
7+
* silently into an add-column `id` + drop-column `user_id`: the old PK column (and
8+
* its constraint) is dropped, the new column is never made PK, and the table is left
9+
* with NO primary key — so every foreign key that references it is rejected at apply
10+
* ("there is no unique constraint matching given keys for referenced table …").
11+
*
12+
* The fix is DETECT-AND-REFUSE: migration generation compares the introspected PK to
13+
* the metadata identity and throws PrimaryKeyChangeError instead of emitting the
14+
* un-appliable SQL. This test proves the refusal fires against the REAL introspected
15+
* schema (a unit assertion on hand-built snapshots is not sufficient evidence that
16+
* introspection reads the live PK correctly) — and that WITHOUT the guard the diff
17+
* still produces the silent add/drop pair, so the guard is what closes the gap.
18+
*
19+
* Gated on MIGRATE_TS_PG_URL like every other pg integration test here; skips cleanly
20+
* when unset.
21+
*/
22+
23+
import { test, expect, beforeAll, afterAll, describe } from "bun:test";
24+
import { Pool } from "pg";
25+
import { Kysely, PostgresDialect, sql } from "kysely";
26+
import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata";
27+
import { buildExpectedSchema } from "../../src/expected-schema.js";
28+
import { introspectPostgres } from "../../src/introspect/postgres.js";
29+
import { diff } from "../../src/diff/index.js";
30+
import { PrimaryKeyChangeError } from "../../src/errors.js";
31+
32+
const PG_URL = process.env["MIGRATE_TS_PG_URL"];
33+
const realDescribe = PG_URL ? describe : describe.skip;
34+
35+
// Metadata: user_profiles keyed on `id` (uuid). The LIVE table (raw SQL below) is
36+
// instead keyed on `user_id` — the PK MOVE the bug is about.
37+
const META = JSON.stringify({
38+
"metadata.root": {
39+
package: "acme",
40+
children: [
41+
{
42+
"object.entity": {
43+
name: "UserProfile",
44+
children: [
45+
{ "source.rdb": { "@table": "user_profiles" } },
46+
{ "field.uuid": { name: "id" } },
47+
{ "field.string": { name: "authUserId", "@required": true } },
48+
{ "identity.primary": { name: "pk", "@fields": ["id"] } },
49+
],
50+
},
51+
},
52+
],
53+
},
54+
});
55+
56+
let k: Kysely<Record<string, unknown>>;
57+
let pool: Pool;
58+
59+
if (PG_URL) {
60+
beforeAll(() => {
61+
pool = new Pool({ connectionString: PG_URL });
62+
k = new Kysely<Record<string, unknown>>({ dialect: new PostgresDialect({ pool }) });
63+
});
64+
afterAll(async () => {
65+
await cleanup();
66+
await k.destroy();
67+
});
68+
}
69+
70+
async function cleanup(): Promise<void> {
71+
await sql.raw(`DROP TABLE IF EXISTS "agent_configs" CASCADE`).execute(k);
72+
await sql.raw(`DROP TABLE IF EXISTS "user_profiles" CASCADE`).execute(k);
73+
}
74+
75+
async function loadRoot(json: string) {
76+
return (await new MetaDataLoader().load([new InMemoryStringSource(json)])).root;
77+
}
78+
79+
realDescribe("PG #258 — a live primary-key move is refused, not silently dropped", () => {
80+
test("adopting a DB whose PRIMARY KEY differs from the metadata identity refuses instead of losing the PK", async () => {
81+
await cleanup();
82+
83+
// LIVE schema: user_profiles keyed on user_id, with a second table whose FK
84+
// references it — the exact shape where a silent PK drop breaks the FK at apply.
85+
await sql
86+
.raw(`CREATE TABLE "user_profiles" ("user_id" bigint PRIMARY KEY, "auth_user_id" text NOT NULL)`)
87+
.execute(k);
88+
await sql
89+
.raw(
90+
`CREATE TABLE "agent_configs" ("id" bigint PRIMARY KEY, ` +
91+
`"created_by" bigint NOT NULL REFERENCES "user_profiles" ("user_id"))`,
92+
)
93+
.execute(k);
94+
95+
const root = await loadRoot(META);
96+
const expected = buildExpectedSchema(root, { dialect: "postgres" });
97+
const actual = await introspectPostgres(k);
98+
99+
// Introspection read the live PK as user_id (the premise of the whole bug).
100+
const liveUP = actual.tables.find((t) => t.name === "user_profiles");
101+
expect(liveUP?.primaryKey).toEqual(["user_id"]);
102+
103+
// WITHOUT the guard: the diff silently degrades into add-column id + drop-column
104+
// user_id — the PK is dropped and never re-added (the un-appliable migration).
105+
const unsafe = await diff({ expected, actual, dialect: "postgres", allow: { dropColumn: true } });
106+
expect(unsafe.changes.some((c) => c.kind === "add-column" && c.column.name === "id")).toBe(true);
107+
expect(unsafe.changes.some((c) => c.kind === "drop-column" && c.column === "user_id")).toBe(true);
108+
109+
// WITH the guard (migration generation): refuse loudly, so no bad SQL is emitted.
110+
let refused: unknown;
111+
try {
112+
await diff({ expected, actual, dialect: "postgres", refusePrimaryKeyChange: true, allow: { dropColumn: true } });
113+
} catch (e) {
114+
refused = e;
115+
}
116+
expect(refused).toBeInstanceOf(PrimaryKeyChangeError);
117+
const err = refused as PrimaryKeyChangeError;
118+
expect(err.table).toBe("user_profiles");
119+
expect(err.livePrimaryKey).toEqual(["user_id"]);
120+
expect(err.expectedPrimaryKey).toEqual(["id"]);
121+
});
122+
});

0 commit comments

Comments
 (0)