fix(metadata-protocol): write the merged autonumber high-water mark before retiring the __global__ counter (#12554)

* fix(metadata-protocol): write the merged autonumber high-water mark before retiring the __global__ counter

The #8686 seed/API tenancy handoff ran an UPDATE of the organization-scoped
_objectstack_sequences row followed by an unconditional DELETE of the
'__global__' one. On a fresh install there is no organization-scoped row yet, so
the UPDATE matched nothing (a success on every dialect), the DELETE ran anyway,
and the counter table was left empty — sending SqlDriver.getNextSequenceValue
back into its one-time MAX(data) bootstrap and re-issuing an already-allocated
business identifier.

The handoff is now one ordered decision per scope: write the merged mark
(INSERT when the destination row is absent, UPDATE when it is not), read it
back, and only then retire the '__global__' row by its own stored key_hash.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6HFzyH98W1YaQXhJUJt6o

* test(metadata-protocol,cli): seed the sequences fixtures with the key the platform actually stores

CI caught the #12394 handoff writing a SECOND counter row for one logical
sequence, on live MySQL and on SQLite alike. Root cause is the fixtures, not
the repair: both hand-seeded `key_hash` as an invented string (`'h1'`/`'h2'`
and `'h_global'`/`'h_org'`), which was inert for as long as the repair
addressed counter rows by `(object, field, tenant_id)`. #12394 addresses the
destination row by `key_hash` — the table's only key — so an invented hash
describes a table no install can hold: the org row reads ABSENT and a second
row is inserted beside it.

Measured: the driver stores `key_hash = sha256(object US tenant US field US
scope)` for every row it writes, and `ensureSequencesKeyHashShape` recomputes
the same hash for every legacy row it migrates.

- cli: take the hash from the driver's own `sequenceKeyHash`, so the fixture is
  the same bytes the only production writer would have written and cannot drift.
- metadata-protocol live-MySQL: spell the derivation independently (this package
  does not depend on driver-sql), making it a third spelling and therefore a pin
  on it; give `key_hash` its real PRIMARY KEY.
- metadata-protocol unit: new #12394 suite over a KEYED store that answers the
  probe by its parameter and enforces the primary key. The INSERT-vs-UPDATE
  decision had no unit coverage keyed by a real hash — every existing fake
  matched on statement shape and handed back its one row for any key.

`sequenceKeyHash` is exported from the module for that suite; it is NOT
re-exported from the package index, so the published surface is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6HFzyH98W1YaQXhJUJt6o

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: os-warren

.changeset/seed-tenancy-counter-handoff-writes-first.md

@@ -0,0 +1,50 @@
+---
+'@objectstack/metadata-protocol': patch
+---
+
+fix(metadata-protocol): the #8686 tenancy backfill writes the merged autonumber high-water mark before it deletes anything (#12394)
+
+The seed/API tenancy handoff destroyed the counter it was supposed to move. It ran two
+independent statements — an `UPDATE` of the organization-scoped `_objectstack_sequences`
+row, then an unconditional `DELETE` of the `'__global__'` one — and on a **fresh install**
+there is no organization-scoped row yet, because no API create has happened. The `UPDATE`
+matched nothing, which is a success on every dialect; the `DELETE` ran regardless; the
+counter table was left empty. `SqlDriver.getNextSequenceValue` then re-entered the
+`if (!existing)` bootstrap its own docstring reserves for first allocation, re-derived the
+counter from `MAX(data)`, and **re-issued a business identifier that had already been
+handed out** — measured on 17.1.0: `ACC-000009` on two different records.
+
+The zero-row case is the *normal* first-boot shape, not an edge case: it is precisely the
+shape `buildSplitProbeSql`'s `LEFT JOIN` was widened to catch, so the repair fired on
+exactly the installs where its merge loop body never executed.
+
+The handoff is now one ordered decision per scope:
+
+1. **write** the merged mark — `INSERT` when the organization-scoped row is absent,
+   `UPDATE` when it exists;
+2. **read it back** — "the statement did not throw" was never evidence a row was written,
+   and an `UPDATE` matching zero rows is exactly the defect above;
+3. **then** retire the `'__global__'` row, addressed by its own stored `key_hash`, so a
+   retirement can only ever hit the row whose mark was just merged.
+
+A throw at any step leaves the `'__global__'` row in place — which is the state the next
+boot's split probe detects and retries — so a failed repair now loses nothing.
+
+Per **scope**, because a `{YYYYMMDD}` / `{field}` / per-parent format runs one counter row
+per rendered prefix. The old merge was scope-blind in both directions: it could raise every
+scope's counter to one merged value, and it deleted every scope's `'__global__'` row.
+
+The merge rule itself is unchanged and is the 2026-08-15 ruling's: the greater of the two
+**counters**, never the data max. That rule is the whole point — a counter is allowed to
+sit ahead of its rows (a rolled-back insert burns a number, by design), and that gap is
+exactly what the old handoff threw away.
+
+Graded **patch**: a defect repair inside an existing migration. It adds no export to
+`@objectstack/metadata-protocol`'s public index — the new SQL builders are module-scoped
+for their own unit tests, matching the index's own recorded rule that an export added so a
+test can import a value is the shape to catch before it ships.
+
+No change to the allocator. Reaching `if (!existing)` is not evidence of lost state — a new
+tenant, a new day and a new `{field}` group each reach it legitimately, and a destroyed
+counter leaves no row behind to tell the two apart — so a guard there would fire on the hot
+path and still not detect this. The repair belongs where the state was destroyed.

packages/cli/src/utils/platform-migrations-arming.integration.test.ts

@@ -149,9 +149,24 @@ async function writeDamagedInstall(): Promise<void> {
     t.bigInteger('last_value').notNullable().defaultTo(0);
     t.timestamp('updated_at');
   });
+  // [#12394] `key_hash` is DERIVED, never invented. It used to be seeded here as
+  // the placeholders `'h1'`/`'h2'`, which was harmless only for as long as the
+  // repair addressed counter rows by `(object, field, tenant_id)`: nothing read
+  // the column, so any string did. #12394's handoff addresses the destination
+  // row by `key_hash` — the key the table is actually keyed on — so a fixture
+  // carrying an invented hash describes a table the platform cannot produce, and
+  // the repair correctly reads the org row as ABSENT and inserts a SECOND one.
+  //
+  // Taking the hash from the driver's own `sequenceKeyHash` rather than
+  // re-spelling it here is the point: this fixture is now the same bytes the
+  // only production writer of this table would have written, and it cannot drift
+  // from it. `ensureSequencesKeyHashShape` recomputes the same hash for every
+  // legacy row it migrates, so this is the shape of every real install.
+  const keyHash = (object: string, tenantId: string, field: string, scope = ''): string =>
+    (seed as any).sequenceKeyHash(object, tenantId, field, scope);
   await k(SEQUENCES_TABLE).insert([
-    { key_hash: 'h1', object: 'crm_case', tenant_id: GLOBAL_TENANT, field: 'case_number', scope: '', last_value: SEEDED_LAST_VALUE },
-    { key_hash: 'h2', object: 'crm_case', tenant_id: ORG_ID, field: 'case_number', scope: '', last_value: 1 },
+    { key_hash: keyHash('crm_case', GLOBAL_TENANT, 'case_number'), object: 'crm_case', tenant_id: GLOBAL_TENANT, field: 'case_number', scope: '', last_value: SEEDED_LAST_VALUE },
+    { key_hash: keyHash('crm_case', ORG_ID, 'case_number'), object: 'crm_case', tenant_id: ORG_ID, field: 'case_number', scope: '', last_value: 1 },
   ]);
   await seed.disconnect();
 }

packages/metadata-protocol/src/migrations/seed-tenancy-backfill.live-mysql.test.ts

@@ -46,6 +46,7 @@
  */
 
 import { afterAll, beforeAll, describe, expect, it } from 'vitest';
+import { createHash } from 'node:crypto';
 import mysql from 'mysql2/promise';
 import {
   backfillSeedTenancy,
@@ -68,6 +69,32 @@ const DB = currentLiveMysqlDatabase();
 const OBJECT = 'os9381_case';
 const FIELD = 'case_number';
 
+/**
+ * The row key of `_objectstack_sequences`, spelled the way its only production
+ * writer spells it (#12394).
+ *
+ * This fixture used to seed `key_hash` as the placeholders `'h_global'` and
+ * `'h_org'`. That was invisible for as long as the repair addressed counter rows
+ * by `(object, field, tenant_id)` — nothing read the column, so any string did.
+ * #12394's handoff addresses the destination row by `key_hash`, which is the key
+ * the table is actually keyed on, so an invented hash describes a table no
+ * install can hold: the repair reads the organization row as ABSENT and inserts
+ * a SECOND counter for one logical sequence.
+ *
+ * Spelled here rather than imported because `metadata-protocol` does not depend
+ * on `driver-sql` — which makes this a THIRD independent spelling of the same
+ * derivation, and therefore a pin on it: a separator or field-order change in
+ * `seed-tenancy-backfill.ts` stops matching these rows and this suite goes red.
+ * The separator is the ASCII unit separator, written as the escape \u001f and
+ * never as a raw control byte — the same discipline the module and the driver
+ * both keep.
+ */
+function sequenceKeyHash(object: string, tenantId: string, field: string, scope: string): string {
+  return createHash('sha256')
+    .update(`${object}\u001f${tenantId}\u001f${field}\u001f${scope}`)
+    .digest('hex');
+}
+
 if (!MYSQL_URL && EXPECT_LIVE) {
   describe('#9381 live MySQL', () => {
     it('OS_TEST_MYSQL_URL must be set — this runner declared it provisioned a server', () => {
@@ -113,9 +140,18 @@ describe.skipIf(!MYSQL_URL)('#9381 seed-tenancy backfill on a LIVE MySQL', () =>
     // Column names spelled the way the driver's own `createSequencesTable`
     // spells them; `last_value` is quoted here for the same reason the migration
     // has to quote it (see the reserved-word assertion below).
+    //
+    // [#12394] `key_hash` carries its real PRIMARY KEY. The driver declares it
+    // `.notNullable().primary()`, and it is the ONLY key this table has — no
+    // unique index stands behind `(object, tenant_id, field, scope)`. Seeding it
+    // as a plain column let a repair that wrote a SECOND row for one logical
+    // counter land quietly as an extra row instead of an `ER_DUP_ENTRY`; with
+    // the real key here, that defect can only ever be an error on the two
+    // dialects that enforce it.
     await conn.query(
       `CREATE TABLE \`${SEQUENCES_TABLE}\` (` +
-        '`key_hash` VARCHAR(64), `object` VARCHAR(64), `tenant_id` VARCHAR(64), ' +
+        '`key_hash` VARCHAR(64) NOT NULL PRIMARY KEY, `object` VARCHAR(64), ' +
+        '`tenant_id` VARCHAR(64), ' +
         '`field` VARCHAR(64), `scope` VARCHAR(255) NOT NULL DEFAULT \'\', ' +
         '`last_value` INT, `updated_at` DATETIME(3))',
     );
@@ -127,8 +163,12 @@ describe.skipIf(!MYSQL_URL)('#9381 seed-tenancy backfill on a LIVE MySQL', () =>
     await conn.query("INSERT INTO `sys_organization` (`id`) VALUES ('org_live')");
     await conn.query(
       `INSERT INTO \`${SEQUENCES_TABLE}\` (\`key_hash\`, \`object\`, \`tenant_id\`, \`field\`, \`last_value\`) ` +
-        `VALUES ('h_global', '${OBJECT}', '${GLOBAL_TENANT}', '${FIELD}', 38), ` +
-        `('h_org', '${OBJECT}', 'org_live', '${FIELD}', 4)`,
+        `VALUES (?, '${OBJECT}', '${GLOBAL_TENANT}', '${FIELD}', 38), ` +
+        `(?, '${OBJECT}', 'org_live', '${FIELD}', 4)`,
+      [
+        sequenceKeyHash(OBJECT, GLOBAL_TENANT, FIELD, ''),
+        sequenceKeyHash(OBJECT, 'org_live', FIELD, ''),
+      ],
     );
     // The card's own repro: seeded rows carry NULL, API rows carry the org, and
     // CASE-00001/2 were minted on BOTH sides.

packages/metadata-protocol/src/migrations/seed-tenancy-backfill.null-seam.test.ts

@@ -250,9 +250,17 @@ describe('#10789 a seam that answers with no rows still reports no-split', () =>
     // installs it exists for. This seam answers every SELECT and hands back a
     // NON-result-set for every write — and the repair must still complete.
     const writes: string[] = [];
-    const exec: SeedTenancyExec = async (sql: string) => {
-      if (sql.startsWith('UPDATE') || sql.startsWith('DELETE')) {
+    // #12394: the handoff reads its own write back before retiring anything, so
+    // the counter row is modelled rather than assumed — the WRITES still answer
+    // with a non-result-set, which is what this case exists to pin.
+    let orgCounter: Record<string, unknown> | undefined;
+    const exec: SeedTenancyExec = async (sql: string, params?: unknown[]) => {
+      if (sql.startsWith('UPDATE') || sql.startsWith('DELETE') || sql.startsWith('INSERT')) {
         writes.push(sql.slice(0, 6));
+        if (sql.startsWith('INSERT')) orgCounter = { last_value: Number(params?.[5]) };
+        if (sql.startsWith('UPDATE') && sql.includes('_objectstack_sequences')) {
+          orgCounter = { last_value: Number(params?.[0]) };
+        }
         return { affectedRows: 3 }; // not a result set, by design
       }
       if (sql.includes('WHERE 1 = 0')) return [];
@@ -263,7 +271,8 @@ describe('#10789 a seam that answers with no rows still reports no-split', () =>
       }
       if (sql.includes(ORGANIZATION_TABLE)) return [{ id: 'org_a' }];
       if (sql.includes('rows_holding')) return [];
-      if (sql.includes('tenant_id')) return [{ tenant_id: 'org_a', last_value: 1 }];
+      if (sql.includes('"key_hash" = ?')) return orgCounter ? [orgCounter] : [];
+      if (sql.includes('tenant_id')) return [{ key_hash: 'hash-global', scope: '', last_value: 38 }];
       return [];
     };
 
@@ -274,6 +283,9 @@ describe('#10789 a seam that answers with no rows still reports no-split', () =>
     expect(result.organizationId).toBe('org_a');
     expect(writes).toContain('UPDATE');
     expect(writes).toContain('DELETE');
+    // The write that used to be missing: on a fresh install there is no
+    // organization-scoped counter row to raise, so the mark is INSERTed.
+    expect(writes).toContain('INSERT');
   });
 });