feat(cli): os migrate duplicates reports the rows blocking the kernel:ready index tightenings (#11031)

Three migrations replace a declared UNIQUE index with the NULL-safe — and
sometimes active-rows-only — form it was always meant to have, at `kernel:ready`
on a serving boot: `ensureMetadataOverlayIndexes` (`sys_metadata`, one index per
overlay state), `ensureViewDefinitionActiveIndex` (`sys_view_definition`) and
`ensureSysSettingIdentityIndex` (`sys_setting`). Each is a tightening, so rows
an installation already holds can block it; the migration then refuses under
ADR-0120 D4 — previous index kept, no row touched, boot continues — and reports
at `error` on the boot channel.

That channel was the only one, and not by omission. These indexes are invisible
to the drift differ by construction, twice over: after the tightening
`isRuntimeManagedIndex` excludes the index (without that exclusion a boot would
propose rebuilding away the guarantee it had just created), and before it each
migration deliberately reuses the DECLARED index's name, so the reconciler's
name-matched slot reads as filled whichever physical form is really there. A
prior round measured it with a matched control — one database carrying the same
duplicate damage under a declared organization-unique index and under
`sys_view_definition`'s runtime one — and `os migrate plan` named the declared
one in full while saying nothing whatsoever about the runtime one.

Maintainer ruling, 2026-08-22: the reporting path is `os migrate duplicates`,
which already boots read-only and owns the "inventory, never repair" contract,
keeping `os migrate plan`'s drift contract untouched. `plan` describes work
`os migrate apply` will do; this work is applied by the next SERVING boot, by a
different applier.

What lands:

* `@objectstack/metadata-protocol` gains `runtime-index-preflight.ts` —
  `runtimeIndexProbes()` and `collectRuntimeIndexPreflight()`. The descriptors
  read each migration's OWN exported builders rather than restating the keys,
  so the pre-flight and the boot report cannot describe different duplicates,
  and the `sys_setting` probe uses the migration's MySQL spelling on MySQL,
  where the bare form is ERROR 1064 on the reserved word `key`.
* The report gains `runtimeIndexPreflight` (one entry per index:
  blocked/clear/table-absent/unreadable, a blocked one naming every colliding
  key group and its row count) plus `summary.runtimeIndexesBlocked` and
  `summary.runtimeIndexBlockingRows`. `reportVersion` moves 1 → 2: every
  version-1 field keeps its name, shape and meaning, and the bump says there is
  more in the document for a consumer that validates it strictly.
* `collectDuplicateIdentifierReport`'s new option is REQUIRED rather than
  optional. An optional section defaults to `[]`, and `[]` is also what a clean
  database produces, so a caller that forgot to wire it would ship a clean bill
  of health from a probe that never ran — the #10677 failure, one section over.
  Required moves that mistake to a compile error.
* Liveness is keyed on whether the seam returns a RESULT SET, reusing the
  sibling migration's `isResultSet` rather than copying it. A no-op seam would
  otherwise report all four tightenings as `table-absent`, which is that same
  failure again wearing a different status.
* Nine referral sites repointed, not deleted (the ruling is explicit): three
  conflict-error strings that told the operator to "run `os migrate plan`" — an
  instruction the measurement proved false — and the six doc comments that state
  the same referral as part of the D4 disposition. Their pins now assert both
  that the new command is named and that the false one is gone.

Nothing about a migration's behaviour changes: no tightening is armed, deferred
or altered, and `plan` is untouched. The pre-flight only makes the refusal's
evidence readable one command before the restart.

Read-only is pinned LOGICALLY — schema plus every row, ordered — in both the new
unit suite and the CLI integration test, never by a file hash: a raw hash over a
SQLite file moves on any read-write open and would accuse this command of
mutating the install it exists to describe.

Fixes #8725


Claude-Session: https://claude.ai/code/session_019bmVFqoQPq63zhKrxdYG1r

Co-authored-by: Claude <noreply@anthropic.com>

Author: Elon Musk

.changeset/migrate-duplicates-kernel-ready-preflight.md

@@ -0,0 +1,62 @@
+---
+"@objectstack/metadata-protocol": minor
+"@objectstack/cli": minor
+---
+
+`os migrate duplicates` now reports the rows blocking the three `kernel:ready`
+NULL-safe index tightenings, and the three migrations' conflict messages point
+there instead of at `os migrate plan` (#8725).
+
+**The gap.** Three migrations replace a declared UNIQUE index with the NULL-safe
+— and sometimes active-rows-only — form it was always meant to have, at
+`kernel:ready` on a serving boot:
+
+| table | index(es) | migration |
+| --- | --- | --- |
+| `sys_metadata` | overlay `active` + `draft` | `ensureMetadataOverlayIndexes` |
+| `sys_view_definition` | `idx_sys_view_def_active` | `ensureViewDefinitionActiveIndex` |
+| `sys_setting` | the declared row identity | `ensureSysSettingIdentityIndex` |
+
+Each is a tightening, so rows an installation already holds can block it. The
+migration then refuses — previous index kept, no row touched, boot continues —
+and reports at `error` on the boot channel. That channel was the only one:
+these indexes are invisible to `os migrate plan` **by construction**, twice
+over. After the tightening, `isRuntimeManagedIndex` excludes the index (without
+that exclusion a boot would propose rebuilding away the guarantee it had just
+created); before it, each migration deliberately reuses the *declared* index's
+name, so the reconciler's name-matched slot reads as filled whichever physical
+form is really there. Measured with a matched control — one database carrying
+the same duplicate damage under a declared index and under
+`sys_view_definition`'s runtime one — `plan` named the declared one in full and
+said nothing whatsoever about the runtime one.
+
+**What is new.** The report gains a `runtimeIndexPreflight` section, one entry
+per index, each `blocked` (with every colliding key group and its row count),
+`clear`, `table-absent` (`sys_setting` arrives with the optional settings
+service) or `unreadable` (with the driver's own message), plus
+`summary.runtimeIndexesBlocked` and `summary.runtimeIndexBlockingRows`.
+`reportVersion` moves `1` → `2`. Every `1` field keeps its name, shape and
+meaning; the bump says there is more in the document, for consumers that
+validate it strictly.
+
+The probes are the migrations' own duplicate-listing statements —
+`@objectstack/metadata-protocol` exports `collectRuntimeIndexPreflight` and
+`runtimeIndexProbes`, which read those builders rather than restating the keys,
+so the pre-flight and the boot report cannot describe different duplicates. On
+MySQL the `sys_setting` probe uses the migration's MySQL spelling, where the
+bare form is `ERROR 1064` on the reserved word `key`.
+
+**The referral, repointed rather than deleted** (maintainer ruling, 2026-08-22).
+All three conflict messages told the operator to "run `os migrate plan`" as an
+alternative way to list the blocking rows, and that instruction was false: they
+now name `os migrate duplicates`, which answers it. The six doc comments that
+state the same referral as part of the ADR-0120 D4 disposition are updated with
+them.
+
+**Nothing about a migration's behaviour changes.** No tightening is armed,
+deferred or altered, and `os migrate plan`'s drift contract is untouched. The
+pre-flight only makes the refusal's evidence readable one command before the
+restart — from a command that boots read-only and writes nothing, which is
+pinned logically (schema plus every row, ordered) rather than by a file hash: a
+raw hash over a SQLite file moves on any read-write open and would accuse this
+command of mutating the install it exists to describe.

content/docs/deployment/cli.mdx

@@ -677,7 +677,7 @@ where the data lives.
 | `os migrate value-shapes` | Scan stored reference and structured-JSON field values against the platform's value contract, and record the deployment's migration flag when clean |
 | `os migrate summary-nulls` | Backfill roll-up `count` / `sum` columns still stored as `NULL` on parent rows created before the insert-time seed. Repairs values; no flag, nothing depends on it having run |
 | `os migrate meta --stored` | Replay the metadata conversion chain over this deployment's `sys_metadata` rows and rewrite the ones still carrying a pre-protocol shape. Hygiene, not a gate — nothing depends on it having run |
-| `os migrate duplicates` | Report business identifiers already minted twice across the organization partitions — a read-only inventory as JSON on stdout. Renumbers nothing and writes nothing at all; run it before the boot-time tenancy repair, which overwrites part of the evidence |
+| `os migrate duplicates` | Report business identifiers already minted twice across the organization partitions, and the rows blocking the boot-time NULL-safe index tightenings — a read-only inventory as JSON on stdout. Renumbers nothing and writes nothing at all; run it before the boot-time tenancy repair, which overwrites part of the evidence |
 
 ```bash
 os migrate files-to-references            # Dry run: full report, writes nothing
@@ -1011,6 +1011,57 @@ that could not be probed is listed under `skipped` with its reason, because
 a driver with no raw SQL seam (memory, MongoDB) fails the whole run with
 `error: "no_sql_seam"` rather than returning an empty inventory.
 
+##### The `kernel:ready` index pre-flight
+
+The report carries a second section, `runtimeIndexPreflight`, answering a
+different question: **will the next server start be able to finish tightening
+the platform's own unique indexes?**
+
+Three migrations run at `kernel:ready` on a serving boot (`os dev`, `os serve`,
+`os start`) and replace a declared UNIQUE index with the NULL-safe — and
+sometimes active-rows-only — form it was always meant to have:
+
+| Table | Index | What the tightening adds |
+| :--- | :--- | :--- |
+| `sys_metadata` | overlay `active` and `draft` | package-less overlays stop being NULL-distinct |
+| `sys_view_definition` | `idx_sys_view_def_active` | shared and environment-level views stop being NULL-distinct, and only active rows are constrained |
+| `sys_setting` | the declared row identity | tenant- and global-scope rows stop being NULL-distinct on `user_id` |
+
+Each is a **tightening**, so rows an installation already holds can block it.
+When that happens the migration refuses — the previous index stays in place, no
+row is touched, and the server keeps running — and reports it at `error` in the
+boot log. Until this section existed that log line was the only channel: these
+indexes are invisible to `os migrate plan` by construction, because the drift
+reconciler deliberately excludes runtime-managed indexes (otherwise the next
+boot would propose rebuilding away the guarantee it just created), and because
+each migration reuses the *declared* index's name, so the reconciler's slot for
+it reads as correctly filled whichever form is physically there.
+
+So the pre-flight lives here instead, on the command that already boots
+read-only and repairs nothing. It runs the migrations' own duplicate-listing
+queries — the exact statements the boot log prints — and reports one entry per
+index:
+
+| `status` | Meaning |
+| :--- | :--- |
+| `blocked` | Rows collide under the tightened key. `groups` lists each colliding key and how many rows hold it. The next serving boot will refuse this index |
+| `clear` | The probe ran and nothing collides |
+| `table-absent` | The table is not installed here. `sys_setting`, for instance, arrives with the optional settings service |
+| `unreadable` | The probe could not run; `detail` carries the driver's message |
+
+`summary.runtimeIndexesBlocked` and `summary.runtimeIndexBlockingRows` are the
+same finding counted at the head of the document.
+
+Read `blocked` as **work to do before the restart, not damage**: nothing is
+lost while an index stays untightened, but the guarantee it carries is not in
+force until the listed rows are resolved — and only an operator can decide which
+of two colliding rows survives, which is why the platform refuses rather than
+picking one.
+
+`--object` does not narrow this section. It is a fixed set of platform indexes
+rather than a slice of your registry, and `filter` describes the object scan
+only.
+
 ### Scaffolding
 
 | Command | Alias | Description |

docs/qa/platform-checklist/areas/cli.json

@@ -1047,7 +1047,7 @@
       "title": "os migrate duplicates: a read-only JSON inventory of identifiers minted across partitions — within-partition repeats excluded, nothing written, the live two-counter condition reported, and runnable BEFORE the #8686 repair destroys the evidence",
       "since": "v17",
       "status": "active",
-      "revision": 2,
+      "revision": 3,
       "priority": "P1",
       "surface": "cli",
       "personas": ["operator (local shell, pre-repair audit)"],
@@ -1066,15 +1066,16 @@
         "boot the scratch app once with `os dev -d file:/tmp/<run>/dup.db` so the base schema exists; stop it",
         "seed via direct SQL per the fixture recipe: the cross-partition duplicate, the within-partition repeat, an organizations row for '<org>', and the paired sequence counters",
         "md5sum the DB file; run `os migrate duplicates > report.json; echo $?`; md5sum again and byte-compare",
-        "jq the report: .report/.reportVersion/.generatedAt/.database/.globalPartition/.filter/.counters/.scanned/.skipped/.duplicates/.liveConditions/.summary",
+        "jq the report: .report/.reportVersion/.generatedAt/.database/.globalPartition/.filter/.counters/.scanned/.skipped/.duplicates/.liveConditions/.runtimeIndexPreflight/.summary",
+        "seed the kernel:ready blocker too (#8725): two ACTIVE sys_view_definition rows with the SAME name and organization_id/owner both NULL — then jq .runtimeIndexPreflight and .summary.runtimeIndexesBlocked",
         "run `os migrate duplicates --object <the-object>` and `--object <an-object-with-no-findings>` — capture .filter in both payloads",
         "run `os migrate duplicates --database-url file:/tmp/<run>/dup.db` and confirm it reaches the same DB (the flag also honors OS_DATABASE_URL)",
         "negative: from the memory-driver scratch config run `os migrate duplicates; echo $?` and capture the refusal payload",
         "stderr/stdout split: confirm report.json parses as ONE JSON document — the boot's own log lines must have gone to stderr (#6217)"
       ],
       "acceptance": [
         {
-          "clause": "the report is the declared machine-readable contract on stdout: report 'duplicate-identifiers', reportVersion 1, generatedAt, database, globalPartition, filter, counters {table, status read|absent}, scanned[], skipped[], duplicates[], liveConditions[], summary — and each duplicate carries object/field/value/holderCount/partitions plus per-holder id/organization/partition/createdAt (createdAt null when the object has no such column, never a failed probe)",
+          "clause": "the report is the declared machine-readable contract on stdout: report 'duplicate-identifiers', reportVersion 2, generatedAt, database, globalPartition, filter, counters {table, status read|absent}, scanned[], skipped[], duplicates[], liveConditions[], runtimeIndexPreflight[], summary {…, runtimeIndexesBlocked, runtimeIndexBlockingRows} — and each duplicate carries object/field/value/holderCount/partitions plus per-holder id/organization/partition/createdAt (createdAt null when the object has no such column, never a failed probe)",
           "oracle": "log",
           "verify": "jq walks every declared key of the seeded run's payload; shape pinned by duplicates.contract.test.ts — cite its pass for the full-shape guarantee, drive the CLI for the instance",
           "evidence": "report.json + the jq walk"
@@ -1091,6 +1092,12 @@
           "verify": "the two md5sums match; duplicates.pre-repair.test.ts pins the same invariant down to _objectstack_sequences",
           "evidence": "the md5 pair"
         },
+        {
+          "clause": "the kernel:ready pre-flight (#8725) reports the RUNTIME-migration class os migrate plan cannot see: one entry per index the three kernel:ready migrations tighten (four — the overlay migration owns two), each blocked|clear|table-absent|unreadable, a blocked one naming every colliding key group and its row count",
+          "oracle": "log",
+          "verify": ".runtimeIndexPreflight names idx_sys_view_def_active as blocked with the seeded view name, organization_id_key '__global__' and owner_key '' — and the SAME database run through `os migrate plan` mentions neither the index nor the view name (the matched control: the declared-index duplicate above IS reported by plan, this one is not)",
+          "evidence": "the pre-flight section beside the plan output for one database"
+        },
         {
           "clause": "the live condition (#8928 point 4) fires exactly when a __global__ counter sits beside an org-scoped counter for the same object/field — reported as a prediction with globalLastValue and the per-org counters, and counters.status says where it was read from ('absent' still yields a complete duplicates inventory)",
           "oracle": "log",

packages/cli/src/commands/migrate/duplicates.contract.test.ts

@@ -21,7 +21,9 @@ import { tmpdir } from 'node:os';
 import { join } from 'node:path';
 import { SqlDriver } from '@objectstack/driver-sql';
 import {
+  collectRuntimeIndexPreflight,
   normalizeRows,
+  runtimeIndexProbes,
   GLOBAL_TENANT,
   ORGANIZATION_FIELD,
   SEQUENCES_TABLE,
@@ -120,7 +122,7 @@ afterAll(async () => {
   try { rmSync(dir, { recursive: true, force: true }); } catch { /* ignore */ }
 });
 
-const collect = (objectFilter?: string) =>
+const collect = async (objectFilter?: string) =>
   collectDuplicateIdentifierReport({
     exec,
     normalize: normalizeRows,
@@ -131,14 +133,31 @@ const collect = (objectFilter?: string) =>
     sequencesTable: SEQUENCES_TABLE,
     client: 'better-sqlite3',
     now: () => new Date('2026-08-17T12:00:00.000Z'),
+    // The real pre-flight against the real fixture — never a stand-in. This
+    // database has none of the four platform tables, so every entry is
+    // `table-absent`, and the `blocked` shape is pinned in its own test below
+    // over a database that really carries the damage.
+    runtimeIndexPreflight: await collectRuntimeIndexPreflight(exec, { client: 'better-sqlite3' }),
     ...(objectFilter ? { objectFilter } : {}),
   });
 
+/**
+ * The four probes, as `@objectstack/metadata-protocol` declares them.
+ *
+ * Read from the producer rather than restated here: the descriptor (table,
+ * index name, key parts, row scope, the two statements) is the migration's own
+ * definition of its key, and a second copy in this file would be a second
+ * definition to keep in step. What this file pins is that the report carries
+ * that descriptor through UNCHANGED, plus a status and its groups.
+ */
+const PROBES = runtimeIndexProbes({ client: 'better-sqlite3' });
+
 describe('#8928 os migrate duplicates — the report document', () => {
   it('is exactly this shape, whole', async () => {
     expect(await collect()).toEqual({
       report: 'duplicate-identifiers',
-      reportVersion: 1,
+      // #8725 added `runtimeIndexPreflight` and its two summary counters.
+      reportVersion: 2,
       generatedAt: '2026-08-17T12:00:00.000Z',
       database: 'better-sqlite3 (fixture)',
       globalPartition: '__global__',
@@ -212,16 +231,40 @@ describe('#8928 os migrate duplicates — the report document', () => {
           organizationCounters: [{ organization: 'org_x', lastValue: 4 }],
         },
       ],
+      // One entry per index the three `kernel:ready` migrations tighten — FOUR,
+      // because the overlay migration builds one per state and either can be
+      // blocked on its own. Present whatever the outcome: an index left out
+      // would make "nothing blocks it" and "it was never probed" the same
+      // absence, which is the rule this command already applies to `skipped`.
+      runtimeIndexPreflight: PROBES.map((probe) => ({
+        ...probe,
+        status: 'table-absent',
+        groups: [],
+      })),
       summary: {
         objectsScanned: 2,
         fieldsScanned: 3,
         duplicateValues: 3,
         duplicateRows: 6,
         liveConditions: 1,
+        runtimeIndexesBlocked: 0,
+        runtimeIndexBlockingRows: 0,
       },
     });
   });
 
+  it('names the four kernel:ready indexes, and the migration behind each', async () => {
+    const report = await collect();
+    expect(
+      report.runtimeIndexPreflight.map((p) => `${p.migration}:${p.table}:${p.index}`),
+    ).toEqual([
+      'ensureMetadataOverlayIndexes:sys_metadata:idx_sys_metadata_overlay_active',
+      'ensureMetadataOverlayIndexes:sys_metadata:idx_sys_metadata_overlay_draft',
+      'ensureViewDefinitionActiveIndex:sys_view_definition:idx_sys_view_def_active',
+      'ensureSysSettingIdentityIndex:sys_setting:uniq_sys_setting_organization_id_namespace_key_scope_user_id',
+    ]);
+  });
+
   it('does NOT report a value repeated inside ONE partition — the narrow ruled definition', async () => {
     // `REF-1` is held twice, both times by `org_x`. That is a repeat the
     // partitioned unique index already refuses; reporting it would be the wider
@@ -248,6 +291,7 @@ describe('#8928 os migrate duplicates — the report document', () => {
       organizationField: ORGANIZATION_FIELD,
       sequencesTable: SEQUENCES_TABLE,
       client: 'better-sqlite3',
+      runtimeIndexPreflight: [],
     });
     expect(report.skipped).toEqual([
       expect.objectContaining({