You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* chore(spec)!: retire IDataEngine.batch? per ADR-0119 D3 (#4618)
`batch?` was declared on `IDataEngine` for as long as that contract existed
and was never implemented by any engine — ObjectQL has no `batch` method and
there is no other engine in the tree — and never called: `DataEngineRequest`
was imported by exactly one file, the contract declaring the member.
Its whole specification was a three-word doc comment, which settles nothing
about partial failure, ordering, cross-object references, rollback scope, or
what `transaction: false` meant. Its neighbours `getDefaultDriverName?` /
`getDriverByName?` earn their optionality by naming an implementer and a
probing caller; this one named nothing.
The tell that nobody designed against it: `DataEngineBatchRequestSchema`
nested the request union recursively — a batch could contain batches — with
no statement about what that meant. The only test was a type pin asserting
the property was defined, which could not fail while the declaration existed.
What it claimed is now covered by members that are real: ADR-0119 D1 made
`transaction` reachable through the contract, D4 made `batchData`'s `atomic`
honest, and the wire batch has always gone through `POST {basePath}/batch`.
So this deletes a false affordance, not a capability (ADR-0049).
- Remove the member and the `DataEngineRequest` import from the contract.
- Remove `DataEngineBatchRequestSchema` and its arm from the request union.
Keep the union itself: every remaining arm is now unread too, but retiring
a published wire protocol is a different decision, tracked separately.
- Drop the type-pin test and the schema's self-parsing suite.
- Registry entry `data-engine-batch-retired` carries the FROM → TO for the
upgrade guide and spec-changes.json.
Deliberately no `retiredKey()` tombstone: a tombstone delivers through a
parse, and nothing ever parsed this schema. Its three authorable-surface
baseline lines, its json-schema.manifest entry and the stale
docs-import-surface baseline line are dropped here, deliberately. The
enforced channel is tsc.
Refs: ADR-0119 D3, ADR-0049, ADR-0078, #4618, #4612
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NKcGqCYCCpMkB5UW8jNPXx
* feat(runtime,cli,core): ADR-0119 D2 boot reconciliation + `os migrate resume` (#4617)
Completes ADR-0119 D2. The runner and sys_migration_journal landed in #4668;
this is the discovery channel that makes an interrupted run findable by
someone who does not already know it happened.
- `MigrationRecoveryPlugin` (@objectstack/runtime) — at kernel:ready, scans
for runs that started and never concluded and warns per run: chunks
committed, chunks with UNKNOWN outcome (chunk_started with no chunk_done),
whether a compensation was left half-finished, and the command that acts.
Also owns the `migration-plans` registry service.
- `os migrate resume` (@objectstack/cli) — lists interrupted runs (read-only
default, per #2186), or acts on one with --run under confirmation. Exits
non-zero when a run ends `failed`, so a scripted recovery cannot move on
from a migration that needs a human.
- `MigrationPlanRegistry` (@objectstack/core) — where a resume finds the plan.
Boot discovers, the CLI acts. Resuming is a large, irreversible, potentially
hour-long write against production data; doing that as an unrequested side
effect of a process starting is the kind of behaviour an operator finds out
about from a graph. It is also not always possible at boot — a resume needs
the plan's live callbacks, and the package owning them may not be loaded in
whichever process happened to restart first. The per-plan `onCrash` policy
still decides WHAT acting means; it does not decide WHEN, and "when" is the
part a human should own. Deferring is safe because of the runner's
re-entrancy: `started ∧ ¬done` is durable, so a run stays exactly as
recoverable an hour later as it was at boot.
The registry exists because a journal cannot hold a plan: forward/compensate
are functions and load() reads the live database, so none of it crosses a
process boundary — hence the journal stores the plan HASH. A run whose plan
no loaded package registers is REPORTED, never silently skipped: "nothing to
resume" and "the code that owns this run is not here" are different facts.
Degradation is deliberate in both directions. No engine or no journal object
(a lean kernel) → skipped in silence, because such a kernel has no
interrupted runs and a warning there would train operators to ignore this
plugin's output. A scan that FAILS is reported — "I could not check" is not
"there is nothing to find".
11 new runtime tests pin the split (boot writes nothing to the journal), the
three states an operator must tell apart, and both degradation paths; 2 new
core tests cover the registry.
Refs: ADR-0119 D2, #4617, #4668, ADR-0078
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NKcGqCYCCpMkB5UW8jNPXx
* fix(spec): true up the variant-docs and strictness ledgers the batch removal moved (#4618)
CI's "Spec property liveness" job runs four gates, not one. `check:liveness`
itself was fine; the two that broke are the ledgers that track the SHAPE of
the code the batch removal changed.
- `variant-docs.json`: the outer `DataEngineRequestSchema` union carried a
`batch` variant, and a SECOND entry described the inner union inside
`DataEngineBatchRequestSchema.requests` — the same member set minus `batch`.
With the batch schema gone the inner union no longer exists and the outer
one narrowed to exactly the inner one's old key, so the two entries collapse
into one. Keeping both would have left an entry whose union is gone, which
is what the gate reported.
- The strictness ledger's `data-engine.zod.ts` row (14 → 13 `z.object(` sites)
and the `data/` section header it sums into (162 → 161).
Both are hand-maintained maps of the code, and the point of the gates is that
a map which drifts is worse than none because it gets followed. Verified by
running all 17 `@objectstack/spec` check:* scripts, not just the four in the
failing job.
Refs: #4618, ADR-0119 D3
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NKcGqCYCCpMkB5UW8jNPXx
---------
Co-authored-by: Claude <noreply@anthropic.com>
feat(runtime,cli,core): boot reconciliation and `os migrate resume` for the migration journal — an interrupted run can no longer go unnoticed (ADR-0119 D2, #4617)
8
+
9
+
Completes ADR-0119 D2. The runner and `sys_migration_journal` landed in #4668; this is the discovery channel that makes an interrupted run findable by someone who does not already know it happened.
10
+
11
+
**`MigrationRecoveryPlugin` (`@objectstack/runtime`)** — at `kernel:ready`, scans the journal for runs that started and never concluded, and warns per run: how many chunks committed, which have an **unknown** outcome (`chunk_started` with no `chunk_done`), whether a compensation was left half-finished, and the exact command that will act. It also owns the `migration-plans` registry service.
12
+
13
+
**`os migrate resume` (`@objectstack/cli`)** — lists interrupted runs (read-only, the default), or acts on one with `--run <id>`, under confirmation. Exits non-zero when a run ends `failed`, so a scripted recovery cannot move on from a migration that needs a human.
14
+
15
+
**`MigrationPlanRegistry` (`@objectstack/core`)** — where a resume finds the plan it has to re-run.
16
+
17
+
## Boot discovers, the CLI acts
18
+
19
+
This is the design decision, and it is deliberate rather than incidental.
20
+
21
+
Resuming is a large, irreversible, potentially hour-long write against production data. Doing that as an unrequested side effect of a process starting is the kind of behaviour an operator finds out about from a graph. It is also not always possible at boot: a resume needs the plan's live callbacks, and the package that owns them may not be loaded in whichever process happened to restart first.
22
+
23
+
So boot surfaces the run and names the command; the command acts, under explicit operator intent. ADR-0119 D2's per-plan `onCrash` policy still decides **what** acting means — resume forward from the first chunk lacking `chunk_done`, or unwind what committed — it just does not decide **when**, and "when" is the part a human should own.
24
+
25
+
Deferring is safe precisely because of the runner's re-entrancy: `started ∧ ¬done` is durable, so an interrupted run stays exactly as recoverable an hour later as it was at boot. Nothing decays while the operator decides.
26
+
27
+
## Why a plan registry exists at all
28
+
29
+
A journal cannot hold a plan. `forward` and `compensate` are functions and `load()` reads the live database, so none of it crosses a process boundary — which is why the journal records the plan **hash**, not the plan. Recovery therefore needs the plan handed back by the code that owns it, and `migration-plans` is that seam: between "the journal knows a run stopped at chunk 7" and "something in this process knows what chunk 7 was supposed to do".
30
+
31
+
A run whose plan no loaded package registers is **reported**, never silently skipped — the operator is told which plan id is missing. "Nothing to resume" and "the code that owns this run is not here" are different facts, and only one of them is safe to ignore.
32
+
33
+
## Degradation
34
+
35
+
No engine, or no `sys_migration_journal` registered (a lean kernel that never composed platform-objects) → the scan is skipped in **silence**: such a kernel has no interrupted runs to find, and a warning there would train operators to ignore this plugin's output, which is the one thing it cannot afford. A scan that **fails**, by contrast, is reported — "I could not check" and "there is nothing to find" are different answers.
36
+
37
+
11 new tests pin the split (boot writes nothing to the journal), the three states an operator must tell apart (clean / interrupted / half-unwound), and both degradation paths.
chore(spec)!: retire `IDataEngine.batch?` — declared for the life of the contract, implemented by nothing, called by no one (ADR-0119 D3, #4618)
6
+
7
+
**FROM → TO**
8
+
9
+
| Removed | Use instead |
10
+
|---|---|
11
+
|`IDataEngine.batch?(requests, { transaction })`|`IObjectQLEngine.transaction(cb)` for in-process multi-write atomicity |
12
+
| — a batch over ONE object | the metadata protocol's `batchData` with `options.atomic: true`|
13
+
| — a cross-object batch over the wire |`POST {basePath}/batch`|
14
+
|`DataEngineBatchRequestSchema` / `data/DataEngineBatchRequest` JSON schema | nothing — it described only the removed member |
15
+
16
+
**One-line fix:** delete the `batch` implementation from any engine that has one (there were none in this repo) and route multi-write atomicity through `engine.transaction(cb)`.
17
+
18
+
## Why
19
+
20
+
`batch?` was declared on `IDataEngine` for as long as that contract has existed and was **never implemented by any engine** — `ObjectQL` has no `batch` method, and there is no other engine in the tree. It also had **no caller**: `DataEngineRequest` was imported by exactly one file, the contract declaring the member.
21
+
22
+
Its entire specification was a three-word doc comment, "Batch Operations (Transactional)", which settles nothing about partial failure, ordering, cross-object references, rollback scope, or what `transaction: false` was supposed to mean — the questions a batch API exists to answer. Contrast its neighbours `getDefaultDriverName?` / `getDriverByName?`, whose optionality is evidenced: each names its implementer and its probing caller.
23
+
24
+
The tell that nobody ever designed against it is in the schema. `DataEngineBatchRequestSchema.requests` nested the request union **recursively** — a batch could contain batches — with no statement anywhere about what that meant for ordering or rollback.
25
+
26
+
The only test was a type pin: an ad-hoc object literal carrying a `batch` property, asserting the property was defined. It could not fail while the declaration existed, and would have passed unchanged for the member's entire life with no engine implementing it. A test that asserts a contract member is *declared* is not evidence the contract is *honoured*.
27
+
28
+
A declared capability that cannot be exercised is ADR-0049's enforce-or-remove target. What this one claimed is now covered by members that are real — ADR-0119 D1 made `transaction` reachable through the contract, D4 made `batchData`'s `atomic` honest — so the removal deletes a false affordance, not a capability.
29
+
30
+
## Scope notes
31
+
32
+
-**The wire batch is untouched.**`POST {basePath}/batch` validates with `CrossObjectBatchRequestSchema` / `BatchUpdateRequestSchema` from `api/batch.zod.ts` — a different schema that never had anything to do with the removed one.
33
+
-**`DataEngineRequestSchema` stays**, minus its `batch` arm. Every remaining arm now has zero readers in this repo (there is no Virtual Data Engine implementation, only this schema describing one), which makes the whole block a further enforce-or-remove candidate — tracked separately, because retiring a published wire protocol is a different decision from retiring `batch?` and does not belong in a change whose title promised something narrower.
34
+
-**Deliberately no `retiredKey()` tombstone.** A tombstone delivers its prescription through a *parse*, and nothing ever parsed `DataEngineBatchRequestSchema`. A prescription nobody can receive is noise (the `spec-property-retirement` playbook's third route). Its three `authorable-surface.json` baseline lines and its `json-schema.manifest.json` entry are therefore dropped in this change, deliberately, along with the now-stale `docs-import-surface.baseline.json` line that excused its missing type export. The enforced channel here is `tsc`, and it points at callers.
Copy file name to clipboardExpand all lines: docs/audits/2026-07-unknown-key-strictness-ledger.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -455,12 +455,12 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
455
455
|`notification.zod.ts`| 1 | authorable (p) |**#4610 dropped two sites** — the `./ui``Notification` (toast/banner instance) and `NotificationConfig` (toaster global config) shapes were removed: zero importers in all three repos, and both shadowed live names owned elsewhere (`./api` owns the inbox row). What remains is `NotificationActionSchema`, part of the presentation vocabulary the ui entry keeps |
Copy file name to clipboardExpand all lines: docs/protocol-upgrade-guide.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -257,6 +257,9 @@ The same enforce-or-remove pass reaches the event vocabulary: `DataEventType` dr
257
257
-**`data-field-changed-event-retired`** — `api.DataEventType 'data.field.changed'` → the `data.record.updated` event, whose payload already carries the per-field detail: `changes` (the changed fields), plus `before` / `after`
258
258
- Why not automatic: `data.field.changed` was declared in `DataEventType` and emitted by nothing — the engine's `publishDataEvent` sends `data.record.{created,updated,deleted}` and (since #4639) `data.records.{updated,deleted}`, and no other producer exists in either repository. A subscriber that switched on it was waiting on an event no producer sends: the branch never ran, and because the surrounding `switch` still compiled, nothing anywhere reported the gap (ADR-0078's silently-inert declaration, on the event vocabulary). `DataEventSchema` could not have carried the semantics even if something had emitted it — the payload is record-shaped (`recordId`, `changes`, `before`, `after`) with no `field` / `oldValue` / `newValue` slot — so the member promised a granularity the contract has no room for. Per-field detail is therefore not lost: it has always ridden on `data.record.updated` as `changes`, which is one event per write rather than N events on a wide table. This is a runtime EVENT surface — no stack, example or template authors an event name (webhooks subscribe through the separate authorable `WebhookTriggerType`, whose vocabulary was already trimmed to producers that exist, #3196) — so there is no source for the chain to rewrite, and deliberately no schema tombstone: a removed ENUM MEMBER cannot carry a retiredKey() fix-it error the way an authorable object key can (the same limit the sharing-rule `full` retirement hit above). The enforced channels are tsc, which fails any consumer still naming the value in a `DataEventType` position, and the enum parse, which now rejects the name instead of accepting an event that never arrives. A genuine per-field stream, if one is ever wanted, gets its own honest contract the way #4639 gave bulk writes theirs. ADR-0049 / ADR-0078, #4673.
259
259
- Done when: No consumer subscribes to or switches on `data.field.changed`; per-field change detail is read from a `data.record.updated` event's `changes` map (with `before` / `after` for the surrounding state). Deleting the dead branch changes no observable behaviour — it never executed — so the migration is removing code that could not run, not rebuilding a capability.
260
+
-**`data-engine-batch-retired`** — `contracts.IDataEngine.batch / data.DataEngineBatchRequestSchema` → `IObjectQLEngine.transaction(cb)` for in-process multi-write atomicity; the metadata protocol's `batchData` with `options.atomic: true` for a batch over one object; `POST {basePath}/batch` on the wire
261
+
- Why not automatic: `batch?` was declared on `IDataEngine` for as long as that contract existed and was never implemented by any engine: `ObjectQL` has no `batch` method and there is no other engine in the tree. It also had no caller — `DataEngineRequest` was imported by exactly one file, the contract declaring the member. Its entire specification was a three-word doc comment ("Batch Operations (Transactional)"), which settles nothing about partial failure, ordering, cross-object references, rollback scope, or what `transaction: false` was supposed to mean — the questions a batch API exists to answer. Contrast its neighbours `getDefaultDriverName?` / `getDriverByName?`, whose optionality is evidenced: each names its implementer and its probing caller. The tell that nobody ever designed against it is in the schema: `DataEngineBatchRequestSchema.requests` nested the request union RECURSIVELY, so a batch could contain batches, with no statement anywhere about what that meant for ordering or rollback. The only test was a type pin — an ad-hoc object literal carrying a `batch` property, asserting the property was defined — which could not fail while the declaration existed and would have passed unchanged for the member's whole life with no engine implementing it. What it claimed is now covered by members that are real, so the removal deletes a false affordance rather than a capability: ADR-0119 D1 made `transaction` reachable through the contract and D4 made `batchData`'s `atomic` honest, while the wire batch has always validated with `CrossObjectBatchRequestSchema` / `BatchUpdateRequestSchema` from `api/batch.zod.ts` — a different schema entirely, untouched here. TS/API surfaces only: an engine is CODE, never stack metadata, so there is no source for the chain to rewrite. Deliberately no schema tombstone either — nothing ever parsed `DataEngineBatchRequestSchema`, so a `retiredKey()` prescription would have no one to reach; its three `authorable-surface.json` baseline lines and its `json-schema.manifest.json` entry are dropped in the same change, deliberately. The enforced channel is tsc. ADR-0049 / ADR-0078, #4618.
262
+
- Done when: No code calls `engine.batch(...)` and no type references `DataEngineBatchRequest`; in-process multi-write atomicity goes through `IObjectQLEngine.transaction(cb)`, a batch over one object through `batchData` with `options.atomic: true`, and a cross-object batch over the wire through `POST {basePath}/batch`. Because no engine implemented the member, an implementation left behind still compiles and is simply never reached; a CALLER of it no longer type-checks — and there were none.
0 commit comments