feat(spec): a refinement that never reaches the published JSON Schema now makes a noise (#18729)

Refs #18670 (item 1)

Clause-②: no

Triage on #18670 wrote the handoff order and reserved the second half
for the maintainer. This PR is **item 1 only**: census the refinements
whose rule never reaches the published JSON Schema, and make that fact
**make a noise**. It narrows no published shape, removes no refinement,
and adds no CI job — the ratchet lives inside
`packages/spec/scripts/build-schemas.ts`, which
`check:authorable-surface` already runs.

## The premises, re-measured — two reproduce, one does not

The card was filed by the dispatching seat, so every claim in it was
re-measured here with its own control.

**① `z.toJSONSchema` drops a refinement — REPRODUCES.** On zod **4.4.3**
(the version `packages/spec` resolves), a plain record, the same record
with a `.refine()`, and the same record with an **aborting** `.refine()`
project byte-identically. Lit control in the same run:
`z.string().min(1)` does move the bytes (`"minLength":1`), so the
instrument can see a projected constraint. Pinned in
`packages/spec/scripts/dropped-refinements.test.ts`.

**② `TraceSamplingConfig.condition.anyOf[0]` — REPRODUCES, byte for
byte.** The generated file reads
`{"type":"object","propertyNames":{"type":"string"},"additionalProperties":{}}`.

**③ The consequence, as written — DOES NOT REPRODUCE at that slot.**
Measured against the live schema:

| claim | measured |
|---|---|
| the published `anyOf[0]` accepts `{dialect:'cel'}`, "which the runtime
refuses" | the **runtime accepts it too**. `condition` is
`z.union([z.record(z.string(), z.unknown()), ExpressionInputSchema])`;
the permissive record branch absorbs the object, so published and
runtime **agree** here |
| "the string branch carries no non-empty constraint" | it carries
**`minLength: 1`** |
| "the runtime requires non-blank after trim" | it does not — `" "` is
**accepted**; only `""` is refused, which `minLength: 1` refuses as well
|

So the tracing slot is not a specimen of the gap. **The gap itself is
real and the card's own dedupe words point straight at the right file**:
`ExpressionInputSchema` carries the refinement "Expression requires at
least one of `source` or `ast`", and the published
`packages/spec/json-schema/shared/ExpressionInput.json` states only
`"required": ["dialect"]`. `{"dialect":"cel"}` is **accepted by the
published file and refused by the runtime** — the card's sentence, one
file up from where it was written.

## The census

Measured by the generator itself, on this branch, at zod 4.4.3:

| reading | value |
|---|---|
| refinement call sites in `packages/spec/src/**` (excluding
`*.test.ts`) | **126** — `.refine(` 48 · `.superRefine(` 75 · `.check(`
3 |
| published schemas carrying at least one **dropped** refinement |
**240** |
| dropped refinement **sites** on published schemas | **688** |
| sites that **did** reach the published file | **0** |
| sites with no JSON form on either side to compare | **3** |

The fan-out between 126 and 688 is the point: one refinement on a shared
schema lands on every published file that embeds it. Per namespace: `ui`
123 · `api` 251 · `data` 102 · `system` 81 · `automation` 46 · `kernel`
32 · `ai` 14 · `shared` 13 · `security` 10 · `identity` 9 ·
`integration` 1.

⚠️ The dispatch's dark probe counted `.refine(` in 21 files. That count
included `*.test.ts` and, more importantly, **did not look for
`.superRefine(`** — which is 75 of the 126 sites. The population is
larger than the card implies, in the card's own direction.

The whole census is committed as
`packages/spec/dropped-refinements.baseline.json`, keyed by published
file, each entry naming the **paths** at which a rule is dropped.

## The noise, in three places

1. **On the artifact.** Each affected file under
`packages/spec/json-schema/**` now carries `x-dropped-refinements`,
naming its own sites — the sibling of the `x-unprojectable-branches`
annotation #16431 already writes for the opposite direction. `x-`
keywords are ignored by every validator, so **the set of documents each
schema accepts is byte-for-byte what it was**.
2. **In the build log.** Every `gen:schema` / `check:authorable-surface`
run reports the accepted population in full, the same discipline as the
never-published ledger above it.
3. **As a ratchet.** A published schema that drops a refinement and is
not in the ledger fails the generator; so does a ledger entry whose site
list the build no longer observes, in either direction. The failure
prints the corrected entry in full.

## The detector is MEASURED, not asserted

`collectDroppedRefinements` does not trust the sentence above. For every
node carrying a `custom` check it builds the same node **without** those
checks — `clone()` recomputes the constraint bag from the check list —
and compares the two projections byte for byte. A zod release that
learns to project refinements therefore turns those sites `projected`
and the ledger goes red asking to be emptied, instead of reading as
current forever.

Two false readings were measured and closed while building it, both
pinned:

- `clone()` does not carry `.describe()` text (it lives in
`z.globalRegistry`, keyed by instance), so without a meta copy every
described node read as `projected`.
- Walking a `lazy` node's `_cachedInner` memo reaches a **second
instance** of the same graph, whose recursive `$ref` layout differs. All
**80** sites the first build called `projected` were that, and none of
them were about a refinement.

## Proof it can fire, and proof it stays silent

A live ablation on this branch, both legs proven on disk rather than by
exit code.

- **Mutated** — one real `.refine()` added to `TestAssertionSchema` in
`packages/spec/src/qa/testing.zod.ts` (a namespace with zero recorded
drops). Marker grep on disk: `0 -> 1`; anchor grep `1 -> 0`. `pnpm
--filter @objectstack/spec gen:schema` exits **1**, naming four
published schemas and the exact path under each:

```
 4 published schema(s) drop a refinement and are not declared in dropped-refinements.baseline.json:
     + qa/TestAssertion  (1 site(s))
         ROOT  (object)      [the generator prints this position in angle brackets; respelled here]
     + qa/TestScenario  (1 site(s))
         steps.element.assertions.element  (object)
     + qa/TestStep  (1 site(s))
         assertions.element  (object)
     + qa/TestSuite  (1 site(s))
         scenarios.element.steps.element.assertions.element  (object)
```

- **Restored** — `git checkout HEAD -- PATH`; `git hash-object` equals
the HEAD blob (`198a4236e9c3e8f7a1c58c55dd4db7cbabc28c98`) and `git diff
HEAD` is empty for the target. The same command then exits **0** and
prints the accepted population. The ablation script carries `trap ...
EXIT INT TERM` with absolute paths, and treats an empty hash as a
failure.
- **Silent** — the whole `qa/` namespace had zero entries before that
mutation, and the unit suite asserts silence on a live refinement-free
schema (`AggregationFunction`) and on a synthetic schema whose only
constraints project.

## What this PR deliberately does NOT do

⛔ It does not narrow any published shape. Teaching the projection to
emit what a refinement constrains — `propertyNames`, `not`, `minLength`
and friends cover a lot of them — or declaring the artifact a floor,
both change a public contract. That is triage's item 2, and it stops
here with this report.

⛔ It does not delete or weaken a refinement. The runtime rule is
correct; it is the projection that is silent.

One boundary worth naming: the annotation reaches the JSON file and
**not** `content/docs/references/**` — a full `gen:docs` on this branch
produces a zero-line diff. Rendering it on the reference pages is a
docs-surface change and was left out.

## Verification

Every reading below is from commit `74b5af39e1`, the final commit on
this branch, with each exit code captured after a redirect and never
through a pipe.

| command | exit |
|---|---|
| `pnpm lint` (`eslint . --no-inline-config` — the whole repo, no
narrowing) | **0** |
| `pnpm --filter @objectstack/spec exec vitest run --project local` |
**0** — 487 files, 13900 tests |
| `pnpm --filter @objectstack/spec typecheck` (`tsc --noEmit` + scripts
+ test layers) | **0** |
| `pnpm --filter @objectstack/spec check:generated` | **0** — all 15
generated artifacts up to date |
| `pnpm --filter @objectstack/spec check:authorable-surface` (the mode
that runs this ratchet) | **0** |
| `pnpm check:published-files` · `check:type-check-coverage` ·
`check:nul-bytes` · `check:merge-driver` | **0** |
| `check-adr-0087-registration` · `check-empty-changeset` ·
`check-changeset-no-major`, each `--base origin/main` | **0** |

The 66 families derived by `node scripts/pm/dispatch-gates.mjs
--commands --repo objectstack-ai/objectstack` were all run; 62 exit 0
and the four that could not be answered locally are named in the
acceptance notes above and in the dev report on the card.

Two readings worth recording because they were NOT about this diff:
`check:published-files` first exited 1 with "`origin/main` and HEAD have
no merge base in this checkout" — a shallow clone whose graft floor had
moved past the branch point. `git fetch --deepen 300` restored the merge
base to the recorded branch point and the gate then exits 0. And a
control for every zero reading above: the control-character sweep `grep
-naP` over the five changed files exits 1 (no match) while the same
expression over a file carrying one ESC byte exits 0 and prints it.

## Acceptance notes

- **Noticed, not filed — the annotation stops at the JSON tree.**
`content/docs/references/**` renders from `packages/spec/json-schema/**`
but drops unknown `x-` keys, so a full `gen:docs` on this branch
produces a zero-line diff. An author reading the reference page still
sees nothing. Carrying it onto the page is a docs-surface change; nobody
is mid-flight in that file today.
- **Noticed, not filed — `aborting` reads the check-level flag only.**
`.refine(fn, { abort: true })` is visible; a `ctx.addIssue({ fatal: true
})` written inside a `superRefine` body is a property of the issue
raised at parse time, not of the check the graph carries, so it reads
`false`. It is a report detail and never the verdict — such a site is
still a drop. Pinned both ways in the test file.
- **Noticed, not filed — the ledger's diff amplifies.** One refinement
on a shared schema lands on every published file that embeds it, so
adding one `.refine()` can move several ledger entries at once (the
ablation moved four). That is the true fan-out, and the gate prints
every corrected entry in full, but a reviewer should expect the shape.
- **Measured, not a finding — three derived families cannot be answered
from one worktree.** `check:dual-build-cjs-loads`,
`check:lean-entry-closure` and `check:type-check-debt` answer
`PREREQUISITE NOT MET` (exit 3) because they sweep the BUILT output of
all 81 workspace packages; a spec-closure build does not satisfy them
and a full workspace build is CI's `Build Core` job.
`check:pm-dispatch-gates` is still running at 540s and is reported NOT
MEASURED, not red. All four are reported as absences, never as passes.

## Patch round 2 — `Test Core (1/6)` was red, and it was this PR's

CI on `05cdeb2611` read 29 success, 3 expected skips, one failure:
`@objectstack/spec#test:repo`, 36 tests down in
`scripts/build-schemas-check-mode.test.ts`, every one of them an
`expect(status).toBe(0)` that got a 1. The shard is green on `main`, so
it was ours.

**Why neither earlier round saw it.** `packages/spec` declares two
vitest projects and two turbo tasks: `test` is `vitest run --project
local`, `test:repo` is `vitest run --project repo`. Both rounds verified
with `test` alone. `test:repo` owns the 31 files that read outside the
package — including every fixture that spawns the real generator — and
CI runs both. The ratchet this PR adds lives in the generator, so `repo`
was exactly the project that could see it and exactly the one never run.

### Defect 1 — the census was one reading per schema-evaluation mode,
not one reading

`lazySchema()` (`src/shared/lazy-schema.ts`) returns the real schema
under `OS_EAGER_SCHEMAS=1` and a **Proxy** over it otherwise.
`gen:schema` and `check:authorable-surface` both export that flag; the
check-mode fixtures deliberately spawn the generator **without** it, and
say so in their header.

The walk keyed its visited set on the schema **instance**. So a
sub-schema reached both directly and through a `lazySchema()` edge was
**one** node to the eager walk and **two** to the lazy one. Measured on
`ui/View`, whose `list` / `listViews.valueType` and `form` /
`formViews.valueType` pairs each reach one schema by both routes: **11**
dropped sites eager, **13** lazy. Six ledger entries then disagreed with
the build in one mode and agreed in the other — the same generator, the
same tree, two censuses.

The key is now the node's `_zod` internals, with the Proxy resolved to
the internals its facade prototype-delegates to. Two keys were rejected
on measurement, and both rejections are recorded in the code:

- the **instance** — not mode-invariant, the defect above;
- the **def** — over-collapses in the other direction. `clone()` with no
argument hands a second instance the first's def object, so keying on it
dropped `…options[3].object.fields.valueType` from `system/ChangeSet`
and `system/MigrationOperation`, a reading the accepted ledger does not
make. That attempt was measured and reverted, not shipped.

`_zod` is per instance where the def is not, so it moves nothing in
eager mode and makes the lazy walk agree with it. **The committed ledger
is untouched** — no entry added, none removed, no count flattened.

### Defect 2 — the sandboxes never mounted the new ledger

Each fixture builds a temp package tree that copies `scripts/` and
symlinks `src/`, `node_modules/`, `package.json`, then seeds the
committed artefacts the generator reads.
`dropped-refinements.baseline.json` was never added to the five
builders, so the generator refused on a **missing** ledger before
reaching whatever each fixture was about. The mount is now a list of
committed package-root ledgers, so the next one is one entry rather than
a sixth call to remember.

### The pin, and its ablation

`scripts/dropped-refinements.test.ts` gains three cases: a shared
sub-schema reached by two routes is one site, the same holds across a
`lazySchema()` edge, and — the lit control — two genuinely distinct
nodes carrying the same rule are still two sites.

⚠️ The first draft of that pin **asserted nothing**, and the ablation is
what caught it: with the refinement one level down, the property schema
is the same instance by either route, so the walk deduped on that alone
and the pin passed against the very defect it was written for. Moving
the rule onto the shared node makes identity the thing under test.
Ablating the fix now reads `[ 'direct', 'viaLazy' ]` where it expects `[
'direct' ]`; restore is `git checkout HEAD -- PATH` proven by an empty
`git diff HEAD`, under a `trap ... EXIT INT TERM` with absolute paths.

### Verification — this round, at `6eeebd726d`

Exit codes captured into a file and read from `$?` after the redirect,
never through a pipe.

| command | before | after |
|---|---|---|
| `pnpm --filter @objectstack/spec test:repo` (project `repo`) | **1** —
36 failed / 500 passed | **0** — 536 passed, 31 files |
| `pnpm --filter @objectstack/spec test` (project `local`) | not
re-measured this round | **0** — 487 files, 13987 tests |
| `pnpm --filter @objectstack/spec check:generated` | not re-measured
this round | **0** — all 15 artefacts current |
| `pnpm --filter @objectstack/spec check:authorable-surface` | not
re-measured this round | **0** — 737 sites / 243 schemas |
| the same generator **without** `OS_EAGER_SCHEMAS`, tree-wide | not
measured before | **0** — 737 / 243, identical |
| `pnpm --filter @objectstack/spec typecheck` | — | **0** |
| `pnpm lint` (`eslint . --no-inline-config`, whole repo, no narrowing)
| — | **0** |
| `check:nul-bytes` · `check:cross-package-test-inputs` ·
`check:test-source-alias` · `check:type-check-coverage` | — | **0** |

The two mode readings are the load-bearing pair: 737 sites across 243
published schemas **in both modes**, where before the fix the lazy mode
disagreed with the ledger on six entries.

⚠️ **Declared narrowing.** `test:repo` was not run as a single process:
this container kills a foreground command at roughly ten minutes and
that project takes about sixteen in CI. It was run as seven foreground
chunks of the same project and config — one for the 30 other files (451
tests), six covering all 14 describe blocks of
`build-schemas-check-mode.test.ts` (85 tests). 451 + 85 = 536, which is
the count CI reports for the project. No chunk was skipped and none
reported a failure.

⚠️ **NOT MEASURED, reported as an absence and not as a pass:**
`check:type-check-debt` answers `PREREQUISITE NOT MET` (exit **3**, its
own code for "nothing was measured") because its `--re-measure` half
needs the built `dist` of 30 workspace dependencies; its coverage half,
`check:type-check-coverage`, exits 0. A full workspace build is CI's
job, and the gate's own text says so.

⚠️ The round-1 verification table above this section is a reading from
`74b5af39e1` and is left as written. Its census numbers — 240 schemas /
688 sites — are that commit's; the 243 / 737 here is the same
measurement after `a0127cfe26` recorded the 49 sites a sibling landing
added.

### Acceptance notes — patch round 2

- **Noticed, not filed — a new package-root ledger is still a manual
mount.** The list makes it one line instead of five, but nothing
mechanically holds the list equal to the set of ledgers the generator
actually reads; the next one is caught by the same 36 red fixtures
rather than by a gate that names it. Naming it would need the
generator's read set derived statically. Nobody is mid-flight in these
builders today.
- **Noticed, not filed — the census's mode-invariance is pinned on a
synthetic graph, not on the live one.** The live discriminator would be
spawning the generator twice per run, once per mode, which doubles the
slowest gate in the package to pin a property the unit pin already fails
on.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01LvwGppdonww4zGLWZo5rho)_

---
_Generated by [Claude Code](https://claude.ai/code)_

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: os-litant

.changeset/spooky-poems-repeat.md

@@ -0,0 +1,22 @@
+---
+'@objectstack/spec': patch
+---
+
+Say it out loud when a `.refine()` never reaches the published JSON Schema.
+
+`z.toJSONSchema()` has no arm for a `custom` check, so every rule written as a
+`.refine()` / `.superRefine()` is enforced by the runtime and absent from the
+`json-schema/` tree that ships inside this package — a published file that is
+WIDER than the Zod type it was generated from, in the direction where an
+author's (or an AI's) validator says yes and the platform then says no. Measured
+on zod 4.4.3: 688 refinement sites across 240 published schemas, none of which
+projected anything.
+
+Nothing about what the schemas accept changes. Each affected file now carries an
+`x-dropped-refinements` annotation naming the paths whose rules it does not
+state — `x-` keywords are ignored by every validator, so the accepted document
+set is byte-for-byte what it was — and the generator reports the population on
+every run and refuses to grow it silently
+(`packages/spec/dropped-refinements.baseline.json`).
+
+Clause-②: no