diff --git a/.changeset/9102-import-boundary-registry-meta.md b/.changeset/9102-import-boundary-registry-meta.md new file mode 100644 index 0000000000..76d8d33f7c --- /dev/null +++ b/.changeset/9102-import-boundary-registry-meta.md @@ -0,0 +1,39 @@ +--- +'@object-ui/types': minor +--- + +Carry the protocol's registry metadata across both schema derivations, not just the +description (objectui#9102). + +**Emitted-surface change.** `stripImportedDefaults` and `deriveStrictAuthoringSchema` +both derive new zod graphs by patching a copy of a node's `_zod.def` and calling its own +constructor. A zod 4 description — and every other registry key — is not `def` state: it +lives in `z.globalRegistry`, keyed by the node. So a def-copying rebuild reproduced `def` +faithfully and reproduced the node's metadata not at all. + +objectui#9086 repaired the description at one of the two sites. This change carries the +rest of the vocabulary and repairs the other site. + +What moves on the emitted surface: + +- **The import boundary.** `z.toJSONSchema` on a `@objectstack/spec` datasource config + emitted `{description, type}` for `host` where the spec emits `{default, description, + title, type}`. It now emits `{description, title, type}`. `title` and + `externalVocabulary` were being dropped from every `ZodDefault` node that carried them. + `default` stays absent, deliberately: not substituting an author's omitted keys is + decision batch #90, and the pin file asserts the carry did not quietly undo it. +- **The strict authoring face.** It rebuilds every container it walks, so it kept only + the descriptions on leaves it returned untouched. Every described container on the node + face — the great majority of them — arrived on the derived twin with none. + +The carry set is **bounded and enumerated** (`CARRIED_REGISTRY_META_KEYS`), not a blanket +spread, and `id` is refused by name: `globalRegistry.add()` writes the registry's shared +`_idmap` whenever the metadata it is handed contains one, so carrying an `id` would +repoint a global id map at this package's derived node. A census re-derives the key +vocabulary over every published spec subpath and fails when the protocol carries a key on +neither list, so the bound costs boundedness and not fidelity. + +**No accept set moves, and nothing is mutated.** `.meta()` clones, so the spec's own +objects — which the derived node literally is on the boundary's already-optional branch — +are left as they were found. A subtree with no `ZodDefault` still comes back +reference-equal. diff --git a/packages/types/src/__tests__/registry-meta-carry-9102.test.ts b/packages/types/src/__tests__/registry-meta-carry-9102.test.ts new file mode 100644 index 0000000000..0b392d509f --- /dev/null +++ b/packages/types/src/__tests__/registry-meta-carry-9102.test.ts @@ -0,0 +1,950 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * THE IMPORT BOUNDARY CONVEYS THE PROTOCOL'S REGISTRY METADATA, AT BOTH SITES + * (objectui#9102). + * + * objectui#9086 taught `../zod/imported-defaults.ts` to carry a node's + * DESCRIPTION across a derivation. objectui#9102 measured the two things that + * fix left standing: + * + * 1. **The carry was one key wide, and the wrong key was defended.** The + * boundary's docblock enshrined "the description and nothing else" on a + * rationale about `id` — and on this surface `id` does not occur at all, + * while `title` and `externalVocabulary` sit on `ZodDefault` nodes the + * walker unwraps. `@objectstack/spec` emits `{default, description, title, + * type}` for a datasource `host`; this package emitted `{description, + * type}`. ⭐ Narrower than the protocol on a published surface is the + * direction the platform has ruled against. + * 2. **The sibling rebuild was never widened.** `../strict-authoring-face.ts` + * carried the identical `new Ctor({...def, ...patch})` spelling in its own + * local copy, and it rebuilds EVERY container it walks — so the derived + * strict twin dropped the descriptions this repository's own mirrors + * declare, wholesale. + * + * Both now go through one helper, `../zod/node-derivation.ts`. That sharing is + * itself asserted below, because two copies is how the two drifted apart. + * + * ## What this file pins + * + * 1. **The zod facts the carry rests on.** If a later zod puts registry state + * in `def`, stops cloning on `.meta()`, or stops writing `_idmap` from + * `add()`, the design below changes — that must be RED here, not discovered + * years later by someone re-deriving it. + * 2. **⭐ The carry set is a real bound AND the protocol fits inside it.** The + * vocabulary is enumerated rather than spread, so the census re-derives + * every registry key on every published spec subpath and fails when one is + * on neither the carry list nor the refusal list. Without that half, a + * bounded list silently re-creates the defect one key later. + * 3. **⛔ `id` is refused, and `_idmap` is not touched.** Not "the spec happens + * not to use it here" — that is a fact about today's spec. + * 4. **Both sites, re-derived.** The published spec surface for site ①, the + * published node face for site ②. + * 5. **⭐ The identity property, which this fix must not buy its way past.** A + * subtree with no `ZodDefault` still comes back REFERENCE-EQUAL. + * 6. **⛔ The spec's graph is not mutated.** + * + * Every count below carries a control that FIRES. A census that matched nothing + * is green for reasons that have nothing to do with objectui#9102. + */ + +import { describe, it, expect } from 'vitest'; +import { z } from 'zod'; +import { readdirSync, readFileSync } from 'node:fs'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { + CARRIED_REGISTRY_META_KEYS, + REFUSED_REGISTRY_META_KEYS, + carryRegistryMeta, + cloneWithDef, +} from '../zod/node-derivation.js'; +import { stripImportedDefaults } from '../zod/imported-defaults.js'; +// ⚠️ THROUGH THE BARREL, deliberately. `../strict-authoring-face.ts` is the deep +// module of a declared module cycle, and `strict-authoring-face-8345.test.ts` +// pins the barrel as its SOLE entry — a direct specifier here turns that pin +// red, which is how this import was caught. The source-reading assertions at +// the bottom of this file address that module by PATH, never by specifier. +import { deriveStrictAuthoringSchema } from '../zod/index.zod.js'; +import { SchemaNodeSchema } from '../zod/base.zod.js'; + +const HERE = dirname(fileURLToPath(import.meta.url)); +const SRC_DIR = join(HERE, '..'); + +/* ── the graph reader ─────────────────────────────────────────────────────── */ + +interface ZodDef { + type: string; + shape?: Record; + options?: z.ZodType[]; + items?: z.ZodType[]; + element?: z.ZodType; + rest?: z.ZodType; + valueType?: z.ZodType; + left?: z.ZodType; + right?: z.ZodType; + in?: z.ZodType; + out?: z.ZodType; + innerType?: z.ZodType; + getter?: () => z.ZodType; +} +const defOf = (node: z.ZodType): ZodDef => (node as unknown as { _zod: { def: ZodDef } })._zod.def; +const isZod = (v: unknown): v is z.ZodType => + v !== null && (typeof v === 'object' || typeof v === 'function') && '_zod' in (v as object); +/** + * The node's registry metadata AS THE CARRY READS IT — through the published + * accessor, ⛔ never through a registry lookup keyed by the node object. + * + * ⭐ The two are different readings on this surface, and that is the whole of + * objectui#9102's second round. `@objectstack/spec` publishes most schemas as + * lazy cross-module `new Proxy(functionTarget, …)` wrappers; the proxy's `get` + * trap resolves the real schema and BINDS any function it returns, so + * `node.meta()` runs `real.meta()` and answers the real's entry, while + * `z.globalRegistry.get(proxy)` answers only the real's ANCESTORS — nothing + * ever registered the proxy itself. A census written on the map route is blind + * to every proxied node's own metadata and reports a clean surface it never + * looked at. + */ +const metaOf = (node: z.ZodType): Record | undefined => + node.meta() as Record | undefined; + +/** The map route, kept ONLY so the differential between the two can be asserted. */ +const metaViaRegistryMap = (node: z.ZodType): Record | undefined => + z.globalRegistry.get(node) as Record | undefined; + +/** A node's zod traits — the set `$constructor` stamps on every instance. */ +const traitsOf = (node: z.ZodType): string[] => [ + ...(node as unknown as { _zod: { traits: Set } })._zod.traits, +]; + +/** + * Is this node one of `@objectstack/spec`'s lazy cross-module proxies? + * + * ⛔ NOT a `typeof === 'function'` test ALONE. That test opens the probe below, + * but as a cheap pre-filter rather than as the discriminator — nothing zod + * builds is callable, so it is also the step that already answers `false` for a + * `$ZodObjectJIT` instance. Such an instance is an ORDINARY OBJECT, EVERY `z.object()` carries + * that trait, and the "JIT" names eval-compiled PARSE CODE; the trait is + * therefore no discriminator either, since the wrapper forwards `_zod` — traits + * included — to the real schema behind it and answers `$ZodObjectJIT` right + * along with it. ⭐ The discriminating probe is the proxy invariant: the wrapper + * installs an `ownKeys` trap over a FUNCTION target, so + * `Object.getOwnPropertyNames` cannot satisfy the invariant and THROWS, where + * an ordinary node answers normally. Both halves are re-derived below rather + * than asserted here, in the control this docblock's claims rest on. + * + * ⚠️ Also spelled as a helper so TypeScript does not narrow the argument to + * `never` at the call site: `z.ZodType` is not declared callable, so an inline + * `typeof n === 'function'` makes every later property read an error on a node + * that answers the guard perfectly well at runtime. + */ +const isSpecLazyProxy = (node: z.ZodType): boolean => { + if (typeof node !== 'function') return false; + try { + Object.getOwnPropertyNames(node); + return false; + } catch { + return true; + } +}; + +/** Children of a node, labelled so a derived twin's matching child can be found. */ +const childrenOf = (s: z.ZodType): [string, z.ZodType][] => { + const d = defOf(s); + const out: [string, z.ZodType][] = []; + switch (d.type) { + case 'object': for (const [k, v] of Object.entries(d.shape ?? {})) out.push([`.${k}`, v]); break; + case 'union': (d.options ?? []).forEach((o, i) => out.push([`|${i}`, o])); break; + case 'array': if (d.element) out.push(['[]', d.element]); break; + case 'tuple': + (d.items ?? []).forEach((it, i) => out.push([`[${i}]`, it])); + if (d.rest) out.push(['[...]', d.rest]); + break; + case 'record': if (d.valueType) out.push(['{}', d.valueType]); break; + case 'intersection': + if (d.left) out.push(['&L', d.left]); + if (d.right) out.push(['&R', d.right]); + break; + case 'pipe': + if (d.in) out.push(['>in', d.in]); + if (d.out) out.push(['>out', d.out]); + break; + case 'lazy': if (d.getter) out.push(['~lazy', d.getter()]); break; + case 'default': case 'optional': case 'nullable': + case 'nonoptional': case 'readonly': case 'catch': + if (d.innerType) out.push([`(${d.type})`, d.innerType]); + break; + default: break; + } + return out; +}; + +const reaches = (root: z.ZodType, type: string): boolean => { + const seen = new Set(); + const stack = [root]; + let budget = 20_000; + while (stack.length && budget-- > 0) { + const n = stack.pop()!; + if (seen.has(n)) continue; + seen.add(n); + if (defOf(n).type === type) return true; + for (const [, c] of childrenOf(n)) stack.push(c); + } + return false; +}; + +/** + * The import boundary's SECOND identity-property exception, carried by + * objectui#9088 and deliberately not repaired here: zod spells "no rest + * element" as `def.rest === null`, the walker compares it against `undefined`, + * so every rest-less tuple is rebuilt whether or not anything changed beneath + * it. Excused below exactly as the objectui#9034 pin excuses it. + */ +const hasRestlessTuple = (root: z.ZodType): boolean => { + const seen = new Set(); + const stack = [root]; + let budget = 20_000; + while (stack.length && budget-- > 0) { + const n = stack.pop()!; + if (seen.has(n)) continue; + seen.add(n); + const d = defOf(n); + if (d.type === 'tuple' && (d as { rest?: unknown }).rest !== undefined && d.rest === null) return true; + if (d.type === 'lazy') continue; + for (const [, c] of childrenOf(n)) stack.push(c); + } + return false; +}; + +/* ── the published spec surface, read out of the spec's own `exports` map ──── */ + +interface Census { + subpathsDeclared: number; + subpathsLoaded: number; + loadFailures: string[]; + roots: [string, z.ZodType][]; + /** Every registry key seen on the surface (accessor route) -> distinct nodes carrying it. */ + keyPopulation: Map; + /** The same census taken on the registry-map route, so the blind spot is measurable. */ + keyPopulationViaMap: Map; + /** Nodes carrying a key other than `description`, by the path they were found at. */ + nonDescriptionNodes: string[]; + /** `@objectstack/spec` lazy proxies reached on this surface. */ + proxyNodes: number; + /** Proxies whose accessor route answers metadata at all. */ + proxiesWithMeta: number; + /** Of those, how many the registry-map route reads IDENTICALLY, and how many it does not. */ + proxyMetaMapAgrees: number; + proxyMetaMapDisagrees: number; + /** Proxies carrying non-`description` metadata — the population the map route would silently drop. */ + proxyNonDescriptionNodes: string[]; + /** Nodes where `.description` answers but `.meta()?.description` does not — the retired fallback's occasions. */ + descriptionFallbackOccasions: string[]; + /** Registry entry + parent + def snapshots that MOVED across both derivations. */ + mutatedNodes: string[]; + snapshottedNodes: number; + /** Rebuilt nodes whose carried keys survived, and those that did not. */ + rebuiltWithMeta: number; + rebuiltWithMetaKept: number; + rebuiltWithMetaLost: string[]; + nodesVisited: number; +} + +/** + * Built at MODULE SCOPE, not in a `beforeAll`. The census dynamically imports + * every published spec subpath, and a cold Vite transform inside a hook is + * billed to `hookTimeout`, so the test would pass or fail on machine load + * rather than on the code it covers. At module scope the same cost lands in the + * import phase, which no timeout applies to — which is also what the + * `object-ui/no-dynamic-import-in-test-hook` rule enforces. + */ +const buildCensus = async (): Promise => { + const pkg = (await import('@objectstack/spec/package.json', { with: { type: 'json' } })) as { + default: { exports: Record }; + }; + const subpaths = Object.keys(pkg.default.exports).filter( + (k) => k !== './package.json' && k !== './openapi.json', + ); + + const loadFailures: string[] = []; + const roots: [string, z.ZodType][] = []; + let subpathsLoaded = 0; + for (const sp of subpaths) { + const specifier = sp === '.' ? '@objectstack/spec' : `@objectstack/spec/${sp.slice(2)}`; + let mod: Record; + try { + mod = (await import(/* @vite-ignore */ specifier)) as Record; + } catch (err) { + loadFailures.push(`${specifier}: ${String(err).split('\n')[0]}`); + continue; + } + subpathsLoaded++; + for (const [name, value] of Object.entries(mod)) { + if (isZod(value)) roots.push([`${specifier}#${name}`, value]); + } + } + + const keyPopulation = new Map(); + const keyPopulationViaMap = new Map(); + const nonDescriptionNodes: string[] = []; + const proxyNonDescriptionNodes: string[] = []; + const descriptionFallbackOccasions: string[] = []; + let proxyNodes = 0; + let proxiesWithMeta = 0; + let proxyMetaMapAgrees = 0; + let proxyMetaMapDisagrees = 0; + const seenKeys = new Set(); + let nodesVisited = 0; + + // Every node's registry entry, `_zod.parent` and def identity BEFORE either + // derivation runs, so "the spec's graph is left as it was found" can be + // asserted as a real differential instead of proxied through a side effect. + const snapshot = new Map(); + const stateOf = (node: z.ZodType): string => + JSON.stringify({ + meta: metaOf(node) ?? null, + map: metaViaRegistryMap(node) ?? null, + hasParent: (node as unknown as { _zod: { parent?: unknown } })._zod.parent !== undefined, + type: defOf(node).type, + keys: defOf(node).shape ? Object.keys(defOf(node).shape!).join(',') : null, + }); + + const censusKeys = (node: z.ZodType, path: string, depth: number): void => { + if (depth > 60 || seenKeys.has(node)) return; + seenKeys.add(node); + nodesVisited++; + snapshot.set(node, stateOf(node)); + + const meta = metaOf(node); + const viaMap = metaViaRegistryMap(node); + const proxy = isSpecLazyProxy(node); + if (proxy) proxyNodes++; + + if (meta) { + for (const key of Object.keys(meta)) keyPopulation.set(key, (keyPopulation.get(key) ?? 0) + 1); + if (Object.keys(meta).some((k) => k !== 'description')) { + nonDescriptionNodes.push(path); + if (proxy) proxyNonDescriptionNodes.push(path); + } + if (proxy) { + proxiesWithMeta++; + if (JSON.stringify(viaMap ?? null) === JSON.stringify(meta)) proxyMetaMapAgrees++; + else proxyMetaMapDisagrees++; + } + } + if (viaMap) { + for (const key of Object.keys(viaMap)) keyPopulationViaMap.set(key, (keyPopulationViaMap.get(key) ?? 0) + 1); + } + if (node.description !== undefined && meta?.description === undefined) { + descriptionFallbackOccasions.push(path); + } + + for (const [label, c] of childrenOf(node)) censusKeys(c, `${path}${label}`, depth + 1); + }; + for (const [name, root] of roots) censusKeys(root, name, 0); + + // The differential: every node the walk REBUILT, against its source. + let rebuiltWithMeta = 0; + let rebuiltWithMetaKept = 0; + const rebuiltWithMetaLost: string[] = []; + const seenPair = new Set(); + const pair = (before: z.ZodType, after: z.ZodType, path: string, depth: number): void => { + if (depth > 60 || seenPair.has(before)) return; + seenPair.add(before); + if (before === after) return; + + const bd = defOf(before); + const bm = metaOf(before) ?? {}; + const am = (isZod(after) ? metaOf(after) : undefined) ?? {}; + const carriedKeys = Object.keys(bm).filter((k) => CARRIED_REGISTRY_META_KEYS.includes(k)); + if (carriedKeys.length) { + rebuiltWithMeta++; + if (carriedKeys.every((k) => JSON.stringify(am[k]) === JSON.stringify(bm[k]))) rebuiltWithMetaKept++; + else rebuiltWithMetaLost.push(`${path} [${bd.type}] ${JSON.stringify(bm)} -> ${JSON.stringify(am)}`); + } + + if (bd.type === 'default') { + const ad = isZod(after) ? defOf(after) : undefined; + const inner = bd.innerType!; + if (ad?.type === 'optional' && ad.innerType) pair(inner, ad.innerType, `${path}(default)`, depth + 1); + else if (isZod(after)) pair(inner, after, `${path}(default)`, depth + 1); + return; + } + + const aMap = new Map(isZod(after) ? childrenOf(after) : []); + for (const [label, child] of childrenOf(before)) { + const twin = aMap.get(label); + if (twin) pair(child, twin, `${path}${label}`, depth + 1); + } + }; + for (const [name, root] of roots) pair(root, stripImportedDefaults(root), name, 0); + + // ⛔ THE NON-MUTATION DIFFERENTIAL, taken AFTER both derivations have run over + // every root. The snapshot above was taken before either did, so a carry that + // wrote metadata in place — rather than onto a clone — moves a value here. + // The previous spelling of this pin watched a SIDE EFFECT (does the source + // still hold a default?) and stayed green under a mutating carry, because a + // mutating carry does not remove defaults. This one reads the thing it names. + for (const [, root] of roots) deriveStrictAuthoringSchema(root); + const mutatedNodes: string[] = []; + for (const [node, before] of snapshot) { + if (stateOf(node) !== before) mutatedNodes.push(`${defOf(node).type} :: ${before}`); + } + + return { + subpathsDeclared: subpaths.length, + subpathsLoaded, + loadFailures, + roots, + keyPopulation, + keyPopulationViaMap, + nonDescriptionNodes, + proxyNodes, + proxiesWithMeta, + proxyMetaMapAgrees, + proxyMetaMapDisagrees, + proxyNonDescriptionNodes, + descriptionFallbackOccasions, + mutatedNodes, + snapshottedNodes: snapshot.size, + rebuiltWithMeta, + rebuiltWithMetaKept, + rebuiltWithMetaLost, + nodesVisited, + }; +}; + +const census: Census = await buildCensus(); + +/* ── 1. the zod facts the carry rests on ──────────────────────────────────── */ + +describe('the zod 4 facts the carry rests on (objectui#9102)', () => { + it('registry metadata is NOT `def` state, so a def-copying rebuild carries none of it', () => { + const node = z.object({ k: z.string() }).meta({ title: 'T', description: 'D', xRef: 'R' }); + expect(metaOf(node)).toMatchObject({ title: 'T', description: 'D', xRef: 'R' }); + expect( + 'title' in (defOf(node) as unknown as Record), + 'zod now keeps registry metadata in `def` — the explicit carry may be redundant. Re-measure before deleting it.', + ).toBe(false); + + const Ctor = (node as unknown as { constructor: new (d: ZodDef) => z.ZodType }).constructor; + const raw = new Ctor({ ...defOf(node) }); + expect( + metaOf(raw), + 'a def-copying rebuild now carries registry metadata by itself — `carryRegistryMeta` may be redundant', + ).toBeUndefined(); + }); + + it('⭐ `.meta()` CLONES, so carrying metadata never mutates its target', () => { + // The whole non-mutation argument rests on this. The derived node IS one of + // the spec's own objects on the boundary's already-optional branch, so a + // carry that wrote in place would relabel `@objectstack/spec` for every + // other consumer in the workspace. + const target = z.string().optional(); + const carried = carryRegistryMeta(z.string().meta({ title: 'T' }), target); + expect(carried).not.toBe(target); + expect(metaOf(target), 'the target was relabelled in place').toBeUndefined(); + expect(metaOf(carried)).toMatchObject({ title: 'T' }); + }); + + it('⛔ `globalRegistry.add()` writes `_idmap` when — and only when — the metadata carries an `id`', () => { + // This is the mechanism `id` is refused on. If zod stops writing `_idmap` + // from `add()`, the refusal is no longer load-bearing and should be + // re-argued rather than inherited. + const marker = `objectui-9102-idmap-${Math.random().toString(36).slice(2)}`; + expect(z.globalRegistry._idmap.has(marker)).toBe(false); + z.string().meta({ title: 'no id here' }); + expect(z.globalRegistry._idmap.has(marker), 'the control is not measuring what it claims').toBe(false); + const withId = z.string().meta({ id: marker }); + expect( + z.globalRegistry._idmap.get(marker), + 'zod no longer indexes `id` in `_idmap` — re-argue REFUSED_REGISTRY_META_KEYS rather than inheriting it', + ).toBe(withId); + }); + + it('`globalRegistry.get()` inherits down a parent chain and declines to inherit `id`', () => { + const marker = `objectui-9102-inherit-${Math.random().toString(36).slice(2)}`; + const base = z.string().meta({ id: marker, title: 'T' }); + const child = base.describe('D'); + expect(metaOf(child)).toMatchObject({ title: 'T', description: 'D' }); + expect( + metaOf(child)?.id, + 'zod now inherits `id` down a parent chain — every derived node would claim its source\'s id', + ).toBeUndefined(); + }); + + it('⭐ nothing zod builds is callable — the callables on this surface are SPEC PROXIES', () => { + // objectui#9102's first round blamed `$ZodObjectJIT` for the callables it + // met, and a second round then said the trait was ABSENT here. Both clauses + // are false, so the fact is RE-DERIVED here rather than asserted: it is the + // sentence `../zod/node-derivation.ts` rests its guard's rationale on, and + // a docblock that diagnoses the wrong mechanism is what this card exists to + // stop. A `$ZodObjectJIT` instance is an ordinary object, every `z.object()` + // carries the trait, and the "JIT" names eval-compiled parse code. + // + // The separator is the proxy invariant instead: `@objectstack/spec` wraps + // schemas in `new Proxy(functionTarget, …)`, and that wrapper's `ownKeys` + // trap cannot satisfy the proxy invariant over a function target, so + // `Object.getOwnPropertyNames` THROWS. An ordinary node answers normally — + // asserted here as the firing control, so `isSpecLazyProxy` cannot be + // passing by answering `true` to everything. + const plainObject = z.object({ k: z.string() }); + expect( + typeof plainObject, + 'a `z.object()` is callable after all — every docblock here that calls the callables PROXIES is stale', + ).toBe('object'); + expect( + traitsOf(plainObject), + 'a plain `z.object()` no longer carries the `$ZodObjectJIT` trait — the claim that EVERY one does, ' + + 'and with it the reason the trait cannot separate proxies from ordinary nodes, no longer holds', + ).toContain('$ZodObjectJIT'); + expect(Object.getOwnPropertyNames(plainObject), 'the control node is not inspectable').toBeInstanceOf(Array); + expect(isSpecLazyProxy(plainObject), 'the probe answers `true` for an ordinary node').toBe(false); + + const proxies = census.roots.filter(([, r]) => isSpecLazyProxy(r)); + expect( + proxies.length, + 'no spec root is a lazy proxy any more — either the spec stopped wrapping, or this probe broke. ' + + 'Everything below about the accessor route rests on this population.', + ).toBeGreaterThan(0); + expect(() => Object.getOwnPropertyNames(proxies[0]![1])).toThrow(/ownKeys/); + expect( + proxies.some(([, r]) => traitsOf(r).includes('$ZodObjectJIT')), + 'no spec proxy forwards the real\'s `$ZodObjectJIT` trait any more — the trait would then separate ' + + 'proxies from ordinary nodes after all, and both docblocks understate what the probe is for', + ).toBe(true); + }); + + it('⭐ `.meta()` and a registry lookup DISAGREE through a spec proxy, and `.meta()` is the true one', () => { + // Why `carryRegistryMeta` reads `source.meta()`. The proxy's `get` trap + // resolves the real schema and BINDS the function it hands back, so + // `proxy.meta()` runs `real.meta()`. A registry lookup keyed by the proxy + // cannot reach that: nothing registered the proxy, and the map's + // parent-chain walk arrives only at the real's ANCESTORS. + expect( + census.proxiesWithMeta, + 'no proxy on the published spec surface carries metadata — the accessor route is untested here', + ).toBeGreaterThan(0); + expect( + census.proxyMetaMapDisagrees, + 'the registry-map route now agrees with the accessor on every proxy — the spec may have stopped ' + + 'wrapping, and `carryRegistryMeta` reading `.meta()` would no longer be load-bearing. Re-measure.', + ).toBeGreaterThan(0); + // The map route is not merely different, it is POORER: it sees strictly + // fewer descriptions than the accessor over the same walk. + expect( + (census.keyPopulation.get('description') ?? 0) - (census.keyPopulationViaMap.get('description') ?? 0), + 'the two routes now see the same number of descriptions — the blind spot this file measures is gone', + ).toBeGreaterThan(0); + }); + + it('the retired `.description` fallback would have no occasion to fire', () => { + // The first round carried `source.description` when the registry view was + // silent. Once the carry reads `.meta()`, that branch is dead — and dead + // structurally, not by luck: zod's `description` getter IS + // `globalRegistry.get(inst)?.description` for the instance it was installed + // on, and through a proxy both routes resolve to the same real. Measured + // rather than argued, so re-adding the branch has to answer this number. + expect(census.descriptionFallbackOccasions.slice(0, 10)).toEqual([]); + }); +}); + +/* ── 2. the carry set is a real bound, and the protocol fits inside it ─────── */ + +describe('⭐ the carry set is bounded AND complete for the protocol (objectui#9102)', () => { + it('positive control — the census loaded a real surface, with nothing swallowed', () => { + expect(census.loadFailures, 'a subpath failed to load and was counted as clean').toEqual([]); + expect(census.subpathsLoaded).toBe(census.subpathsDeclared); + expect(census.subpathsDeclared, 'the spec publishes far fewer subpaths than it did').toBeGreaterThan(12); + expect(census.roots.length, 'no schema-shaped exports found — every count below is vacuous').toBeGreaterThan(500); + expect(census.nodesVisited, 'the walk barely moved — every count below is vacuous').toBeGreaterThan(5_000); + }); + + it('positive control — the surface really does carry keys beyond `description`', () => { + expect( + census.nonDescriptionNodes.length, + 'nothing on the published spec surface carries registry metadata other than `description` — ' + + 'the whole carry set below is untested and objectui#9102 would have nothing to fix', + ).toBeGreaterThan(0); + expect( + [...census.keyPopulation.keys()].filter((k) => k !== 'description').length, + 'only one distinct key beyond `description` — the enumeration is barely exercised', + ).toBeGreaterThan(1); + }); + + it('⭐ every registry key the protocol publishes is either CARRIED or REFUSED by name', () => { + // The half that makes a bounded list safe. A bounded carry set drops a NEW + // key silently, which is the same "narrower than the protocol" defect + // objectui#9102 closes, one key later. This converts that silent drop into + // a failing gate: the day `@objectstack/spec` carries a key on neither + // list, someone decides deliberately whether it belongs in + // CARRIED_REGISTRY_META_KEYS. + // + // ⚠️ Read through `.meta()` — the same route the carry reads. The first + // round took this census on `z.globalRegistry.get(node)`, which cannot see + // a spec proxy's own entry at all, so it reported a vocabulary it had never + // looked at for most of the surface. The proxy population is asserted + // non-empty above, which is what stops this from silently reverting to the + // narrower reading. + const classified = new Set([...CARRIED_REGISTRY_META_KEYS, ...REFUSED_REGISTRY_META_KEYS]); + const unclassified = [...census.keyPopulation.keys()].filter((k) => !classified.has(k)).sort(); + expect( + unclassified, + 'the protocol publishes a registry key this package neither carries nor refuses. Decide which it is ' + + 'and add it to CARRIED_REGISTRY_META_KEYS or REFUSED_REGISTRY_META_KEYS in `../zod/node-derivation.ts`.', + ).toEqual([]); + // The map route would have missed nodes outright, not just keys. Stated as + // a differential so "the census looked at everything" is measured. + expect( + (census.keyPopulation.get('description') ?? 0), + 'the accessor census sees no more than the map census — the blind spot is unmeasured here', + ).toBeGreaterThan(census.keyPopulationViaMap.get('description') ?? 0); + }); + + it('⛔ `id` is refused rather than merely absent, and every carried key is really used', () => { + expect(REFUSED_REGISTRY_META_KEYS).toContain('id'); + expect(CARRIED_REGISTRY_META_KEYS).not.toContain('id'); + // Every key on the carry list is a key the protocol actually publishes here. + // A list that outgrew its surface reads as measured and is not. + const unused = CARRIED_REGISTRY_META_KEYS.filter((k) => !census.keyPopulation.has(k)); + expect( + unused, + 'a key on CARRIED_REGISTRY_META_KEYS no longer occurs anywhere on the published spec surface — ' + + 'it is being carried on a claim nothing re-derives', + ).toEqual([]); + }); + + it('⚠️ the accessor census REACHES the proxies — carriage itself is pinned by the control below', () => { + // ⛔ This is NOT "zero such nodes, therefore safe" — that was the first + // round's assertion and it could never be non-empty, because it asked the + // registry map about an object the registry has never heard of. It now asks + // the accessor route, which is the one that answers. + // + // ⚠️ What the two assertions below provide is that the census REACHES + // proxies at all, and that the paths it produced for the non-`description` + // population are well formed. They do ⛔ NOT provide carriage, and this + // test's title used to claim they did: the population is empty TODAY (every + // metadata-bearing proxy carries `description` only), so an assertion over + // it would assert over nothing and stay green whatever the carry did. + // + // Carriage is pinned by the hand-built proxy control further down — + // `⭐ a PROXIED node's metadata survives the carry — and the map route would + // have dropped it` — which puts a `title` on a proxied node and measures + // that the carry reproduces it, and would have failed on the map route. + for (const path of census.proxyNonDescriptionNodes.slice(0, 10)) { + expect(typeof path, 'the census produced a malformed path').toBe('string'); + } + expect( + census.proxyNodes, + 'no proxy was reached at all — this assertion and the one above it are both vacuous', + ).toBeGreaterThan(0); + }); +}); + +/* ── 3. hand-built controls, both sites, known to fire ────────────────────── */ + +describe('hand-built controls — each fires on the region it tests', () => { + it('site ① — a `ZodDefault` member keeps `title` and `externalVocabulary` across the strip', () => { + const src = z.object({ + k: z.string().default('x').meta({ description: 'D', title: 'T', externalVocabulary: 'V' }), + }); + expect(defOf(defOf(src).shape!.k).type, 'the control does not exercise the `default` arm').toBe('default'); + + const out = stripImportedDefaults(src); + expect(defOf(defOf(out).shape!.k).type, 'the default survived the strip').not.toBe('default'); + expect(metaOf(defOf(out).shape!.k)).toMatchObject({ description: 'D', title: 'T', externalVocabulary: 'V' }); + }); + + it('site ① — a rebuilt CONTAINER keeps its own metadata', () => { + const src = z.object({ k: z.string().default('x') }).meta({ title: 'CONTAINER-T', xRef: 'CONTAINER-R' }); + const out = stripImportedDefaults(src); + expect(out, 'the control does not exercise `cloneWithDef` — nothing was rebuilt').not.toBe(src); + expect(metaOf(out)).toMatchObject({ title: 'CONTAINER-T', xRef: 'CONTAINER-R' }); + }); + + it('site ② — a rebuilt container keeps its description and its metadata', () => { + const src = z.object({ k: z.string().describe('LEAF') }).meta({ description: 'CONTAINER', title: 'T' }); + const out = deriveStrictAuthoringSchema(src); + expect(out, 'the control does not exercise the rebuild — nothing was derived').not.toBe(src); + expect(out.description).toBe('CONTAINER'); + expect(metaOf(out)).toMatchObject({ title: 'T' }); + expect(defOf(out).shape!.k.description, 'the leaf lost its own description').toBe('LEAF'); + }); + + it('site ② — the `z.lazy` arm carries metadata too', () => { + // The live population of described `z.lazy` nodes on this face is empty, so + // this is a hand-built control rather than a census that would assert + // nothing. + const src = z.lazy(() => z.object({ k: z.string() })).meta({ description: 'LAZY', title: 'LT' }); + const out = deriveStrictAuthoringSchema(src); + expect(defOf(out).type, 'the control does not exercise the `lazy` arm').toBe('lazy'); + expect(metaOf(out)).toMatchObject({ description: 'LAZY', title: 'LT' }); + }); + + it('⛔ neither site emits `id`, and neither repoints `_idmap`', () => { + const marker = `objectui-9102-carry-${Math.random().toString(36).slice(2)}`; + const src = z.object({ k: z.string().default('x') }).meta({ id: marker, title: 'T' }); + expect(z.globalRegistry._idmap.get(marker), 'the source did not register its own id').toBe(src); + + for (const derived of [stripImportedDefaults(src), deriveStrictAuthoringSchema(src)]) { + expect(derived, 'the control does not exercise a rebuild').not.toBe(src); + expect(metaOf(derived)).toMatchObject({ title: 'T' }); + expect(metaOf(derived)?.id, 'a derivation emitted an `id`').toBeUndefined(); + expect( + z.globalRegistry._idmap.get(marker), + 'the registry\'s id map now points at this package\'s derivation instead of the source', + ).toBe(src); + } + }); + + it('the carry INVENTS nothing where the source carried nothing', () => { + const bare = z.string().optional(); + expect(carryRegistryMeta(z.string(), bare), 'a metadata-free source still produced a clone').toBe(bare); + const src = z.object({ k: z.string().default('x') }); + expect(metaOf(stripImportedDefaults(src))).toBeUndefined(); + }); + + it('⭐ a subtree with NO default comes back REFERENCE-EQUAL, metadata intact', () => { + const src = z.object({ k: z.string().describe('INNER') }).meta({ title: 'IDENTITY' }); + expect(stripImportedDefaults(src)).toBe(src); + expect(metaOf(stripImportedDefaults(src))).toMatchObject({ title: 'IDENTITY' }); + }); + + it('⭐ a PROXIED node\'s metadata survives the carry — and the map route would have dropped it', () => { + // The control that makes the proxy half of this file real rather than + // descriptive. `@objectstack/spec`'s wrapper shape, reduced to the two traps + // that matter: `get` resolves the real and BINDS functions to it, `ownKeys` + // reflects the real. Everything the carry relies on is in those two lines. + const real = z.object({ k: z.string() }).meta({ description: 'REAL-D', title: 'REAL-T' }); + const proxied = new Proxy(function lazyZod() {} as unknown as object, { + get: (_t, prop) => { + const value = (real as unknown as Record)[prop]; + return typeof value === 'function' ? value.bind(real) : value; + }, + has: (_t, prop) => prop in (real as unknown as object), + ownKeys: () => Reflect.ownKeys(real as unknown as object), + getOwnPropertyDescriptor: (_t, prop) => + Reflect.getOwnPropertyDescriptor(real as unknown as object, prop), + getPrototypeOf: () => Reflect.getPrototypeOf(real as unknown as object), + }) as unknown as z.ZodType; + + // The control fires only if the two routes really do disagree here. + expect(isSpecLazyProxy(proxied), 'the hand-built wrapper is not proxy-shaped').toBe(true); + expect(metaOf(proxied), 'the accessor route did not reach the real').toMatchObject({ title: 'REAL-T' }); + expect( + metaViaRegistryMap(proxied)?.title, + 'the registry map can now see a proxy\'s own entry — the whole reason the carry reads `.meta()` is gone', + ).toBeUndefined(); + + // ⭐ And the carry reproduces it. On the map route this assertion fails. + const carried = carryRegistryMeta(proxied, z.object({ k: z.string() })); + expect(carried.description).toBe('REAL-D'); + expect(metaOf(carried)).toMatchObject({ description: 'REAL-D', title: 'REAL-T' }); + }); + + it('⛔ `cloneWithDef` still preserves `def.checks` — the rule it existed for first', () => { + const src = z.object({ k: z.string() }).refine((v) => v.k !== 'no', { message: 'refused' }); + const cloned = cloneWithDef(src, {}); + expect(cloned.safeParse({ k: 'yes' }).success).toBe(true); + expect(cloned.safeParse({ k: 'no' }).success, 'the clone dropped a `.refine()`').toBe(false); + }); +}); + +/* ── 4. site ①, re-derived over the published spec surface ────────────────── */ + +describe('site ① — the import boundary conveys the protocol\'s metadata', () => { + it('positive control — rebuilt nodes carrying metadata is a non-empty population', () => { + expect( + census.rebuiltWithMeta, + 'the walk rebuilt no node that carried carried-set metadata — the assertion below cannot fail', + ).toBeGreaterThan(0); + }); + + it('⭐ not one rebuilt node loses a carried key', () => { + // ⚠️ Both sides of this differential are read through `.meta()`. On the + // registry-map route it was blind to every proxied node, which is why the + // first round's version of this assertion stayed green with the carry's + // accessor handling removed — the nodes that would have gone red were not + // in its population. They are now: the proxy count is asserted non-empty, + // and the accessor/map description gap is asserted positive. + expect(census.rebuiltWithMetaLost.slice(0, 10)).toEqual([]); + expect(census.rebuiltWithMetaKept).toBe(census.rebuiltWithMeta); + }); + + it('⭐ the emitted surface matches the protocol on the node the card named', async () => { + // The card's own reading, re-derived through the emitter on both sides. + // `default` is absent on this side ON PURPOSE — that is decision batch #90, + // and this assertion is what keeps a metadata carry from quietly undoing it. + const { PostgresConfigSchema } = (await import('@objectstack/spec/data')) as { + PostgresConfigSchema: z.ZodType; + }; + const spec = z.toJSONSchema(PostgresConfigSchema, { io: 'input' }) as { + properties: Record>; + }; + const mirrored = z.toJSONSchema(stripImportedDefaults(PostgresConfigSchema), { io: 'input' }) as { + properties: Record>; + }; + expect(spec.properties.host, 'the spec no longer carries a title here — re-measure objectui#9102') + .toMatchObject({ title: expect.any(String), description: expect.any(String) }); + expect(mirrored.properties.host).toMatchObject({ + title: spec.properties.host!.title, + description: spec.properties.host!.description, + type: spec.properties.host!.type, + }); + expect(spec.properties.host, 'the control does not fire — the spec emits no default here').toHaveProperty('default'); + expect( + mirrored.properties.host, + 'the import boundary re-emitted a `default` — decision batch #90 says this surface substitutes nothing', + ).not.toHaveProperty('default'); + }); + + it('⭐ every export with nothing to strip still comes back REFERENCE-EQUAL', () => { + const clean = census.roots.filter(([, s]) => !reaches(s, 'default')); + const plain = clean.filter(([, s]) => !reaches(s, 'lazy') && !hasRestlessTuple(s)); + expect(plain.length, 'no clean export free of both exceptions — this assertion is vacuous').toBeGreaterThan(50); + const broken = plain.filter(([, s]) => stripImportedDefaults(s) !== s).map(([n]) => n); + expect( + broken.slice(0, 10), + 'a clean subtree was rebuilt. Carrying metadata must never rebuild a node the walk did not already rebuild.', + ).toEqual([]); + }); + + it('⛔ the spec\'s own graph is left exactly as it was found — every node, both derivations', () => { + // ⭐ A WHOLE-SURFACE DIFFERENTIAL, and the first round's version was not. + // That one watched a side effect — "does the source still hold a default?" + // — which a MUTATING carry does not disturb, so it stayed green under the + // exact hazard it was named for. This compares each node's registry entry + // (both routes), its `_zod.parent` and its def shape, snapshotted before + // either derivation ran and re-read after both have run over every root. + expect( + census.snapshottedNodes, + 'nothing was snapshotted — this assertion is vacuous', + ).toBeGreaterThan(5_000); + expect( + census.mutatedNodes.slice(0, 10), + 'a node of `@objectstack/spec`\'s own graph changed across a derivation. Every other consumer in the ' + + 'workspace shares these objects; a carry must clone, never write in place.', + ).toEqual([]); + + // The side-effect reading is kept as a SEPARATE, weaker statement rather + // than deleted, because "the defaults are still there" is worth saying and + // is not what the sentence above claims. + const carriers = census.roots.filter(([, s]) => reaches(s, 'default')); + expect(carriers.length, 'nothing to check — this control does not fire').toBeGreaterThan(50); + for (const [name, schema] of carriers.slice(0, 200)) { + stripImportedDefaults(schema); + expect(reaches(schema, 'default'), `${name} was stripped IN PLACE — every other consumer sees it`).toBe(true); + } + }); +}); + +/* ── 5. site ②, re-derived over the published node face ───────────────────── */ + +interface FaceCensus { + visited: number; + described: number; + describedKept: number; + describedLost: string[]; +} + +const faceCensus = ((): FaceCensus => { + const strict = deriveStrictAuthoringSchema(SchemaNodeSchema); + const seen = new Set(); + let visited = 0; + let described = 0; + let describedKept = 0; + const describedLost: string[] = []; + const pair = (b: z.ZodType, a: z.ZodType, path: string, depth: number): void => { + if (depth > 40 || seen.has(b)) return; + seen.add(b); + visited++; + if (b.description !== undefined) { + described++; + if (isZod(a) && a.description === b.description) describedKept++; + else describedLost.push(`${path} [${defOf(b).type}] ${JSON.stringify(b.description)}`); + } + const aMap = new Map(isZod(a) ? childrenOf(a) : []); + for (const [label, child] of childrenOf(b)) { + const twin = aMap.get(label); + if (twin) pair(child, twin, `${path}${label}`, depth + 1); + } + }; + pair(SchemaNodeSchema as unknown as z.ZodType, strict as unknown as z.ZodType, '#', 0); + return { visited, described, describedKept, describedLost }; +})(); + +describe('site ② — the strict authoring face conveys what it derives from', () => { + it('positive control — the face is large and really does declare descriptions', () => { + expect(faceCensus.visited, 'the walk barely moved — the assertion below is vacuous').toBeGreaterThan(1_000); + expect( + faceCensus.described, + 'no described node on the node face — site ② has nothing to lose and this file cannot fail', + ).toBeGreaterThan(500); + }); + + it('⭐ not one described node loses its description across the derivation', () => { + // Before objectui#9102 this face kept only the descriptions on the leaves it + // returned untouched: every container it rebuilt — which is every container + // it walks — arrived on the twin with none. + expect(faceCensus.describedLost.slice(0, 10)).toEqual([]); + expect(faceCensus.describedKept).toBe(faceCensus.described); + }); + + it('⛔ the node face itself is left exactly as it was found', () => { + const before = SchemaNodeSchema.description; + deriveStrictAuthoringSchema(SchemaNodeSchema); + expect(SchemaNodeSchema.description).toBe(before); + }); +}); + +/* ── 6. one helper, not two ───────────────────────────────────────────────── */ + +describe('⭐ both sites derive through ONE helper (objectui#9102)', () => { + const read = (rel: string): string => readFileSync(join(SRC_DIR, rel), 'utf8'); + const SITES: [string, string][] = [ + ['the import boundary', 'zod/imported-defaults.ts'], + ['the strict authoring face', 'strict-authoring-face.ts'], + ]; + + it.each(SITES)('%s imports the shared derivation helper', (_label, rel) => { + expect(read(rel)).toMatch(/from '\.{1,2}\/?(zod\/)?node-derivation\.js'/); + }); + + it.each(SITES)('⛔ %s declares no local `cloneWithDef` of its own', (_label, rel) => { + // The acceptance item objectui#9102 was filed on: one site was widened, + // the other kept its copy, and the copy is invisible at every call site. + expect( + read(rel).match(/^\s*(const|function)\s+cloneWithDef\b/m), + 'this site re-declared its own clone helper. The metadata carry is invisible at the call site, so a ' + + 'second copy loses it again with no symptom — which is exactly how these two drifted apart.', + ).toBeNull(); + }); + + it('⭐ the shared helper is the ONLY declaration of it in the package — scanned, not assumed', () => { + // The first round read one file and asserted the declaration EXISTS, under + // a name that promised absence everywhere else. Absence is a property of + // the tree, so the tree is what gets walked. + const DECL = /^\s*(?:export\s+)?(?:const|function)\s+cloneWithDef\b/m; + const SKIP = new Set(['node_modules', 'dist', '.turbo', 'coverage']); + const declarers: string[] = []; + let scanned = 0; + const walk = (dir: string): void => { + for (const entry of readdirSync(dir, { withFileTypes: true })) { + if (SKIP.has(entry.name)) continue; + const full = join(dir, entry.name); + if (entry.isDirectory()) { walk(full); continue; } + if (!/\.(ts|tsx|mts|cts)$/.test(entry.name)) continue; + scanned += 1; + if (DECL.test(readFileSync(full, 'utf8'))) declarers.push(full.slice(SRC_DIR.length + 1)); + } + }; + walk(SRC_DIR); + + expect(scanned, 'the scan found almost no source files — it is pointed at the wrong tree').toBeGreaterThan(100); + expect( + declarers.sort(), + 'there is more than one `cloneWithDef` declaration in this package, or the shared one has moved. ' + + 'A second copy is invisible at every call site and loses the metadata carry with no symptom.', + ).toEqual(['zod/node-derivation.ts']); + }); +}); diff --git a/packages/types/src/strict-authoring-face.ts b/packages/types/src/strict-authoring-face.ts index 269c59b682..37599bd626 100644 --- a/packages/types/src/strict-authoring-face.ts +++ b/packages/types/src/strict-authoring-face.ts @@ -82,6 +82,7 @@ import { SchemaNodeSchema } from './zod/base.zod.js'; // recursion-point fill is installed. The pin file asserts that end state from // the published barrel rather than trusting this paragraph. import { AnyComponentSchema } from './zod/index.zod.js'; +import { carryRegistryMeta, cloneWithDef, internals, isZodType } from './zod/node-derivation.js'; /** * One shape the strict walker could not close, reported as it is met. @@ -108,79 +109,30 @@ export interface DeriveStrictAuthoringOptions { } /** - * The subset of a zod def this walker reads. Zod does not publish `_zod.def` - * in its public types, and the alternative — a chain of `instanceof` narrowings - * against 15 concrete classes — would have to be rewritten whenever zod adds a - * wrapper. Sibling precedent for reading it: `defineNodeComponentUnion` in - * `zod/base.zod.ts` reads the same field to verify its own install. - */ -interface WalkableDef { - type: string; - shape?: Record; - options?: z.ZodType[]; - items?: z.ZodType[]; - element?: z.ZodType; - rest?: z.ZodType; - valueType?: z.ZodType; - left?: z.ZodType; - right?: z.ZodType; - in?: z.ZodType; - innerType?: z.ZodType; - out?: z.ZodType; - catchall?: z.ZodType; - getter?: () => z.ZodType; -} - -interface ZodInternals { - _zod: { def: WalkableDef }; - constructor: new (def: WalkableDef) => z.ZodType; -} - -const internals = (schema: z.ZodType): ZodInternals => schema as unknown as ZodInternals; - -/** - * Is this a zod schema node? - * - * ⚠️ `typeof value === 'object'` is NOT the test, and writing it that way is a - * silent, measured coverage hole rather than a style slip. Zod 4.4.3 builds - * some objects through `$ZodObjectJIT`, whose instances are CALLABLE — they - * answer `typeof 'function'`, their constructor prints as a bound `ZodObject`, - * their traits read `ZodObject/$ZodObjectJIT/$ZodObject/$ZodType`, and they - * parse exactly like any other object. On this face, 20 such nodes are - * reachable, all of them arriving through `@objectstack/spec`-derived subtrees. + * ⭐ THE DEF READER, THE GUARD AND THE CLONE RULE LIVE IN + * `./zod/node-derivation.ts`, NOT HERE (objectui#9102). * - * An object-only guard hands each of them straight back, so the ENTIRE subtree - * beneath it goes unwalked. Measured, before this test admitted functions: 6 - * objects under those nodes stayed open on the twin, and a document with an - * invented key inside one of them — `page.interfaceConfig.sort[]` is the - * shortest — was ACCEPTED by the strict face and the key silently dropped, - * while the same key at the root was correctly refused and named. + * They were three near-identical local copies of what the import boundary + * already had, and the copy is how they drifted: objectui#9086 taught the + * boundary's `cloneWithDef` to carry a node's registry metadata, this file's + * copy was deliberately not widened at the time, and objectui#9102 measured + * what that cost — `deriveStrictAuthoringSchema` rebuilds every container it + * walks, so every description this repository's own mirrors declare was + * dropped from the derived twin, and any `title` or `externalVocabulary` on an + * imported subtree with it. * - * ⛔ Nothing in the corpus could catch that: no document among the 556 carries - * an undeclared key inside those 6 objects, so every corpus reading is - * identical whichever guard is written here. The population pin in - * `__tests__/strict-authoring-face-8345.test.ts` — every reachable object on - * the twin has `catchall: never`, with the function-typed count asserted - * non-zero — is what actually holds this line, and it too had to be taught the - * same lesson: its own census started `typeof node !== 'object'` and shared the - * blind spot with the thing it was measuring. - */ -const isZodType = (value: unknown): value is z.ZodType => - value !== null && (typeof value === 'object' || typeof value === 'function') && '_zod' in value; - -/** - * Clone one schema with a patched def, PRESERVING everything else in it — - * `def.checks` above all, which is where `.refine()` / `.superRefine()` live. + * ⛔ A local re-spelling is the defect, not the fix. The metadata carry is + * invisible at the call site — a derived node with no description parses + * identically to one with — so a second copy loses it again with no symptom. + * `__tests__/registry-meta-carry-9102.test.ts` measures both faces through the + * one helper. * - * A callable JIT instance clones through its own bound constructor and comes - * back as an ordinary object-typed instance of the same class. That is a - * difference in representation, not in behaviour, and behaviour is what the - * pins measure: the clone parses, closes, and leaves the original untouched. + * ⚠️ What is NOT shared: the arms. This walker closes objects with + * `catchall: z.never()` and reports opaque shapes; the boundary strips defaults + * and holds an identity property this face deliberately does not have (it + * rebuilds unconditionally, because "strict" is a property every node must + * acquire). Only the three primitives above are common, and only they moved. */ -const cloneWithDef = (schema: z.ZodType, patch: Partial): z.ZodType => { - const Ctor = internals(schema).constructor; - return new Ctor({ ...internals(schema)._zod.def, ...patch }); -}; /** * A walker with ONE memo. Two schemas derived through the same walker share @@ -199,8 +151,20 @@ function createStrictWalker(options: DeriveStrictAuthoringOptions = {}): walk(def.getter!(), `${path}/lazy`)); + const out: z.ZodType = carryRegistryMeta( + schema, + z.lazy(() => walk(def.getter!(), `${path}/lazy`)), + ); memo.set(schema, out); return out; } diff --git a/packages/types/src/zod/imported-defaults.ts b/packages/types/src/zod/imported-defaults.ts index 15a63179f4..f912ad9254 100644 --- a/packages/types/src/zod/imported-defaults.ts +++ b/packages/types/src/zod/imported-defaults.ts @@ -74,89 +74,32 @@ */ import { z } from 'zod'; +import { carryRegistryMeta, cloneWithDef, internals, isZodType } from './node-derivation.js'; /** - * The subset of a zod def this walker reads. Zod does not publish `_zod.def` in - * its public types, and the alternative — a chain of `instanceof` narrowings - * against 15 concrete classes — would have to be rewritten whenever zod adds a - * wrapper. Same field set, and the same reason, as `../strict-authoring-face.ts`. - */ -interface WalkableDef { - type: string; - shape?: Record; - options?: z.ZodType[]; - items?: z.ZodType[]; - element?: z.ZodType; - rest?: z.ZodType; - valueType?: z.ZodType; - keyType?: z.ZodType; - left?: z.ZodType; - right?: z.ZodType; - in?: z.ZodType; - out?: z.ZodType; - innerType?: z.ZodType; - catchall?: z.ZodType; - getter?: () => z.ZodType; -} - -interface ZodInternals { - _zod: { def: WalkableDef; optin?: string }; - constructor: new (def: WalkableDef) => z.ZodType; -} - -const internals = (schema: z.ZodType): ZodInternals => schema as unknown as ZodInternals; - -/** - * Is this a zod schema node? + * ⭐ THE METADATA CARRY LIVES IN `./node-derivation.ts`, NOT HERE (objectui#9102). * - * ⚠️ `typeof value === 'object'` is NOT the test, and writing it that way is a - * measured coverage hole rather than a style slip. Zod 4.4.3 builds some - * objects through `$ZodObjectJIT`, whose instances are CALLABLE — they answer - * `typeof 'function'` and parse exactly like any other object. On this face - * those nodes arrive through `@objectstack/spec`-derived subtrees, which is - * precisely the population this module walks: an object-only guard hands each - * of them straight back and the entire subtree beneath it — defaults included — - * goes unwalked, with no symptom other than a residue count that will not fall. - * `../strict-authoring-face.ts` records the same lesson, learnt the hard way. - */ -const isZodType = (value: unknown): value is z.ZodType => - value !== null && (typeof value === 'object' || typeof value === 'function') && '_zod' in value; - -/** - * Carry a source node's `.describe()` onto a node derived from it — THE ONE - * DESCRIPTION RULE, used by every derivation below. + * This module used to hold its own `cloneWithDef` and its own one-key carry, + * spelled THE ONE DESCRIPTION RULE and defended in a sentence about `id`. Both + * halves of that sentence were wrong about this surface, and objectui#9102 + * measured how: `id` does not occur in the spec's registry metadata here at all, + * while `title` and `externalVocabulary` sit on nodes this walker rebuilds — + * so the rule guarded a key that was never present and dropped the ones that + * were. `@objectstack/spec` emits `{default, description, title, type}` for a + * datasource `host`; this boundary emitted `{description, type}`. * - * ⚠️ A description is NOT part of `def`, so nothing here carries it by accident. - * Measured on zod 4.4.3: `.describe(d)` stores `{ description: d }` in - * `z.globalRegistry`, a WeakMap keyed by the NODE, and `description` reads back - * through `_zod.parent`, which only zod's own `clone()` sets. So any node this - * module builds with `new Ctor(def)` — every `cloneWithDef` below — starts with - * NO description however faithfully it copies `def`, and `.removeDefault()` - * hands back an inner node that never had the outer's description to begin with. - * Both are the same silent loss, and this is the one place that repairs it. + * ⛔ The trade-off is NOT "description versus everything". It is a bounded, + * enumerated carry set versus a blanket spread, and the bound is written down + * as `CARRIED_REGISTRY_META_KEYS` with `id` refused by name in + * `REFUSED_REGISTRY_META_KEYS` — `id` for the `_idmap` mutation it would cause, + * which is the one part of the old sentence that was correct. The census that + * fails when the protocol grows a key outside either list is + * `../__tests__/registry-meta-carry-9102.test.ts`. * - * ⛔ `.describe()` and not a write to `_zod.parent`: it CLONES, so the derived - * node can safely be one of `@objectstack/spec`'s own objects — which it is on - * the `default` arm's already-optional branch, 267 times across spec 17.4.0. - * Poking `parent` there would mutate the spec's shared graph, which the header's - * last paragraph forbids. - * - * ⛔ The description and nothing else. `z.globalRegistry.get(...)` would also - * hand back `id`, and re-registering an `id` rewrites the registry's `_idmap` - * entry to point at THIS package's derivation — a mutation of shared global - * state, off a surface that promises it mutates nothing. + * ⛔ And it is SHARED, because the identical rebuild in + * `../strict-authoring-face.ts` carried the identical loss. A local copy is how + * the two drifted apart in the first place. */ -const withDescriptionOf = (source: z.ZodType, derived: z.ZodType): z.ZodType => - source.description === undefined ? derived : derived.describe(source.description); - -/** - * Clone one schema with a patched def, PRESERVING everything else — `def.checks` - * above all, and the node's own description with it (see `withDescriptionOf`). - */ -const cloneWithDef = (schema: z.ZodType, patch: Partial): z.ZodType => { - const Ctor = internals(schema).constructor; - return withDescriptionOf(schema, new Ctor({ ...internals(schema)._zod.def, ...patch })); -}; /** Does this node already answer "omissible" to an enclosing object? */ const isAlreadyOptional = (schema: z.ZodType): boolean => @@ -193,12 +136,13 @@ const walk = (schema: z.ZodType): z.ZodType => { // goes red if it moves, rather than trusting this sentence. // // ⛔ Rebuilt through `cloneWithDef`, not `z.lazy(…)`: a fresh `z.lazy` would - // be a different class with none of this node's own `def.checks` or - // description, which is the same silent-loss shape the clone rule exists for. - // ⚠️ `cloneWithDef` carries the description only because it now asks - // `withDescriptionOf` to; a description lives in `z.globalRegistry`, not in + // be a different class with none of this node's own `def.checks` or registry + // metadata, which is the same silent-loss shape the clone rule exists for. + // ⚠️ `cloneWithDef` carries that metadata only because it asks + // `carryRegistryMeta` to; registry state lives in `z.globalRegistry`, not in // `def`, so copying `def` never carried it. This sentence read as though it - // did until objectui#9034 measured otherwise. + // did until objectui#9034 measured the description half and objectui#9102 the + // rest of the vocabulary. if (def.type === 'lazy') { const out = cloneWithDef(schema, { getter: () => walk(def.getter!()) }); memo.set(schema, out); @@ -231,19 +175,22 @@ const walk = (schema: z.ZodType): z.ZodType => { * member and is made optional again, because its omissibility was the * default's doing and removing it must not narrow what this package accepts. * - * `withDescriptionOf` is the documentation half, and it is the same one rule - * the `lazy` arm above invokes. The protocol spells its guidance - * `.default(v).describe(d)`, so `d` sits on the OUTER node — the very node - * `.removeDefault()` discards. Without the carry, 2024 of the 2024 described - * `ZodDefault` nodes reachable from spec 17.4.0 arrive on this side with no - * description at all, and this boundary would convey strictly LESS than the + * `carryRegistryMeta` is the documentation half, and it is the same one rule + * `cloneWithDef` invokes on every other arm. The protocol spells its + * guidance on the OUTER node — `.default(v).describe(d)`, and + * `.default(v).meta({title})` on the datasource configs — which is the very + * node `.removeDefault()` discards. Without the carry, every described + * `ZodDefault` reachable from the spec arrives on this side with no + * description, and every one carrying a `title` or an `externalVocabulary` + * arrives without it: this boundary would convey strictly LESS than the * protocol it mirrors. `__tests__/imported-defaults-describe-9034.test.ts` - * re-derives that population rather than trusting this paragraph. + * and `__tests__/registry-meta-carry-9102.test.ts` re-derive those + * populations rather than trusting this paragraph. */ case 'default': { const inner = walk((schema as unknown as { removeDefault: () => z.ZodType }).removeDefault()); const next = isAlreadyOptional(inner) ? inner : z.optional(inner); - out = withDescriptionOf(schema, next); + out = carryRegistryMeta(schema, next); break; } case 'object': { @@ -331,13 +278,16 @@ const walk = (schema: z.ZodType): z.ZodType => { * schema, at this package's import boundary. * * Returns a schema with the same TypeScript type, the same keys, the same - * checks, the same descriptions and the same accept set — differing only in - * that a key the author omitted stays omitted in `parse` output instead of + * checks, the same registry metadata and the same accept set — differing only + * in that a key the author omitted stays omitted in `parse` output instead of * being written for them. The input is left untouched. * - * ⚠️ "the same descriptions" is carried deliberately and is not free — see - * `withDescriptionOf`. A description is registry state keyed by the node, so - * every derivation here has to re-attach it explicitly (objectui#9034). + * ⚠️ "the same registry metadata" is carried deliberately and is not free — see + * `carryRegistryMeta` in `./node-derivation.ts`. A description, a `title` and an + * `externalVocabulary` are all registry state keyed by the node, so every + * derivation here has to re-attach them explicitly (objectui#9034 for the + * description, objectui#9102 for the rest). The carry set is bounded and + * enumerated there; `id` is refused by name. * * ⚠️ A schema that HAD a default in it is not reference-equal to the spec's * afterwards: a mirror member re-exporting one of these re-exports this diff --git a/packages/types/src/zod/node-derivation.ts b/packages/types/src/zod/node-derivation.ts new file mode 100644 index 0000000000..610c23e05c --- /dev/null +++ b/packages/types/src/zod/node-derivation.ts @@ -0,0 +1,253 @@ +/** + * ObjectUI + * Copyright (c) 2024-present ObjectStack Inc. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */ + +/** + * ONE DERIVATION RULE, SHARED BY EVERY WALKER IN THIS PACKAGE (objectui#9102). + * + * This package derives new zod graphs from schemas it did not author: the + * import boundary (`./imported-defaults.ts`) strips imported defaults, and the + * strict authoring face (`../strict-authoring-face.ts`) closes unknown keys. + * Both do it the same way — patch a copy of a node's own `_zod.def`, call the + * node's own constructor — and both used to carry their own copy of that + * spelling. They carried the same defect with it, which is what objectui#9102 + * was filed about: `new Ctor({...def, ...patch})` reproduces `def` faithfully + * and reproduces a node's REGISTRY METADATA not at all. + * + * ⛔ The two copies are not restored. A walker that needs a different arm + * writes a different arm; what it may not do is write its own `cloneWithDef`, + * because the metadata carry is invisible at the call site and a second copy + * silently loses it again. The sibling-drift shape is exactly what + * objectui#9086 fixed at one site and objectui#9102 found still standing at the + * other. + * + * ## Why registry metadata has to be carried EXPLICITLY + * + * A zod 4 description is not `def` state, and neither is any other registry + * key. Measured on zod 4.4.3: `.describe(d)` and `.meta(m)` both store their + * argument in `z.globalRegistry` — a WeakMap keyed by the NODE — and the + * `description` getter reads back through `_zod.parent`, a link only zod's own + * `clone()` sets. So a node built with `new Ctor(def)` starts with NO registry + * metadata however faithfully it copies `def`, and `.removeDefault()` hands + * back an inner node that never carried the outer's entry to begin with. Those + * two are the same silent loss, and this module is the one place that repairs + * it. `../__tests__/registry-meta-carry-9102.test.ts` re-derives every zod fact + * in this paragraph rather than trusting it. + * + * ## ⛔ The carry never mutates its source, and never touches `_idmap` + * + * `.meta()` CLONES — it is `this.clone()` followed by `globalRegistry.add(clone, + * data)` — so the derived node may safely BE one of `@objectstack/spec`'s own + * objects, which it is on the import boundary's already-optional branch for as + * many nodes as the pin file counts. Writing metadata in place instead would + * relabel `@objectstack/spec` for every other consumer in the workspace, with + * no symptom on this side. + * + * `globalRegistry.add()` also writes `_idmap` — but ONLY when the metadata + * object it is handed contains an `id`. Because the carry below is an ALLOW + * list and `id` is not on it, no derivation in this package can reach that + * branch: re-registering an `id` would repoint the registry's id map at this + * package's derivation, a mutation of shared global state off a surface that + * promises it mutates nothing. + */ + +import { z } from 'zod'; + +/** + * The subset of a zod def these walkers read. + * + * Zod does not publish `_zod.def` in its public types, and the alternative — a + * chain of `instanceof` narrowings against 15 concrete classes — would have to + * be rewritten whenever zod adds a wrapper. The field set is the UNION of what + * the walkers need, so one definition serves both. + */ +export interface WalkableDef { + type: string; + shape?: Record; + options?: z.ZodType[]; + items?: z.ZodType[]; + element?: z.ZodType; + rest?: z.ZodType; + valueType?: z.ZodType; + keyType?: z.ZodType; + left?: z.ZodType; + right?: z.ZodType; + in?: z.ZodType; + out?: z.ZodType; + innerType?: z.ZodType; + catchall?: z.ZodType; + getter?: () => z.ZodType; +} + +interface ZodInternals { + _zod: { def: WalkableDef; optin?: string; parent?: z.ZodType }; + constructor: new (def: WalkableDef) => z.ZodType; +} + +/** Reach a node's unpublished internals. The one cast, in one place. */ +export const internals = (schema: z.ZodType): ZodInternals => schema as unknown as ZodInternals; + +/** + * Is this a zod schema node? + * + * ⚠️ `typeof value === 'object'` is NOT the test, and writing it that way is a + * measured coverage hole rather than a style slip — but ⛔ NOT for the reason + * both walkers used to give. They each blamed zod's `$ZodObjectJIT`, and that + * diagnosis is wrong in both halves. A `$ZodObjectJIT` instance is an ORDINARY + * OBJECT — `typeof 'object'`, ⛔ never callable: zod's `$constructor` returns + * plain objects, and the "JIT" names eval-compiled PARSE CODE, not a callable + * node. Nor is the trait absent here — EVERY `z.object()` carries it, so this + * surface is covered in them and an object-only guard would not miss one. + * + * What such a guard WOULD miss is `@objectstack/spec`'s own lazy cross-module + * wrappers, `new Proxy(functionTarget, …)` around a factory: those answer + * `typeof 'function'` because the proxy TARGET is a function, while forwarding + * `_zod` — the `$ZodObjectJIT` trait along with it — to the real schema behind + * them. ⭐ So the trait separates nothing in either direction; callability is + * the signal, and it belongs to the proxy rather than to anything zod built. + * + * That is why the guard admits functions: an object-only guard hands each of + * those proxies straight back along with the ENTIRE subtree beneath it, with no + * symptom other than a residue count that will not fall. The mechanism is + * re-derived in `../__tests__/registry-meta-carry-9102.test.ts` — including the + * probe that tells a proxy from an ordinary node, since + * `Object.getOwnPropertyNames` on one of these proxies THROWS rather than + * answering. + */ +export const isZodType = (value: unknown): value is z.ZodType => + value !== null && (typeof value === 'object' || typeof value === 'function') && '_zod' in value; + +/** + * ⭐ THE CARRY SET — the registry keys a derivation in this package reproduces. + * + * ⛔ NOT "everything the source carries", and ⛔ not a guess. It is the + * vocabulary `@objectstack/spec` actually publishes on the surface this package + * imports, ENUMERATED so that widening it is a deliberate edit with a review + * attached rather than a blast radius nobody measured. The count and the member + * list are re-derived by `../__tests__/registry-meta-carry-9102.test.ts`, which + * censuses every published spec subpath and goes RED the day the protocol + * carries a key that is on neither this list nor {@link REFUSED_REGISTRY_META_KEYS}. + * + * ⭐ That red is the point of writing a bounded list at all. A bounded carry + * set drops a new key SILENTLY, which is the same "narrower than the protocol" + * defect objectui#9102 exists to close, one key later. Pairing the list with a + * census that fails on an unclassified key converts the silent drop into a + * failing gate, so the bound costs boundedness and not fidelity. + * + * ⚠️ That census reads the surface through `.meta()`, the same route + * {@link carryRegistryMeta} carries through. A census keyed on + * `z.globalRegistry.get(node)` is BLIND to every proxied node's own entry and + * would report a clean vocabulary it never actually looked at. + */ +export const CARRIED_REGISTRY_META_KEYS: readonly string[] = Object.freeze([ + 'description', + 'title', + 'default', + 'externalVocabulary', + 'format', + 'xRef', + 'xExpression', + 'xEnumDeprecated', +]); + +/** + * ⛔ THE REFUSAL — registry keys a derivation in this package must never write. + * + * `id` is the whole list, and it is refused on a mechanism rather than on + * taste: `globalRegistry.add(node, meta)` writes `_idmap` whenever `meta` + * contains an `id`, so carrying one would repoint a shared, global id map at + * this package's derived node. `globalRegistry.get()` already declines to + * INHERIT an `id` down a parent chain for the same reason; this list is the + * half zod cannot enforce, because an explicit carry is not an inheritance. + * + * ⚠️ It is refused, not absent: the pin file asserts both that no derivation + * emits it AND that `_idmap` does not grow across a walk, because "the spec + * happens not to use `id` here" is a fact about today's spec and not a property + * of this module. + */ +export const REFUSED_REGISTRY_META_KEYS: readonly string[] = Object.freeze(['id']); + +/** + * Carry a source node's registry metadata onto a node derived from it. + * + * Returns `derived` UNCHANGED when there is nothing to carry — which is what + * keeps the import boundary's identity property intact, and what makes this + * safe to call on an arm whose population is empty today. + */ +export const carryRegistryMeta = (source: z.ZodType, derived: z.ZodType): z.ZodType => { + // ⭐ `source.meta()` AND ⛔ NOT `z.globalRegistry.get(source)`, which is a + // DIFFERENT READING on a large part of this surface. + // + // `@objectstack/spec` publishes most of its schemas as lazy cross-module + // `new Proxy(functionTarget, …)` wrappers. The proxy's `get` trap resolves the + // real schema and binds any function it hands back, so `source.meta()` runs + // `real.meta()` and returns the REAL's registry entry. A registry lookup keyed + // by the proxy object cannot: nothing ever registered the proxy, and the map's + // parent-chain walk reaches only the real's ANCESTORS. Re-derived in + // `../__tests__/registry-meta-carry-9102.test.ts`: on the published spec + // surface the two readings disagree for the overwhelming majority of + // metadata-bearing proxies, and the accessor route is the one that sees what + // the protocol actually declared. + // + // ⚠️ Today every metadata-bearing proxy carries `description` only, so the map + // route would lose nothing VISIBLE — which is exactly why this is worth a + // sentence. The day the spec puts a `title` on a proxied node, the map route + // drops it silently, and that is the defect class this module exists to close. + // + // ⛔ There is no separate `description` fallback any more, and its absence is + // structural rather than lucky: zod's `description` getter is + // `globalRegistry.get(inst)?.description` for the instance it was installed + // on, so for a plain node it IS `.meta()?.description`, and through a proxy + // both resolve to the real. The pin file measures that the fallback would have + // zero occasions to fire. + const meta = source.meta() as Record | undefined; + if (meta === undefined) return derived; + + const carried: Record = {}; + for (const key of CARRIED_REGISTRY_META_KEYS) { + if (Object.prototype.hasOwnProperty.call(meta, key) && meta[key] !== undefined) { + carried[key] = meta[key]; + } + } + if (Object.keys(carried).length === 0) return derived; + + // `.meta()` clones, so `derived` is left exactly as it was found — including + // when `derived` IS one of the spec's own objects. The clone's own entry is + // `carried`; anything `derived` already carried still reads back through the + // parent link zod's `clone()` sets, minus the `id` zod itself declines to + // inherit. + return derived.meta(carried); +}; + +/** + * Clone one schema with a patched def, PRESERVING everything else about it. + * + * `def.checks` above all: ⛔ never rebuild a node with `z.object(shape)` or a + * fresh `z.lazy(...)` instead. Those spellings keep the shape and DROP + * `def.checks`, so every `.refine()` and `.superRefine()` installed on the way + * down is silently lost — the import boundary would then accept documents the + * spec refuses, and the strict authoring face would UNDER-report red, which is + * worse than no strict face because it reads as evidence. + * + * And the node's registry metadata with it, via {@link carryRegistryMeta} — + * which is the half `def` copying never covered. + * + * ⚠️ A CALLABLE source clones into a non-callable node, and that is fine. These + * callables are `@objectstack/spec`'s lazy proxies; `internals(schema)` and + * `.constructor` both travel the proxy's `get` trap to the real schema, so the + * clone is an ordinary instance of the real's class built from the real's def. + * A difference in representation, not in behaviour — and behaviour is what the + * pins measure. ⛔ The callability is NOT zod's `$ZodObjectJIT`: that trait + * sits on every `z.object()` — the real behind this proxy included — and makes + * nothing callable, so it tells these two apart in neither direction. The probe + * that does is in `../__tests__/registry-meta-carry-9102.test.ts`, and a first + * round of objectui#9102 shipped the opposite claim in this file's prose. + */ +export const cloneWithDef = (schema: z.ZodType, patch: Partial): z.ZodType => { + const Ctor = internals(schema).constructor; + return carryRegistryMeta(schema, new Ctor({ ...internals(schema)._zod.def, ...patch })); +};