Skip to content

feat(metadata): versioned ADR-0087 forward conversion at the artifact-ingestion door - #12843

Merged
hotlong merged 6 commits into
mainfrom
claude/issue-12772-artifact-conversion-door
Aug 28, 2026
Merged

feat(metadata): versioned ADR-0087 forward conversion at the artifact-ingestion door#12843
hotlong merged 6 commits into
mainfrom
claude/issue-12772-artifact-conversion-door

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes #12772

What broke

Artifacts built by released 17.x tooling are refused by the 17.2 runtime. Measured through the framework's own artifact door on current main (OS_ARTIFACT_URL boot of the real hotcrm dist/objectstack.json, built by released @objectstack/cli 17.1.0, carrying 75 allowRestore + 75 allowPurge occurrences its source contains zero of):

ERROR Plugin startup failed: com.objectstack.metadata
  "expected": "never", "code": "invalid_type",
  "path": ["permissions", 0, "objects", "crm_lead", "allowRestore"]
  ... the #12497 retiredKey tombstone message verbatim ...

The parse point is NOT packages/cli (the triage's located neighbour, artifact-boot-migration.ts, is physical-schema drift policy only): every framework artifact door — OS_ARTIFACT_URL, OS_ARTIFACT_PATH, the dist/objectstack.json convention, the HMR artifact reload, and the environment-artifact envelope — funnels to MetadataPlugin._parseAndRegisterArtifact in packages/metadata, whose strict ObjectStackDefinitionSchema.parse fires the tombstone. os migrate meta targets sources, not built artifacts, so an operator had no remedy.

The fix — a versioned forward conversion, not an unconditional strip

Per the triage ruling: #12497 retired the keys with the M2 anchor open (#1883 remains open — the keys RETURN with M2), so an unconditional strip becomes wrong the day they are legal again. The conversion is keyed off the artifact's authored protocol declaration:

  • New policy, packages/metadata-core/src/artifact-forward-conversion.ts (applyArtifactForwardConversions, exported beside the protocol handshake it reads the declared range through): when the artifact's declared engines.protocol floor (leading version token, e.g. ^17.1.0 → 17.1.0) is below the running @objectstack/spec version, replay the full ADR-0087 conversion chain — retired entries included — over the definition before the strict parse. This is exactly the stored-row policy (applyConversionsToStoredItem, ADR-0087 addendum), refined by the version evidence an artifact carries and a sys_metadata row does not.
  • Floor at or above the running spec → nothing is replayed; the strict parse, tombstones included, stays the authority. That boundary is what keeps this a conversion rather than an amnesty, and it is pinned.
  • No declared range → full replay (the handshake's grandfathering posture and the stored-row posture for versionless data at rest; conversions only rewrite shapes they positively recognize).
  • The ADR-0087 registry in packages/spec stays the single authority on WHAT converts — this PR consumes applyConversions with includeRetired: true and adds nothing to the registry. No packages/spec or packages/metadata-protocol changes (the declared stop boundaries were not crossed; resolveDeclaredRange in metadata-core gained an export keyword so the policy reads the declared range through the handshake's own source-priority rather than a second opinion).
  • The door (packages/metadata/src/plugin.ts): _parseAndRegisterArtifact applies the policy before every strict parse — bare definitions and both environment-artifact envelope shapes — and surfaces notices operator-visibly and deduped, modeled on the stored-row pass's surfacing: one summary line per conversion per artifact (the real artifact converts 150 sites; 150 identical warn lines would bury the boot log), naming the version evidence, the volume, the first path, the prescription, and the rebuild remedy.

Both directions pinned

  • Accept (packages/metadata/src/plugin-artifact-forward-conversion.test.ts, on a real fixture — the manifest + permissions blocks verbatim from the 17.1-built repro artifact, 75+75 occurrences, engines.protocol ^17.1.0, with a premise-guard test pinning those counts): parses, registers all six permission sets, strips exactly the two retired keys, preserves every other authored grant bit; the operator summary appears once and does not repeat on an HMR-style re-ingest.
  • Refuse (same fixture, manifest floor rewritten at test time to the installed spec version, so the pin survives every future release): still refuses at the tombstone, and the refusal carries the prescription — was removed in @objectstack/spec 17 (#12497, ADR-0049) and the standardized migrate sentence. This door answers no HTTP request (every refusal fires before a server binds), so the pin is on the tombstone message reaching the operator, not an ADR-0112 envelope.
  • Policy unit suite (packages/metadata-core/src/artifact-forward-conversion.test.ts, 16 tests): both directions, newer-authored refusal, bare-major ^17 floor, undeclared range, unresolvable runtime version (window stays closed — amnesty needs positive version evidence), byte-preservation by reference identity (copy-on-write), idempotence, range-floor spellings.

End-to-end positive control

Before (main, f75a38a): OS_ARTIFACT_URL=file:///.../hotcrm/dist/objectstack.json boot refuses with the tombstone. After (this branch, rebuilt dist): the same boot prints the one-line conversion summary and completes — Migration complete (37 plugins started against configured DB), exit 0.

Scope notes

  • The cloud composed-artifact door (OS_COMPOSED_ARTIFACT_URL) is the second consumer and is out of scope for this PR per triage; the accepting seat files the cloud follow-up card with a Blocked-by link here. The policy was landed in @objectstack/metadata-core (spec-only dependency, already the handshake's home) so that door can consume it rather than reimplement.
  • Observation, not changed here: on an artifact boot, AppPlugin also registers the raw bundle's security metadata in memory (registerInMemory), bypassing this conversion. Nothing strict-parses those copies (measured: the full boot is green, and the retired bits gate nothing by definition of the retirement), so no defect is reachable today; noted in the report for triage.

Verification

Verification union ran at head 6d097a6 (this PR's tree, clean): derived gate families via node scripts/pm/dispatch-gates.mjs (no paths — change set derived from merge-base f75a38a), all 24 runnable derived + convention gates green at their own verdict lines; check:type-check-debt --re-measure green ("none above its recorded number") after building the full packages closure; full-repo pnpm lint (eslint with --no-inline-config) exit 0; @objectstack/metadata-core typecheck (src + test programs, new files confirmed present via --listFiles) and both package suites green (14 + 36 test files). scripts/pm/check-half-states.mjs is NOT MEASURED locally (exit 3, PREREQUISITE NOT MET — the container token is the proxy placeholder, not a GitHub credential); CI owns it.

Generated by Claude Code


Generated by Claude Code

claude added 2 commits August 28, 2026 01:59
…artifact-ingestion door

Artifacts built by released 17.x tooling carry then-legal keys (allowRestore/
allowPurge, retired in spec 17.2.0) and were refused by the strict parse in
MetadataPlugin._parseAndRegisterArtifact — the retiredKey tombstone fired with
no operator remedy, since 'os migrate meta' targets sources, not built
artifacts.

New policy in @objectstack/metadata-core (applyArtifactForwardConversions):
replay the full ADR-0087 conversion chain — retired entries included — over an
artifact whose declared engines.protocol floor predates the running spec
version; an artifact authored at the current (or newer) surface converts
nothing and still answers to the tombstone. Versioned, not a blanket strip:
the retired keys return with M2 (#1883), and artifacts authored against that
surface must never be stripped by history.

The door (MetadataPlugin) applies it before every strict parse — bare
definitions and environment-artifact envelopes — and surfaces notices
operator-visibly, deduped per conversion per artifact, modeled on the
stored-row pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVYmuhHW6qZmNBqciaS7BN
@github-actions github-actions Bot added size/xl documentation Improvements or additions to documentation tests tooling labels Aug 28, 2026
@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 3 package(s): @objectstack/metadata-core, @objectstack/metadata, @objectstack/runtime, touching 21 documentable anchor(s). ⚠️ 1 changed file(s) yielded no anchor (packages/metadata-core/src/index.ts), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

5 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/api/client-sdk.mdx (via MetadataPlugin (symbol))
  • content/docs/api/index.mdx (via MetadataPlugin (symbol))
  • content/docs/automation/flows.mdx (via MetadataPlugin (symbol))
  • content/docs/kernel/services-checklist.mdx (via MetadataPlugin (symbol))
  • content/docs/protocol/kernel/metadata-service.mdx (via MetadataPlugin (symbol))

1 release-owned page(s) also name something this change touched. These are read-only:

  • content/docs/releases/implementation-status.mdx (via MetadataPlugin (symbol))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see
  • 1 changed file(s) yielded no anchor (packages/metadata-core/src/index.ts) — pages documenting those are invisible to this run
  • 8 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 47 of 219 client-bound route-ledger rows — the other 172 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 172: 14 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 29 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json b489d3c725e86583410c20a91e59e0c5eb0ba255packageMentionDocs.

Which tree this was computed on

This run read content/docs from 783c4ae68a35a13950b951345965f9540d990ef7 — the merge of head 7eff8966c224309fa6ba68e2b0143397cae9f2f5 into base b489d3c725e86583410c20a91e59e0c5eb0ba255, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 783c4ae68a35a13950b951345965f9540d990ef7 && git checkout 783c4ae68a35a13950b951345965f9540d990ef7
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin b489d3c725e86583410c20a91e59e0c5eb0ba255 7eff8966c224309fa6ba68e2b0143397cae9f2f5 && git checkout -B drift-repro b489d3c725e86583410c20a91e59e0c5eb0ba255 && git merge --no-ff 7eff8966c224309fa6ba68e2b0143397cae9f2f5

node scripts/docs-audit/affected-docs.mjs --json b489d3c725e86583410c20a91e59e0c5eb0ba255

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs b489d3c725e86583410c20a91e59e0c5eb0ba255 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

claude added 2 commits August 28, 2026 03:22
…laration surface

The artifact-forward-conversion module's public types referenced
ConversionNotice from the @objectstack/spec root, so the emitted
dist/index.d.ts imported the root entry - and every downstream type program
reading metadata-core's declarations began loading the ~2MB spec root d.mts
BESIDE the d.ts flavor it already read. Measured: the TEST_DEBT re-measure of
packages/qa/http-conformance crossed CI's ~4GB tsc heap ceiling and OOM'd
(listFiles diff between merge-base and branch: the only additions were
spec/dist/index.d.mts and its chunk; the capped re-measure passes at the
merge base and fails on the branch, same box, same command).

Public surface now speaks ArtifactConversionNotice, a structural mirror
pinned in both assignability directions in the module's test; the runtime
applyConversions import stays and no longer reaches declaration emit. The
rebuilt d.ts carries only the pre-existing spec/data subpath imports, and the
capped (4096MB) http-conformance re-measure is green on this tree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVYmuhHW6qZmNBqciaS7BN
…al in the dispatcher vocabulary

check:dispatcher-error-vocabulary flagged the new
ArtifactConversionNotice.code literal in
packages/metadata-core/src/artifact-forward-conversion.ts as an unclassified
code-stamping site. Classified foreign-vocabulary beside the existing
OS_METADATA_CONVERTED row for spec's apply.ts: the literal sits in a TYPE
position of the structural mirror (declared to keep the spec ROOT import out
of the package's public declaration surface), stamps nothing at runtime, and
the notices flow to an onNotice callback exactly as in the classified spec
site - nothing thrown, no envelope built.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVYmuhHW6qZmNBqciaS7BN

Copy link
Copy Markdown
Contributor Author

Patch rounds on this PR (the body's Verification section describes the original union at 6d097a6; this comment carries the two CI-divergence fixes and the current union):

  1. Type Check · debt ledger red at 6d097a6 — the policy module's public d.ts referenced the @objectstack/spec ROOT, so downstream type programs loaded the 2MB root twice (d.ts + d.mts flavors); the http-conformance TEST_DEBT re-measure crossed CI's ~4GB tsc heap ceiling and OOM'd (the gate correctly refused to record 0). Evidence: tsc --listFiles diff between merge-base and branch programs = exactly +spec/dist/index.d.mts + one chunk; capped re-measure (--max-old-space-size=4096) green at merge base, OOM on branch, same box and command. Fixed in 00714b6: the public surface speaks ArtifactConversionNotice, a structural mirror pinned in both assignability directions against spec's ConversionNotice; the rebuilt d.ts carries only the pre-existing spec/data subpath imports.
  2. Lint & Repo Gates red at 00714b6check:dispatcher-error-vocabulary flagged the mirror's literal code type as an unclassified code-stamping site. Fixed in b02ec81 by the gate's prescribed remedy: a foreign-vocabulary classification row in packages/runtime/src/dispatcher-error-vocabulary.ts, beside the existing row for the same code at spec's apply.ts (type-position literal; stamps nothing at runtime).

Union at head b02ec81, clean tree: all derived + convention gates green; check:dispatcher-error-vocabulary OK (22 sites, all classified); full check:type-check-debt re-measure green under the CI-shaped heap ceiling (NODE_OPTIONS=--max-old-space-size=4096, "none above its recorded number"); full-repo lint exit 0; metadata-core 14/14 and metadata 36/36 test files; @objectstack/runtime typecheck green and the four vocabulary-consuming runtime suites 69/69 tests green.

Generated by Claude Code


Generated by Claude Code

@github-actions

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 33140380738 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

失败的 job(日志抽取,best effort):

  • Test Core (2/6) — 失败步骤: Run this shard's tests

    @objectstack/metadata:test:  FAIL  src/plugin-artifact-forward-conversion.test.ts > artifact door — the conversion is versioned, not a blanket amnesty (#12772) > an artifact claiming the CURRENT spec 
      ↳ 失败原因: @objectstack/metadata:test: AssertionError: expected 'expected "the strict parse must refus…' to match /allowRestore|allowPurge/
    

↳ 失败原因 是判读的关键:超时Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

跨 PR 相同签名(24h,按失败测试文件聚合):

  • src/plugin-artifact-forward-conversion.test.ts — 24h 窗口内只有本 PR 撞到过,暂不汇总(再有一个不同 PR 撞到就会自动开汇总 issue)。
  • ⚠️ 24h 评论账本没读完(超过 5 页仍未读到窗口尽头),所以上面的「不同 PR 数」是下界,不是全量。

历史信号:

  • 本 PR 过去 24h 无队列失败记录(首次)。
  • 过去 24h 队列共有 1 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

…sal surface under the #12845 residue ruling

Main landed acceptRetiredDefaultResidue (maintainer ruling 2026-08-28): a
retired DEFAULTED key's emitted default parses as inert residue and strips
silently at the schema layer; only a NON-default value keeps the #12497
tombstone. The real 17.1-built fixture carries exclusively the emitted
default (150 x false, premise-guarded), so the old current-version refusal
pin now describes a shape the ruled contract accepts - the merge queue
measured exactly that on its merge tree.

The pin is re-aimed, not weakened: current-version + NON-default value
(true, authorable on the 17.1 surface, never emitted by a default) still
refuses with the tombstone and prescription; and a new pin holds the other
half of the ruling at the door - current-version + pure residue parses
clean with the keys stripped by the SCHEMA layer while this door's
versioned window stays closed (no conversion notice), keeping the door's
amnesty versioned.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SVYmuhHW6qZmNBqciaS7BN

Copy link
Copy Markdown
Contributor Author

Patch round 3 (merge-queue dequeue, CI_FAILURE on the queue's merge tree): the refuse-direction pin failed against latest main because main landed 1ffee51 (#12845, maintainer ruling 2026-08-28) after this branch was cut — acceptRetiredDefaultResidue: a retired DEFAULTED key's emitted default now parses as inert residue and strips silently at the SCHEMA layer; only a NON-default value keeps the #12497 tombstone. The real 17.1-built fixture carries exclusively the emitted default (150 × false, premise-guarded), so the old "current version + same keys refuses" pin described a shape the ruled contract now accepts. Reproduced exactly on the merged tree before changing anything.

Fixed in 7eff896 (on top of the origin/main merge f4a6b0c) by re-aiming, not weakening:

  • Refusal pin (kept, sharpened): current-version floor + a NON-default retired-key value (allowRestore: true — authorable on the 17.1 surface, never emitted by a default) still refuses with the tombstone and the full prescription.
  • New residue pin: current-version floor + pure emitted-default residue parses clean with the keys stripped by the SCHEMA layer while this door's versioned window stays closed (zero conversion notices) — pinning the boundary between the schema's silent residue strip and the door's noticed, version-gated strip, which is what keeps the door's amnesty versioned under the new contract.

The door policy itself is unchanged and still load-bearing: the schema tolerance covers only the emitted default, while pre-17.2 artifacts may legally carry true values that only the versioned forward conversion admits (with operator-visible notices).

Union on the merged tree at head 7eff896 (main merged at b489d3c), clean tree: all derived + convention gates green; check:dispatcher-error-vocabulary OK; full check:type-check-debt re-measure green under the CI-shaped heap ceiling; full-repo lint exit 0; metadata-core 14/14 and metadata 36/36 test files (door suite now 6 tests); runtime typecheck + the four vocabulary-consuming suites 69/69. Not re-enqueued per queue discipline — the PM re-enqueues after PR CI goes green.

Generated by Claude Code


Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

🤝 Handover pointer — this PR is code-complete and waiting on a tier-gated review, not on more work.

7eff896: 32/32 CI green, mergeable_state: clean, ready, not enqueued (auto-merge did not survive the earlier dequeue, whose root cause was main's #12845 landing mid-flight — diagnosed on the merged tree and the pin re-aimed, not weakened).

needs:contract-review is armed on this PR and on the card because the head moved after the previous PASS, and the reviewing seat dropped below CONTRACT_REVIEW_TIER mid-flight. Maintainer's instruction, 2026-08-28, verbatim: 「12843 也让新项目经理用 fable 审核」.

The incoming PM's step-by-step — tier proof, the exact delta to review (one test file; everything else in b02ec81...7eff896 came from the merge of main), the clear-and-land sequence, and the post-merge checks — is the handover comment on #12772. The claim on that card is released so it can be claimed cleanly.


Generated by Claude Code

@hotlong
hotlong added this pull request to the merge queue Aug 28, 2026
Merged via the queue into main with commit 54e2d36 Aug 28, 2026
39 checks passed
@hotlong
hotlong deleted the claude/issue-12772-artifact-conversion-door branch August 28, 2026 06:13
os-litant pushed a commit that referenced this pull request Aug 28, 2026
`applyArtifactForwardConversions` — the function this card's fix consumes —
does not exist on origin/main; it is added by PR #12843 (open, unmerged).
Merged here so the AppPlugin change can be written and MEASURED against the
same artifact door it must agree with. This merge disappears when #12843
lands.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/xl tests tooling

Projects

None yet

3 participants