Skip to content

refactor(artifacts): make child outputs exact and retire stale lifecycle state - #4808

Open
Astro-Han wants to merge 25 commits into
apache:mainfrom
Astro-Han:refactor/4788-disposable-artifacts
Open

refactor(artifacts): make child outputs exact and retire stale lifecycle state#4808
Astro-Han wants to merge 25 commits into
apache:mainfrom
Astro-Han:refactor/4788-disposable-artifacts

Conversation

@Astro-Han

Copy link
Copy Markdown
Contributor

Summary

  • return only child-produced Artifact sources from child result artifactIds
  • replace tombstones and status transitions with physical deletion and Session-scoped authority
  • retire unused sources, global reads/deletes, change notifications, readiness shims, and duplicate metadata abstractions
  • simplify SQLite Artifact metadata and discard unsupported or malformed Artifact rows without blocking Session startup
  • remove Artifact-specific purge and publication recovery journals; uncommitted Artifact residue is disposable and cannot block the Runtime Host

Compatibility boundary

Artifact data from older Nightly builds is best-effort and may be discarded during upgrade. Session metadata, messages, RuntimeEvent history, and continued conversation remain outside this disposal boundary. Sources required for continuation—session effects, archived tool results, projected images, uploads, deep research output, and subagent writeback—remain durable.

Validation

  • npm run build
  • npm run typecheck
  • npm run format:check
  • npm run lint
  • ASF license header audit
  • complete @maka/storage suite: 1111 tests, 1103 passed, 8 platform skips, 0 failures
  • Runtime Host reopen-image, archived-tool-result, Session-effect idempotency, and subagent-patch continuation tests
  • source-by-source ablation tests; every retained source has a failing removal counterexample

Closes #4788.

@github-actions github-actions Bot added the effort/XXL Over 2500 readable lines label Sep 4, 2026
@Astro-Han
Astro-Han force-pushed the refactor/4788-disposable-artifacts branch from 4e124ee to 9160177 Compare September 5, 2026 02:40

@jackwener jackwener left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at exact head 4ca24f0ae9ae9cb3e8805247c46e07782ce85761. [P1] — not approving. One P2 alongside it. Three independent reviews reached the same migration finding from different directions; a fourth is still running its deletion-surface pass and I will post separately if it changes anything.

The simplification itself is genuine and I am not asking for any of it back. Tombstones, the purge and publication journals, recovery and orphan state, and the duplicate repository abstraction are really gone — net −1,655 lines. The finding below asks for a correct cutover, not for restoring steady-state machinery.

[P1] The v1→v3 migration deletes continuation-durable Artifacts wholesale

sqlite-artifact-schema.ts:24-30 sees the ordinary v1 status / storage_key columns — which is what main has today, schema 1 — and issues DROP TABLE artifact_records, then creates an empty one. It never classifies rows. Retired sources and retained sources are removed alike.

The codec's skip of unsupported or malformed rows (artifact-metadata-codec.ts:48-67) is the correct, source-selective fail-open, and it is well built. It never sees a v1 row, because the table is gone before it runs.

SqliteArtifactStore.load() now only calls metadataRepository.readAll(); recover() is gone and bytes under artifacts/ are not reindexed. So after the upgrade, getInSession, listTurnArtifacts and readDurableAttachmentBinary return nothing for every source — including the six the Compatibility boundary names as durable.

Reproduced end to end, writing with the exact base c180a2ba production writers and reading with head:

Result
Base writers produce live rows for all seven retained sources valid on base
Open the same root on head v3 count 0; all seven reads not_found
Payload files under artifacts/ still on disk, now orphaned

Two consequences were reproduced independently through base writer/coordinator → head reader/coordinator:

  • A completed session_effect loses its durable idempotency. On base the effect resolved with one model call and the result base recap. After the upgrade, the same sessionId / effectId / reason invoked the model again and committed a different head recap. An effect that had already run once runs twice across an upgrade, with a different outcome.
  • A real archived tool result becomes unreadable. Readable on base; after the upgrade the same durable RuntimeEvent placeholder resolves not_found.

This is universal for current-main state roots, not a corner case. And it contradicts the boundary the PR declares: "Session metadata, messages, RuntimeEvent history, and continued conversation remain outside this disposal boundary. Sources required for continuation … remain durable." The first sentence holds — the Session opens. The second does not.

The shape is the one from #4779: startup and decode stay up while the rest of the chain does not. A test in this PR states the behaviour outright — opens existing Sessions after retiring v1 Artifact metadata (operational-state-store.test.ts:169-224) inserts a live v1 row and asserts session metadata survives and artifact_records count is 0. The catalog being emptied is currently a pinned contract rather than an accident, which is why it needs a decision rather than a patch.

Fix — a selective one-time migration. Preserve valid live retained-source rows in v3 shape; explicitly discard deleted, retired and malformed rows and handle their bytes. That lets the codec's skip do the job it was written for. Regressions should be base-shaped: effect retry, archive read, projection/upload/writeback reopen, and retired-row disposal.

[P2] Change notifications were deleted rather than replaced

artifacts.subscribeChanges and artifacts:changed are gone. The pane refreshes only on sessionId / refresh identity (artifact-pane.tsx:167-172) and after local actions, so a file created during the same session does not appear until a remount or a session switch.

On "exact"

Worth stating plainly because the word carries two meanings here. It is a projection filter, not a rejection. finalizeAndListChildTurnArtifacts (session-manager.ts:1046) filters child results to tool_result / tool_result_projection / subagent_writeback / deep_research and drops tool_result_archive / user_upload / session_effect; storage's listTurnArtifacts remains unfiltered.

The resulting failure semantics are mixed, and the mix is deliberate in most rows:

Gate Inexact input Behaviour
create() with an unknown source throws Invalid Artifact source fail-closed
protocol artifactSource invalidProtocolFrame fail-closed
missing subagent_writeback after finalize throws fail-closed
child-result list, wrong source silently omitted fail-open
metadata JSON or retired source row skipped fail-open
v1 schema whole table dropped fail-open session, fail-closed catalog

The last row is the P1: the session opens and the catalog is gone.

Historical over-broad typed child results stay decoder-compatible, and accepting old id-only transcript records is necessary compatibility rather than a gap. The description's durable list also omits tool_result itself, which the enum retains — minor, but worth fixing whenever the description is next edited.

简体中文

4ca24f0ae9ae9cb3e8805247c46e07782ce85761 上评审。[P1]——不予批准。 另有一条 P2。三次独立评审从不同方向得到同一个迁移发现;第四席仍在跑删除面,若有变化我会另发。

这次简化本身是真实的,我不要求恢复其中任何一部分。 墓碑、purge 与 publication 日志、recovery 与 orphan 状态、重复的仓储抽象确实都没了——净 −1,655 行。下面这条要求的是一次正确的切换,而不是把稳态机制搬回来。

[P1] v1→v3 迁移整批删除了 continuation-durable 的 Artifact

sqlite-artifact-schema.ts:24-30 看到普通的 v1 status / storage_key 列——那正是今天 main 的样子,schema 1——于是 DROP TABLE artifact_records,再建一张空表。它从不对行做分类。 已退役的 source 与保留的 source 一并被清除。

codec 里那个「跳过不支持或损坏行」的逻辑(artifact-metadata-codec.ts:48-67)才是正确的、按 source 选择性的 fail-open,而且写得很好。它永远看不到一条 v1 行,因为表在它运行之前就没了。

SqliteArtifactStore.load() 现在只调用 metadataRepository.readAll();recover() 已删,artifacts/ 下的字节不再被重新索引。 所以升级之后,getInSessionlistTurnArtifactsreadDurableAttachmentBinary每一个 source 都返回空——包括 Compatibility boundary 点名为 durable 的那六类。

端到端复现,用精确的 base c180a2ba 生产 writer 写入、用 head 读取:

结果
base writer 为全部七个保留 source 产出活行 base 上有效
在 head 上打开同一个 root v3 计数 0;七个读取全部 not_found
artifacts/ 下的 payload 文件 仍在磁盘上,已成孤儿

有两条后果通过 base writer/coordinator → head reader/coordinator 各自独立复现:

  • 一个已完成的 session_effect 失去了它的持久幂等性。 在 base 上,该 effect 以一次模型调用解决,结果是 base recap升级之后,同样的 sessionId / effectId / reason 再次调用了模型,并提交了不同的 head recap一个已经跑过一次的 effect,跨升级又跑了一次,而且结果不同。
  • 一个真实的归档工具结果变得不可读。 在 base 上可读;升级后,同一个持久 RuntimeEvent 占位符解析为 not_found

这对当前 main 的状态根是普遍成立的,不是边角情形。 而且它与本 PR 声明的边界相矛盾:「Session 元数据、消息、RuntimeEvent 历史与继续中的对话留在这个处置边界之外。continuation 所需的 source……保持 durable。」 前一句成立——Session 打得开。后一句不成立。

这个形状与 #4779 相同:启动与解码活着,链路其余部分不活。 而本 PR 里有一条测试把这个行为直说了出来——opens existing Sessions after retiring v1 Artifact metadata(operational-state-store.test.ts:169-224)插入一条活的 v1 行,然后断言 session 元数据存活artifact_records 计数为 0「目录被清空」目前是一条被钉住的合同,而不是一次意外,所以它需要的是一个决定,而不是一个补丁。

修法——一次选择性的一次性迁移。 把有效的、活的、保留 source 的行按 v3 形状保留下来;显式丢弃已删除、已退役与损坏的行并处理它们的字节。那样 codec 里那个 skip 才能真正做它被写出来要做的事。 回归测试应当是 base 形状的:effect 重试、归档读取、projection/upload/writeback 重开,以及退役行的处置。

[P2] 变更通知是被删掉而不是被替换

artifacts.subscribeChangesartifacts:changed 都没了。面板只在 sessionId / refresh 身份变化时以及本地操作之后刷新(artifact-pane.tsx:167-172),所以同一个会话期间创建的文件,要等到重新挂载或切换会话才会出现。

关于「exact」

值得直说,因为这个词在这里有两重含义。它是一个投影过滤器,不是一次拒绝。 finalizeAndListChildTurnArtifacts(session-manager.ts:1046)把 child 结果过滤为 tool_result / tool_result_projection / subagent_writeback / deep_research,丢掉 tool_result_archive / user_upload / session_effect;而 storage 的 listTurnArtifacts 仍然不过滤。

由此得到的失败语义是混合的,而且大多数行的混合是有意的:

不 exact 的输入 行为
create() 未知 source Invalid Artifact source fail-closed
protocol artifactSource invalidProtocolFrame fail-closed
finalize 后缺 subagent_writeback 抛错 fail-closed
child 结果列表,source 不对 静默省略 fail-open
metadata JSON 或已退役 source 跳过该行 fail-open
v1 schema 整表 drop session fail-open,catalog fail-closed

最后一行就是那条 P1:会话打得开,而目录没了。

历史上过宽的 typed child result 仍与解码器兼容,而接受只有 id 的旧 transcript 记录是必要的兼容性,不是缺口。另外描述里那份 durable 清单漏了 tool_result 本身(枚举保留了它)——小事,但下次编辑描述时可以顺手补上。


Automated review notice: This comment was posted by an automated review agent operated by jackwener. It is not an independent human review and does not replace one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/XXL Over 2500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A child result's artifactIds is every Artifact on the turn, not the child's output

2 participants