Skip to content

fix(testing): preserve producer JSON in CLI tuple results - #7654

Open
loopx-agent wants to merge 1 commit into
mainfrom
codex/cli-result-wire-identity
Open

loopx-agent wants to merge 1 commit into
mainfrom
codex/cli-result-wire-identity

Conversation

@loopx-agent

@loopx-agent loopx-agent commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

run_json_cli_result returned the process status separately but also injected _returncode into the decoded CLI JSON. This changed captured content before validation or replay and overwrote a producer-owned field with the same name. Return the original parsed JSON unchanged alongside the status.

This is a two-file testing-transport repair. The existing run_json_cli(..., include_returncode=...) behavior remains unchanged. CLI output, admission, signatures and settlement rules are unchanged.

Validation on rebased head 20f3efe099aa61c83029af8cb5903dbde93bfbcf against main cc8bd2f94c98f08bb13fa462bdd294cc550e7a81:

  • The six new cases cover absent/producer-owned metadata, successful/failed process status, and exact readback through the real capture CLI. They fail on the unchanged base and pass after the repair.
  • Six focused transport/caller suites: 52 passed, 13 skipped on the same rebased code; after the final conflict-free base refresh, the six new transport cases reran 6/6.
  • Ruff, Python compile, diff hygiene and the diff-scoped semantic advisory pass with no supported new vocabulary.
  • Standard local premerge passes 10/10 selected checks, including the semantic-vocabulary drift and maintainability checks.
  • Both changed files and this description passed the public-boundary scan.

The browser failures on the previous head were outside this diff: one exceeded a frozen lens-switch budget and one failed during worker cleanup. They are not evidence against the rebased head; fresh CI runs retain their separate gates.

No installed-client, model-outcome or performance qualification is claimed.

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent; gpt-6.1-sol; OpenAI; runtime_reported; reasoning_effort=max.

Request changes conclusion (author-owned PR; GitHub blocks formal self-review)

动机 / Motivation

维护者通过测试 transport 捕获 CLI JSON,验证内容或将原结果交给 capture reader 重放。旧 helper 把进程退出码写入 JSON,可能覆盖生产者同名字段;新实现把退出码保留在 tuple 第一项,使第二项保持原始 JSON。六个新增用例证明 JSON 保真修复有效,但原有 File/SQLite 来源路由测试因未迁移的 metadata 读取而失败。本 PR 不修改实际 CLI 输出、准入、签名、结算或安装客户端,也不证明完整 C1 或模型收益。

改动思路 / Approach

应让真实 CLI 决定返回内容,测试 helper 只负责进程调用、JSON 解码和退出码运输。直接删除对 payload 的赋值,比增加开关或另建 transport 更简单,且能保护生产者自有的同名字段。在已有transport内修复合理,必须把同部署metadata消费者一并迁移,避免拆出一个已知失败状态。两文件transport修复还需包含既有source-route消费者迁移;不要求整个C1。run_json_cli 的失败契约与 tuple API 不同,本轮 AST 对照确认其完全未改,不要求合并两函数或先完成父级 C1。

具体改动 / Changes

审阅完整两文件 diff(+51/-1),head 60678b742a3dfdd3d4674f4308b62eb988e34f6a,不可变 base 385eb853c1d37f5a285f2e8e0b875bd0ea04d3b3。依据修改前的 docs/development/testing-and-quality.md(revision 385eb853c1d37f5a285f2e8e0b875bd0ea04d3b3)中 Refactor Real-Path Gate:实际入口、相同输入的 base/head 对照与敏感性证据已经执行,但现有调用方兼容这一项尚未满足。

关键代码讲解

  • run_json_cli_result 保留命令、UTF-8 解码、空输出和对象检查,仅删除 _returncode 注入,并直接返回 (result.returncode, payload)。非零退出仍交给调用方判断,未改变真实 CLI 的签名或结算。
  • test_real_capture_cli_returns_the_same_wire_through_the_helper 对正确与错误 digest 比较实际 quota context-read 输出,并检查源 capture 不变、不创建 registry/runtime;另四例检查退出码与生产者字段。冻结相同六例:base 6 FAIL,head 6 PASS,证明修复针对真实旧缺陷。
  • 搜索全部 55 个定义/消费者文件发现 test_global_cli_check_and_manager_read_the_original_provider 仍依赖已删除 metadata;这处不在作者两文件 diff 中,却是同部署的实际消费者。

对主干的风险

[P2] 同步迁移仍读取 _returncode 的现有消费者。 helper 返回后,上述测试第 73 行无条件 actual.pop("_returncode")。两项 File/SQLite 用例在 base 均 PASS,在该 head 均因 KeyError: '_returncode' FAIL,尚未执行后续来源/provider、私有信息和只读不变断言。这是本次删除直接触发的回归,不能归为环境问题,也不能用新测试通过抵消。

最小修复是让该 caller 使用 tuple 中已返回的 code,直接检查未污染 JSON,同时保留其全部后续独立断言;无需恢复字段注入。重跑 uv run --extra test python -m pytest tests/control_plane/test_todo_source_runtime_route.py::test_global_cli_check_and_manager_read_the_original_provider -q,必须让两个 provider 都执行到原验收末尾。

另有 98 PASS / 0 skip 的五组相关 capture、Host、工作要求和来源导航测试,Ruff 及标准 premerge(6 direct、2 catalog、8 risk)均 PASS、0 manual hold。这些覆盖不包含上述遗漏,所以没有将预检绿色写成整体通过。没有查询或等待 CI。

语义与 CI 对齐

复用现有 tuple 契约,不新增 enum、CLI 或 authority。返回 JSON 的语义有意改变且已披露,但同部署 caller 未同步导致当前兼容性违规;先修消费者并重跑即可,不要求放宽测试或预算。这里没有 opt-in/default-off 声明,不以不存在功能对象冒充隔离证据。

我的整体评价

修复方向与规模合适,但存在可复现的调用方迁移缺口,本 head REQUEST_CHANGES。长程业务路径和用户交互维持原边界,当前结论不证明安装采用、模型效果或完整 C1;本次也没有渲染界面变化,无需视觉截图。新增六例的旧失败/新通过与现有两例的新回归均保留,修复后应重新核验完整当前 diff 与来源路由结果。

English verdict: REQUEST_CHANGES at 60678b7: migrate the existing tuple consumer; its File/SQLite cases pass on base and fail with KeyError on this head, despite six new cases and 98 related tests passing.

if not isinstance(payload, dict):
raise AssertionError(f"expected JSON object for {command!r}: {payload!r}")
payload["_returncode"] = result.returncode
return result.returncode, payload

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 删除注入后,现有 tests/control_plane/test_todo_source_runtime_route.py:73 仍无条件 actual.pop("_returncode")。同一 File/SQLite 测试在不可变 base 385eb85 两项通过,在此 head 两项均 KeyError,后续来源、provider、隐私与只读断言未执行。请同步迁移该同部署 caller,使用 tuple 已返回的 code 并保留全部原断言;无需恢复 payload 注入。

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
@huangruiteng
huangruiteng force-pushed the codex/cli-result-wire-identity branch from 4a8be4c to 20f3efe Compare October 29, 2026 12:00

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant