Repository navigation
test(stream): pairing invariant + real-stream fixtures + kill coverage (M14 slice 1) - #186
Merged
Merged
Conversation
…e (M14 slice 1, #185) Enforce the pairing invariant at the sink: every id-bearing tool_start must be followed by exactly one tool_end with the same tool_call_id before agent_end. Breaches (unpaired/duplicate end, duplicate start, reused id, call still open at agent_end) are counted in stats.pairing_violations, listed on sink.pairingViolations, and written to the execution log as raw records (reason: pairing_violation) — never silently dropped, never fatal. checkPairing() exposes the same state machine as a pure function for replay checks. Fixtures: claude-error-killed + codex-killed streams (faithfully reconstructed — no live CLI credentials in this environment; see test/fixtures/agent-stream/README.md), covering error results, concurrent calls with out-of-order results, and killed runs. Killed calls close as abandoned with tool_call_id + agent_run_id and count in stats.events.
This was referenced Nov 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #185 (M14 slice 1).
What this changes
src/core/agent-stream.js): inside oneagent_run_idscope, every id-bearingtool_startmust be followed by exactly onetool_endwith the sametool_call_idbeforeagent_end. Breaches are surfaced three ways at once — counted instats.pairing_violations, listed onsink.pairingViolations, and written to the execution log as arawrecord (reason: 'pairing_violation', with aviolationkind field). The offending event itself is still recorded, and a breach never stops the run. The log record matters becauseagent-invokediscards theclose()stats return, so stats alone would be invisible on a real run.unpaired_tool_end,duplicate_tool_end,duplicate_tool_start,reused_tool_call_id(an id names one call inside a run),unpaired_at_agent_end. Id-less calls are still untracked — nothing to pair them to.checkPairing(events)exposes the same state machine as a pure function over normalized events, for replay checks (slice 2).claude-error-killed.ndjson(errortool_result, two concurrent calls whose results arrive out of order, a killed call with no result and noresultline) andcodex-killed.ndjson(item.startedwith noitem.completed/turn.completed). Killed calls close asabandonedwithtool_call_id+agent_run_id, counted instats.events.docs/workflow-manual.mdnow states the violation surfacing.No verdict/threshold changes, no check/comment ledger changes, no new adapter.
Honest note — fixtures are reconstructed, not recorded
No live CLI recording was possible in this environment: the installed codex CLI reports "Not logged in" and no claude CLI is installed (no CLI credentials available to this worker). The new fixtures were faithfully reconstructed from the real stream formats — claude
--output-format stream-json(system/assistant/user/resultlines,tool_resultcarrying onlytool_use_id) and codex--json(thread.started/item.started/item.completed/turn.completed) — using the pre-existingclaude.ndjson/codex.ndjsonfixtures as shape references. This is documented intest/fixtures/agent-stream/README.md. Slice 2 replaces/extends these with >=3 streams recorded from real CLI runs.Tests
node --test test/agent-stream.test.js test/execution-log.test.js→ 35 pass, 0 fail (13 new: 7checkPairingunit cases + 6 sink cases).npm test: everything passes except 4 pre-existing failures, all in the M3/I6d assertion-execution gate and reproduced identically on unmodifiedmain(37a1b1a, separate worktree):test/i6d-acceptance-demo.test.js(1) andtest/xflow-audit.test.js(3) — real-xflow auditCLI environment failures, unrelated to this change. No other file fails;test/bench-harness-isolation.test.jspasses (1/1).agent_endviolations surfaced with the offending events kept in the log, and the same claude ids replayed through two sinks with differentagent_run_ids staying paired with zero cross-talk.