Skip to content

test(stream): pairing invariant + real-stream fixtures + kill coverage (M14 slice 1) - #186

Merged
surlymochan merged 1 commit into
mainfrom
fix/m14s1-pairing-invariant-185
Nov 12, 2026
Merged

surlymochan merged 1 commit into
mainfrom
fix/m14s1-pairing-invariant-185

Conversation

@surlymochan

Copy link
Copy Markdown
Owner

Fixes #185 (M14 slice 1).

What this changes

  • Pairing invariant enforced at the sink (src/core/agent-stream.js): inside one agent_run_id scope, every id-bearing tool_start must be followed by exactly one tool_end with the same tool_call_id before agent_end. Breaches are surfaced three ways at once — counted in stats.pairing_violations, listed on sink.pairingViolations, and written to the execution log as a raw record (reason: 'pairing_violation', with a violation kind field). The offending event itself is still recorded, and a breach never stops the run. The log record matters because agent-invoke discards the close() stats return, so stats alone would be invisible on a real run.
  • Violation kinds: unpaired_tool_end, duplicate_tool_end, duplicate_tool_start, reused_tool_call_id (an id names one call inside a run), unpaired_at_agent_end. Id-less calls are still untracked — nothing to pair them to.
  • checkPairing(events) exposes the same state machine as a pure function over normalized events, for replay checks (slice 2).
  • Fixtures: claude-error-killed.ndjson (error tool_result, two concurrent calls whose results arrive out of order, a killed call with no result and no result line) and codex-killed.ndjson (item.started with no item.completed / turn.completed). Killed calls close as abandoned with tool_call_id + agent_run_id, counted in stats.events.
  • Docs: the pairing lifecycle paragraph in docs/workflow-manual.md now states the violation surfacing.

No verdict/threshold changes, no check/comment ledger changes, no new adapter.

Honest note — fixtures are reconstructed, not recorded

No live CLI recording was possible in this environment: the installed codex CLI reports "Not logged in" and no claude CLI is installed (no CLI credentials available to this worker). The new fixtures were faithfully reconstructed from the real stream formats — claude --output-format stream-json (system/assistant/user/result lines, tool_result carrying only tool_use_id) and codex --json (thread.started/item.started/item.completed/turn.completed) — using the pre-existing claude.ndjson/codex.ndjson fixtures as shape references. This is documented in test/fixtures/agent-stream/README.md. Slice 2 replaces/extends these with >=3 streams recorded from real CLI runs.

Tests

  • node --test test/agent-stream.test.js test/execution-log.test.js → 35 pass, 0 fail (13 new: 7 checkPairing unit cases + 6 sink cases).
  • Full npm test: everything passes except 4 pre-existing failures, all in the M3/I6d assertion-execution gate and reproduced identically on unmodified main (37a1b1a, separate worktree): test/i6d-acceptance-demo.test.js (1) and test/xflow-audit.test.js (3) — real-xflow audit CLI environment failures, unrelated to this change. No other file fails; test/bench-harness-isolation.test.js passes (1/1).
  • New coverage: clean replay of both killed fixtures (0 violations, every start paired exactly once incl. the abandoned end), orphan end / duplicate end / open-at-agent_end violations surfaced with the offending events kept in the log, and the same claude ids replayed through two sinks with different agent_run_ids staying paired with zero cross-talk.

…e (M14 slice 1, #185)

Enforce the pairing invariant at the sink: every id-bearing tool_start
must be followed by exactly one tool_end with the same tool_call_id
before agent_end. Breaches (unpaired/duplicate end, duplicate start,
reused id, call still open at agent_end) are counted in
stats.pairing_violations, listed on sink.pairingViolations, and written
to the execution log as raw records (reason: pairing_violation) — never
silently dropped, never fatal. checkPairing() exposes the same state
machine as a pure function for replay checks.

Fixtures: claude-error-killed + codex-killed streams (faithfully
reconstructed — no live CLI credentials in this environment; see
test/fixtures/agent-stream/README.md), covering error results,
concurrent calls with out-of-order results, and killed runs. Killed
calls close as abandoned with tool_call_id + agent_run_id and count in
stats.events.
@surlymochan
surlymochan merged commit 8081e71 into main Nov 12, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

M14 slice 1: pairing invariant + real-stream fixtures + kill coverage

1 participant