fix: classify harness message shapes that drifted in since 2026-09-02 - #17
Merged
Merged
Conversation
…orts
The 2026-09 drift scan found seven more harness message shapes the
parser still rendered as plain user turns or double-counted as K's
denominator:
- The workflow harness's two framing entries (relayed user request,
computed task) at the top of every subagent workflow transcript.
- The handback-send-enforce and structured-output-enforce nudges, and
the two cut-off-mid-stream resume wordings.
- A Stop hook's condition-evaluation report ("Stop hook feedback:"),
distinct from the goal-activation notice already handled.
- The singular "Background agent "<desc>" was stopped by the user."
wording, which agentsStoppedCount's leading-count regex never
matched.
- Compaction-summary detection now checks the top-level
isCompactSummary field first: classifyHarnessUserMessage's prefix
match ran after the teammate-tag and <task-notification> Contains
checks, so a summary whose restated body quoted either tag was
misclassified, and CLI 2.1.274's
<artifact-content-authored-by-others/> preamble defeated the prefix
match outright. The prefix match still covers transcripts that never
wrote the field.
CountsAsTurn also gets a narrow promptSource carve-out: every sampled
agents-stopped notice (singular and plural) carries
promptSource="system", which the existing blanket "promptSource always
counts" rule turned into a turn despite ADR-008 measuring 0% for this
shape — the same reasoning ADR-009 already applies to promptSource="sdk".
Each shape's turn verdict is measured, not assumed, per the comment on
CountsAsTurn (which this change also reattaches to the right function —
a prior edit had left it doc-commenting IsCompactedHarnessInjection).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxSSNGDv4kKwZZ3oHdhbef
…ssages A teammate message that relays a subagent's final report now wraps it in a "[Subagent hand-back] ... The report follows:" preamble, with every line of the report indented (the harness's defense against a forged frame boundary at column zero). CompactTeammateMessage already classified and compacted these correctly via the outer <agent-message>/<teammate-message> tag, but left the preamble and the line-by-line indent in the compacted body verbatim. CompactTeammateMessage now recognizes the preamble's own bracket tag and strips it, reusing the dedentHarnessFrame helper the workflow frames already use for the same indent convention. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FxSSNGDv4kKwZZ3oHdhbef
Found by the harness drift 2026-09 scan: a fork's parent-context marker in a subagent transcript carries no "message" field, so it fell through parseLineWithToolCalls unparsed instead of becoming EventNoise — the same ADR-008 §1 gap the noiseTypes whitelist already exists to close for entry types the CLI adds over time. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FxSSNGDv4kKwZZ3oHdhbef
Adds the shapes found since 2026-09-02, the agents-stopped exception to ADR-009's promptSource rule, decision 7 (isCompactSummary before tag matching), and the deferred turnCompanion / turnOrigin fields. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FxSSNGDv4kKwZZ3oHdhbef
… set
render.go's harnessRender dispatch already renders IsWorkflowUserRequest
under the harness role (it relays the user's request, but inside a
harness-authored frame), contradicting IsCompactedHarnessInjection's doc
comment, which claimed it renders under the user role and excluded it from
the set. Add it to the set and rewrite the comment around the real
boundary: IsMidTurnUserMessage is the only flag that actually renders
under the user role.
This makes reader.go's ADR-009 human-promptSource reset and
CountsAsTurn's sdk exception apply to the flag too, closing a gap where
{IsWorkflowUserRequest: true, PromptSource: "sdk"} incorrectly counted as
a turn.
Pin both the enumeration and the reader-level ADR-009 interaction with
tests: a parameterized case per harness-drift-2026-09 flag against
IsCompactedHarnessInjection/IsClassifiedAsHarness (with a negative case
for IsMidTurnUserMessage so the table can't pass on an unconditional
true), the new CountsAsTurn sdk-exception row, and a reader-level test
mirroring the existing ADR-009 human-promptSource coverage.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxSSNGDv4kKwZZ3oHdhbef
…s, comment fixes - session.StopHookFeedbackPrefix becomes the single authoritative definition of the "Stop hook feedback:\n" header line, shared by classify.go's stopHookFeedbackPrefix (which anchors further, into the quoted condition's opening bracket) and CompactStopHookFeedback, instead of each package hardcoding its own copy. - Fix CompactStopHookFeedback to TrimSpace before TrimPrefix, matching the file's existing pattern (see CompactCoordinatorMessage): a leading newline in the raw entry previously left "Stop hook feedback:" sitting in the compact output. - Rename handbackSendEnforceTag/structuredOutputEnforceTag to .../EnforcePrefix, matching classify.go's convention for constants matched with HasPrefix. - Fix dedentHarnessFrame's doc comment: "(the harness ever emits a shorter one)" was missing "if". Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FxSSNGDv4kKwZZ3oHdhbef
…ure text Add the project's `Regression: <bug> (harness drift 2026-09)` marker to each bug-fix test's primary case (compaction summary misclassified on a quoted harness tag, Stop hook feedback/workflow frames/enforcement nudges rendered as user text, fork-context-ref falling through unparsed, and the Subagent hand-back preamble leaking into compacted teammate messages). Remove the label from the bare-phrase test in harness_classify_test.go — it guards new code against a false positive, not a past bug. Add a dedentHarnessFrame case (via CompactWorkflowComputedTask) where a nested-list body line carries 4+ spaces, pinning that only the fixed 2-space frame indent is stripped. Replace verbatim real content in test fixtures added on this branch with neutral made-up equivalents: the Stop hook feedback condition, a background-agent description, a subagent hand-back report body, and the workflow user-request text that is this session's own originating prompt. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FxSSNGDv4kKwZZ3oHdhbef
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Drift scan of 1,286 transcripts modified since 2026-09-02 (791 subagent, incl. the new
subagents/workflows/wf_*/layer; CLI 2.1.238–2.1.280). Details and counts are recorded in ADR-008 ("追加樣本（2026-09-23）" and decision 7).New shapes, previously rendered as
user:and counted as turns:[Workflow harness — user request]→[workflow: user request], not a turn (starts a turn 0/108)[Workflow harness — computed task]→[workflow: computed task], counts as a turn (95/108)[handback-send-enforce]/[structured-output-enforce]→[nudge: handback]/[nudge: structured output], turnYour response above was cut off mid-stream(two wordings) →[nudge: cut off], turnFixes to existing handling:
isCompactSummarybefore any tag match (3/37 summaries were misclassified because their body quoted teammate / task-notification tags, or had an<artifact-content-authored-by-others/>prefix)CompactTeammateMessagestrips the[Subagent hand-back] … The report follows:preamble (~400 chars, 69 messages)Stop hook feedback:→[goal feedback], turnBackground agent "…" was stopped by the user.is recognized; agents-stopped notices no longer count as turns even withpromptSource=system(measured 0/7; exception to ADR-009's rule, like 7a53260 for sdk)fork-context-refadded to noise typesDeferred: using
turnCompanion/turnOriginas turn signals (needs its own ADR alongside ADR-009).Verification
go vet,gofmt -l,go test ./... -count=1pass; every item has aRegression: … (harness drift 2026-09)test9fb49e85,b566530e,e04de0c1: compaction summaries now render asharness:+[compaction summary](main renders them verbatim)-n 20 -min-kb 100 -days 90 -overhead 40731 -model opus, same 20 sessions vs main: K rises in 11 of 20 sessions (largest:fc65aeeb11.8 → 12.7,82e7dc199.7 → 10.1) because fewer non-turn entries are counted as turns. Median cold 10-turn saving 65% → 64%, warm 100-turn 35% → 34%; compression median unchanged at 75.8%.🤖 Generated with Claude Code
https://claude.ai/code/session_01FxSSNGDv4kKwZZ3oHdhbef