Skip to content

Restore a placeholder a stream-mode reply plugin splits in two - #340

Merged
fylorn merged 2 commits into
mainfrom
fix/stream-placeholder-carry
Oct 16, 2026
Merged

fylorn merged 2 commits into
mainfrom
fix/stream-placeholder-carry

Conversation

@fylorn

@fylorn fylorn commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Problem

In stream mode (reply: "stream") the gateway restored placeholders in each string a reply plugin handed back on its own (Chain::stream_call and the onReplyTextEnd output, which whole bodies also go through). A plugin that holds back the end of its text can cut a <<TW_SECRET_n>> placeholder in two. Neither half matches the ledger, so the client received the placeholder text instead of the original value, and the next plugin in the chain saw the two unrestored halves.

Change

  • Bridge::unfinished_from: from which byte a string ends in the unfinished start of a placeholder in the ledger (binary search over the placeholders, same approach as hold_from for values). Only ledger placeholders count, so a << in code waits at most for the plugin's next piece.
  • Per plugin and per text block, a trailing unfinished placeholder in the plugin's output is held and restored together with the next piece it hands back. At the end of the block it is joined with the onReplyTextEnd output; an incomplete leftover goes out as is. On the skip-on-error paths the held piece is given back with the rest.
  • Same path: a stream-mode plugin skipped after failing in onReplyTextEnd lost the text it was holding, because the block's state was removed before the call. The state is now kept until the block ends.
  • Same path: when several text blocks are open at once and a plugin is skipped after failing in one of them, what it was still holding in the other blocks came out only when those blocks ended, after their later pieces. It now goes out in front of the next piece of its block.

Tests

New tests in plugin/reply/tests use a stream plugin that hands back all but the last few characters and releases them in onReplyTextEnd, with a secret from the request in the answer. All of them failed before the change (the client got use <<TW_SECRET_1>> now; the end-of-block skip case got an empty block):

  • a_placeholder_a_stream_plugin_splits_is_restored_from_both_halves (several delta splits, one and two plugins, two chunk sizes)
  • the_next_plugin_in_the_chain_sees_whole_placeholders
  • text_given_back_when_a_plugin_is_skipped_comes_out_restored
  • a_stream_plugin_failing_at_the_end_of_a_block_gives_back_what_it_held
  • whole_bodies_restore_a_placeholder_a_stream_plugin_splits (all four formats)
  • a_plugin_skipped_in_one_block_gives_back_another_open_blocks_text_in_order (two interleaved Anthropic text blocks; before the change the second block came out reordered)

Plus a unit test for Bridge::unfinished_from. cargo fmt --check, cargo clippy --workspace --all-targets -D warnings and cargo test --workspace pass locally.

🤖 Generated with Claude Code

fylorn and others added 2 commits October 16, 2026 16:56
In stream mode (`reply: "stream"`) the gateway restored placeholders in
each string a plugin handed back on its own. A plugin that holds back the
end of its text can cut a `<<TW_SECRET_n>>` placeholder in two; neither
half matches the ledger, so the client received the placeholder text
instead of the original value. Whole (non-streaming) bodies were affected
the same way, since a stream-mode plugin gets the block in `onReplyText`
and then `onReplyTextEnd`, and the next plugin in the chain saw the two
unrestored halves.

Each plugin now keeps, per text block, a trailing piece of its output that
is the unfinished start of a placeholder in the ledger, and restores it
together with the next piece it hands back. At the end of the block the
held piece is joined with the `onReplyTextEnd` output; an incomplete
leftover goes out as is. Only placeholders in the ledger are held, so a
`<<` in code waits at most for the plugin's next piece. When a plugin is
skipped after an error, the held piece is given back with the rest.

Also fixed on the same path: a stream-mode plugin skipped after failing in
`onReplyTextEnd` lost the text it was holding, because the block's state
was removed before the call. The state is now kept until the block ends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When several text blocks are open at once and a stream-mode plugin is
skipped after failing in one of them, later pieces of the other blocks
went straight through while what the plugin was still holding in those
blocks (its held text, a held half placeholder, the tail kept back so a
secret is not handed over in halves) only came out when the block ended,
after the later pieces. That held text now goes out in front of the next
piece of its block.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@fylorn
fylorn merged commit 7969eb1 into main Oct 16, 2026
6 checks passed
@fylorn
fylorn deleted the fix/stream-placeholder-carry branch October 16, 2026 09:27
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 16, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant