Repository navigation
Restore a placeholder a stream-mode reply plugin splits in two - #340
Merged
Merged
Conversation
In stream mode (`reply: "stream"`) the gateway restored placeholders in each string a plugin handed back on its own. A plugin that holds back the end of its text can cut a `<<TW_SECRET_n>>` placeholder in two; neither half matches the ledger, so the client received the placeholder text instead of the original value. Whole (non-streaming) bodies were affected the same way, since a stream-mode plugin gets the block in `onReplyText` and then `onReplyTextEnd`, and the next plugin in the chain saw the two unrestored halves. Each plugin now keeps, per text block, a trailing piece of its output that is the unfinished start of a placeholder in the ledger, and restores it together with the next piece it hands back. At the end of the block the held piece is joined with the `onReplyTextEnd` output; an incomplete leftover goes out as is. Only placeholders in the ledger are held, so a `<<` in code waits at most for the plugin's next piece. When a plugin is skipped after an error, the held piece is given back with the rest. Also fixed on the same path: a stream-mode plugin skipped after failing in `onReplyTextEnd` lost the text it was holding, because the block's state was removed before the call. The state is now kept until the block ends. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When several text blocks are open at once and a stream-mode plugin is skipped after failing in one of them, later pieces of the other blocks went straight through while what the plugin was still holding in those blocks (its held text, a held half placeholder, the tail kept back so a secret is not handed over in halves) only came out when the block ended, after the later pieces. That held text now goes out in front of the next piece of its block. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
In stream mode (
reply: "stream") the gateway restored placeholders in each string a reply plugin handed back on its own (Chain::stream_calland theonReplyTextEndoutput, which whole bodies also go through). A plugin that holds back the end of its text can cut a<<TW_SECRET_n>>placeholder in two. Neither half matches the ledger, so the client received the placeholder text instead of the original value, and the next plugin in the chain saw the two unrestored halves.Change
Bridge::unfinished_from: from which byte a string ends in the unfinished start of a placeholder in the ledger (binary search over the placeholders, same approach ashold_fromfor values). Only ledger placeholders count, so a<<in code waits at most for the plugin's next piece.onReplyTextEndoutput; an incomplete leftover goes out as is. On the skip-on-error paths the held piece is given back with the rest.onReplyTextEndlost the text it was holding, because the block's state was removed before the call. The state is now kept until the block ends.Tests
New tests in
plugin/reply/testsuse a stream plugin that hands back all but the last few characters and releases them inonReplyTextEnd, with a secret from the request in the answer. All of them failed before the change (the client gotuse <<TW_SECRET_1>> now; the end-of-block skip case got an empty block):a_placeholder_a_stream_plugin_splits_is_restored_from_both_halves(several delta splits, one and two plugins, two chunk sizes)the_next_plugin_in_the_chain_sees_whole_placeholderstext_given_back_when_a_plugin_is_skipped_comes_out_restoreda_stream_plugin_failing_at_the_end_of_a_block_gives_back_what_it_heldwhole_bodies_restore_a_placeholder_a_stream_plugin_splits(all four formats)a_plugin_skipped_in_one_block_gives_back_another_open_blocks_text_in_order(two interleaved Anthropic text blocks; before the change the second block came out reordered)Plus a unit test for
Bridge::unfinished_from.cargo fmt --check,cargo clippy --workspace --all-targets -D warningsandcargo test --workspacepass locally.🤖 Generated with Claude Code