Repository navigation
Stop Bedrock Claude 400s when replayed thinking no longer matches - #189
Merged
Merged
Conversation
@ai-sdk/amazon-bedrock renames thinking.block_binding.prefix_mismatch_behavior to mismatch_behavior on the InvokeModel route (vercel/ai#20728). AWS now documents mismatch_behavior as a deprecated alias, and some regions ignore it, so a changed prefix before replayed thinking fails with "The block is bound to a different conversation" despite drop_block. The SDK renames before it signs, so the Anthropic-route fetch now renames the field back and re-signs with the SDK's own SigV4 scope. Bodies without the field pass through untouched, so this is a no-op once the SDK stops renaming (vercel/ai#24919) and can then be removed.
…bodies Fetch sends header values as Latin-1 while SigV4 hashes UTF-8, so signing a non-ASCII custom header produced a signature the server rejects. Match the SDK and exclude those headers from the signature while still sending them. Extend the wire tests to cover bearer auth and a gateway signed for a non-default region.
The mocked 400 makes AI SDK's default error logging print to stderr. Capture it in send() and assert the single expected error instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Claude binds each thinking block to the conversation that produced it. When we replay thinking from an earlier turn and something before it has changed (for example, the system prompt), the API rejects the request with 400 "The block is bound to a different conversation". Since ai-lib #138, our Anthropic requests avoid this by sending
thinking.block_binding.prefix_mismatch_behavior: "drop_block", which tells the API to drop the stale block and continue. Bedrock users have still been hitting the 400 intermittently. This PR makes the binding reach Bedrock under the field name AWS documents.The cause is in
@ai-sdk/amazon-bedrock. For Claude over InvokeModel (Bedrock's Anthropic Messages route), it renames the field tomismatch_behavior(vercel/ai#20728). AWS documents that name as a deprecated alias, and Bedrock has not honored it in every region: the upstream fix (vercel/ai#24919, still open) notes that us-east-1 accepted it while eu-central-1 rejected it. Where the alias is ignored, Bedrock sees no binding and fails the request. One affected user confirmed that sendingprefix_mismatch_behaviorfixes it.Before and after, the
thinking.block_bindingwe send on InvokeModel:The fix
The SDK renames the field and signs the request with SigV4 before it calls the
fetchwe pass in, with no hook in between. Because SigV4 signs a hash of the body, renaming the field in our fetch alone would break the signature for everyone using AWS credentials. So the fix is a fetch wrapper that renames the field back and signs the request again:withPrefixMismatchBehaviorinmodel-clients/bedrock-block-binding-fetch.ts.BedrockClient.createModelinstalls it only on the InvokeModel route; Claude over the Converse API and OpenAI models over Mantle never send a binding, so they are unchanged.The wrapper changes as little as it can:
It acts only on the old field. It rewrites a request only when the body has
thinking.block_binding.mismatch_behaviorand noprefix_mismatch_behavior. All other requests, including those without a binding, pass through untouched.It changes one field and the signature. On SigV4 requests it replaces the SDK's signature headers (
authorization,x-amz-date,x-amz-security-token,x-amz-content-sha256) and keeps the URL and other headers. The body is re-serialized as JSON with only the field renamed.It reuses the SDK's signing choices. Region and service come from the credential scope in the SDK's own
Authorizationheader, so Posit Connect's Bedrock gateway and custom endpoints stay signed for the region they verify against. Credentials come from the same provider the SDK used. Signing uses aws4fetch, which the bridge already depends on.Bearer tokens need only the rename. With
AWS_BEARER_TOKEN_BEDROCK, nothing is signed, so the wrapper only rewrites the body.What to watch for
The raw-HTTP logging fetch sits inside the wrapper, so request captures show the body actually sent, not the SDK's version. Once the SDK stops renaming, the wrapper finds nothing to rename and passes everything through, so it can be deleted then. It never adds a second field, which matters because Bedrock rejects a body that has both.
Testing
A new wire test runs the real
BedrockClientand SDK against a stubbed fetch. It checks that the body carriesprefix_mismatch_behavior(this failed before the fix) and that theAuthorizationheader is a valid SigV4 signature over the body actually sent.Against live Bedrock (us-east-1, Opus 5.5, SSO credentials with a session token), the patched client streamed a normal response with
prefix_mismatch_behaviorin the body. The failing case could not be reproduced: in us-east-1 the alias works, and our test account does not reject mismatched thinking unless a binding asks it to. The fix therefore rests on the user report and AWS's documentation.Alternatives considered
node_modules: neither ai-lib nor the Posit Assistant monorepo has patching set up, and every consumer of ai-lib would have to apply the patch.