Skip to content

Stop Bedrock Claude 400s when replayed thinking no longer matches - #189

Merged
wch merged 3 commits into
mainfrom
bedrock-prefix-mismatch-behavior
Nov 12, 2026
Merged

wch merged 3 commits into
mainfrom
bedrock-prefix-mismatch-behavior

Conversation

@wch

@wch wch commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator

Claude binds each thinking block to the conversation that produced it. When we replay thinking from an earlier turn and something before it has changed (for example, the system prompt), the API rejects the request with 400 "The block is bound to a different conversation". Since ai-lib #138, our Anthropic requests avoid this by sending thinking.block_binding.prefix_mismatch_behavior: "drop_block", which tells the API to drop the stale block and continue. Bedrock users have still been hitting the 400 intermittently. This PR makes the binding reach Bedrock under the field name AWS documents.

The cause is in @ai-sdk/amazon-bedrock. For Claude over InvokeModel (Bedrock's Anthropic Messages route), it renames the field to mismatch_behavior (vercel/ai#20728). AWS documents that name as a deprecated alias, and Bedrock has not honored it in every region: the upstream fix (vercel/ai#24919, still open) notes that us-east-1 accepted it while eu-central-1 rejected it. Where the alias is ignored, Bedrock sees no binding and fails the request. One affected user confirmed that sending prefix_mismatch_behavior fixes it.

Before and after, the thinking.block_binding we send on InvokeModel:

"block_binding": { "mismatch_behavior": "drop_block" }
"block_binding": { "prefix_mismatch_behavior": "drop_block" }

The fix

The SDK renames the field and signs the request with SigV4 before it calls the fetch we pass in, with no hook in between. Because SigV4 signs a hash of the body, renaming the field in our fetch alone would break the signature for everyone using AWS credentials. So the fix is a fetch wrapper that renames the field back and signs the request again: withPrefixMismatchBehavior in model-clients/bedrock-block-binding-fetch.ts. BedrockClient.createModel installs it only on the InvokeModel route; Claude over the Converse API and OpenAI models over Mantle never send a binding, so they are unchanged.

The wrapper changes as little as it can:

  • It acts only on the old field. It rewrites a request only when the body has thinking.block_binding.mismatch_behavior and no prefix_mismatch_behavior. All other requests, including those without a binding, pass through untouched.

  • It changes one field and the signature. On SigV4 requests it replaces the SDK's signature headers (authorization, x-amz-date, x-amz-security-token, x-amz-content-sha256) and keeps the URL and other headers. The body is re-serialized as JSON with only the field renamed.

  • It reuses the SDK's signing choices. Region and service come from the credential scope in the SDK's own Authorization header, so Posit Connect's Bedrock gateway and custom endpoints stay signed for the region they verify against. Credentials come from the same provider the SDK used. Signing uses aws4fetch, which the bridge already depends on.

  • Bearer tokens need only the rename. With AWS_BEARER_TOKEN_BEDROCK, nothing is signed, so the wrapper only rewrites the body.

What to watch for

The raw-HTTP logging fetch sits inside the wrapper, so request captures show the body actually sent, not the SDK's version. Once the SDK stops renaming, the wrapper finds nothing to rename and passes everything through, so it can be deleted then. It never adds a second field, which matters because Bedrock rejects a body that has both.

Testing

A new wire test runs the real BedrockClient and SDK against a stubbed fetch. It checks that the body carries prefix_mismatch_behavior (this failed before the fix) and that the Authorization header is a valid SigV4 signature over the body actually sent.

Against live Bedrock (us-east-1, Opus 5.5, SSO credentials with a session token), the patched client streamed a normal response with prefix_mismatch_behavior in the body. The failing case could not be reproduced: in us-east-1 the alias works, and our test account does not reject mismatched thinking unless a binding asks it to. The fix therefore rests on the user report and AWS's documentation.

Alternatives considered

wch added 3 commits November 12, 2026 13:08
@ai-sdk/amazon-bedrock renames thinking.block_binding.prefix_mismatch_behavior
to mismatch_behavior on the InvokeModel route (vercel/ai#20728). AWS now
documents mismatch_behavior as a deprecated alias, and some regions ignore it,
so a changed prefix before replayed thinking fails with "The block is bound to
a different conversation" despite drop_block.

The SDK renames before it signs, so the Anthropic-route fetch now renames the
field back and re-signs with the SDK's own SigV4 scope. Bodies without the
field pass through untouched, so this is a no-op once the SDK stops renaming
(vercel/ai#24919) and can then be removed.
…bodies

Fetch sends header values as Latin-1 while SigV4 hashes UTF-8, so signing a
non-ASCII custom header produced a signature the server rejects. Match the
SDK and exclude those headers from the signature while still sending them.

Extend the wire tests to cover bearer auth and a gateway signed for a
non-default region.
The mocked 400 makes AI SDK's default error logging print to stderr. Capture
it in send() and assert the single expected error instead.
@wch
wch enabled auto-merge (squash) November 12, 2026 20:49
@wch
wch merged commit 5f1d97a into main Nov 12, 2026
4 checks passed
@wch
wch deleted the bedrock-prefix-mismatch-behavior branch November 12, 2026 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant