Repository navigation
Don't cache failed Bedrock Mantle model listings - #183
Merged
Merged
Conversation
listMantleModels turned every failure into [], and the direct Bedrock fetcher cached that [] for the one-hour TTL. After an expired AWS session and a re-auth outside the in-app flow (for example `aws sso login`), Bedrock listed Claude models but no OpenAI (Mantle) models for up to an hour. listMantleModels now returns [] only when a retry would give the same answer: a 404 (no Mantle in the region) or a recognized access-denied body. Every other failure (credential resolution, signature or token rejection, unrecognized 401/403, 5xx, malformed or unexpected bodies) returns null. getMantleModels caches arrays and never caches null, so the next listing retries. The Connect gateway consumer treats null as [], as before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a user's AWS session expires and they sign in again outside the app (for example with
aws sso login), the direct Bedrock provider lists Claude models on the next refresh but no OpenAI (Mantle) models for up to an hour. The Mantle listing turned every failure into an empty list, and the Bedrock model fetcher cached that empty list for the full one-hour cache lifetime. The Converse listing doesn't cache failures, so it recovered on the next refresh while Mantle didn't.This PR makes the Mantle listing distinguish a stable empty answer from a failure:
null: credential resolution failures (expired SSO), network errors, signature or expired-token rejections, unrecognized 401/403 bodies, other non-2xx responses, and malformed or unexpected bodies. The Bedrock fetcher never cachesnull, so the next refresh retries.A failure still lists no Mantle models for that refresh and never affects the Converse models. The Connect gateway consumer treats
nullas an empty list, so its behavior is unchanged.Not included