-
Notifications
You must be signed in to change notification settings - Fork 776
fix(routing): let fallback skip a candidate the request cannot fit #1850
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1969,9 +1969,14 @@ async function handleResponsesInner( | |
| if (parsed._compactionRequest !== true) { | ||
| const inputAdmission = checkInputAdmission(parsed, route.provider, route.providerName, parsed.modelId); | ||
| if (!inputAdmission.admitted) { | ||
| // #1524: this is a LOCAL preflight refusal, not an upstream verdict. A policy or combo | ||
| // fallback must be able to skip this candidate and try one whose context window fits, | ||
| // instead of treating the first incompatible candidate as the end of the chain. The | ||
| // distinct code is what lets the fallback layer tell the two apart -- an upstream | ||
| // `context_length_exceeded` still stops, because retrying it elsewhere is guesswork. | ||
| return formatErrorResponse( | ||
| 413, | ||
| "request_too_large", | ||
| "input_admission_refused", | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When this preflight rejects an oversized non-compaction request, AGENTS.md reference: src/AGENTS.md:L24-L26 Useful? React with 👍 / 👎. |
||
| `Estimated input (~${inputAdmission.estimatedTokens} tokens) is far past the context window ` | ||
| + `of ${parsed.modelId} (${inputAdmission.ceiling} tokens). Start a new session or choose a ` | ||
| + `model with a larger context window.`, | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Even if the response is changed to preserve
input_admission_refused, the real admission message still contains “context window,” soclassifyErrorinferscontext_length_exceededand the earlier stop branch returns before this new structured-code check runs. Consequently,comboFailureDecision(413, <real admission body>, { code: "input_admission_refused" })still returnsstop; check the authoritative structured code before the inferred context error and cover the real admission message in the regression test.AGENTS.md reference: src/AGENTS.md:L24-L26
Useful? React with 👍 / 👎.