Repository navigation
feat(models): free OpenRouter defaults per role with a paid floor fallback - #193
Conversation
…lback Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🟡 Reviewed4 actionable comment(s) · 1 suppressed (anti-noise) · grounded on your code 🤖 developerz.ai — automated review, running on your model and your box. |
There was a problem hiding this comment.
Review summary — 24 file(s), 4 finding(s).
Critical 0 · Major 0 · Minor 4 · Nit 0
Five behaviours changed in this PR have no test that would fail if the change were reverted: the streaming fallback, the transport-error fallback, the bare binding when a role already uses the fallback model, the boot-time image check on the fallback, and the docs guard for the fallback id.
| File | Findings |
|---|---|
src/agent/fallback.ts |
2 minor |
src/agent/models.ts |
1 minor |
src/boot.ts |
1 minor |
🤖 developerz.ai review — automated, running on your model and your box. What is this?
| status === undefined || status === 404 || status === 408 || status === 429 || status >= 500 | ||
| ); | ||
| } | ||
|
|
||
| /** The fallback-worthy reason for an error, or `null` when it must propagate. */ | ||
| export function fallbackReason(error: unknown): string | null { | ||
| if (!APICallError.isInstance(error)) return null; // abort, validation, our own bugs | ||
| if (!isUnavailable(error.statusCode)) return null; | ||
| return error.statusCode === undefined | ||
| ? `transport error: ${error.message}` | ||
| : `HTTP ${error.statusCode}`; |
There was a problem hiding this comment.
minor · test Every fake failure in fallback.test.ts carries a status code, so the status === undefined transport-error branch is never run. Deleting it would still pass the suite while dropped connections stop falling back. Add a failing() variant that throws APICallError with no statusCode, and assert the fallback answers with a transport error: … reason.
| wrapStream: async ({ doStream, params }) => { | ||
| try { | ||
| return await doStream(); | ||
| } catch (error) { | ||
| const reason = fallbackReason(error); | ||
| if (reason === null) throw error; | ||
| fellBack(reason); | ||
| return backup.doStream(params); | ||
| } | ||
| }, |
There was a problem hiding this comment.
minor · test No test calls streamText, so the wrapStream branch never runs.
If it rethrew on a 429, or called backup.doGenerate instead of backup.doStream, fallback.test.ts would still pass. Add a test where the primary's doStream throws a 429 APICallError and assert that the fallback's doStreamCalls has length 1 and one event is recorded.
| const bind = (role: keyof RoleModels, id: string): LanguageModel => | ||
| id === fallbackId | ||
| ? provider.chat(id) | ||
| : withFallback(provider.chat(id), provider.chat(fallbackId), (event) => | ||
| onFallback(role, event), | ||
| ); |
There was a problem hiding this comment.
minor · test Nothing tests the rule that a role whose id equals the fallback is bound bare.
If bind always wrapped, that role would send a failing call to the same model a second time, and no test would notice. In fallback.test.ts, call resolveModels with { driver: X, fallback: X } against the HTTP fake returning 429, and assert requested equals [X] and roles is empty.
| if (config.models.driver === config.models.vision) { | ||
| const { apiKey, baseURL } = resolveProviderConfig(providerOptions); | ||
| selfLook = (await supportsImageInput(baseURL, apiKey, config.models.driver)) === true; | ||
| const ids = [...new Set([config.models.driver, config.models.fallback])]; | ||
| const caps = await Promise.all(ids.map((id) => supportsImageInput(baseURL, apiKey, id))); | ||
| selfLook = caps.every((cap) => cap === true); |
There was a problem hiding this comment.
minor · test No changed test covers the new rule that self-look also needs the fallback to accept images.
Reverting to checking only config.models.driver would pass, and an image could then reach a text-only fallback. Through the boot seams, set driver equal to vision with a fallback that supportsImageInput reports false for, and assert selfLook ends up false.
## What Release 1.10.0: version bump in `package.json`, `server.json` and `src/index.ts`, and the changelog cut. ## Why Ships #193 (free OpenRouter defaults per role with a paid fallback) to npm, so developerz.ai can bump its `.mcp.json` pin, which is 1.9.0 today. 1.9.0 rejects the new `models.fallback` key. ## Verification The version-consistency tests run in CI. After merge, publishing a GitHub release `v1.10.0` triggers `release.yml`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
What
Free OpenRouter defaults for each role, with a paid fallback that can stand in for every role:
particle-ai/particle-1:freenvidia/nemotron-3.5-super-vl:freebaidu/ernie-5.5-flash-preview:freemodels.fallback, new)deepseek/deepseek-v4.1-flash:floor(text, image and tools)Why
ui-debuggerOpenRouter key is over its monthly cap ($18.04 of $10).Changes
src/agent/fallback.ts. A model wrapper re-sends the same call to the fallback on 404, 408, 429, 5xx, a transport error or an empty answer; a 400 still fails loudly. Each switch prints one line on stderr.models.fallback, reported bydescribe. The docs guard understands the:freeand:floorids.Verification
dummy/web.src/agent/fallback.test.ts: a fake 429 on the free id is answered by the paid id. It was red without the fallback.bin/check: 1736 pass, 63 skip, 0 fail. Two android gate tests fail only under Claude Code's shell variables; they fail the same way on an untouched clone..mcp.jsonpin, which is 1.9.0 today. 1.9.0 rejects thefallbackkey.🤖 Generated with Claude Code