Skip to content

feat(models): free OpenRouter defaults per role with a paid floor fallback - #193

Merged
sebyx07 merged 1 commit into
mainfrom
feat/free-openrouter-models
Nov 15, 2026
Merged

sebyx07 merged 1 commit into
mainfrom
feat/free-openrouter-models

Conversation

@sebyx07

@sebyx07 sebyx07 commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

What

Free OpenRouter defaults for each role, with a paid fallback that can stand in for every role:

Role Model
driver particle-ai/particle-1:free
vision nvidia/nemotron-3.5-super-vl:free
summary baidu/ernie-5.5-flash-preview:free
fallback (models.fallback, new) deepseek/deepseek-v4.1-flash:floor (text, image and tools)

Why

  • Cost. The owner ruled on 2026-11-14: free models, logging is fine, cheap.
  • The key is capped. The ui-debugger OpenRouter key is over its monthly cap ($18.04 of $10).

Changes

  • src/agent/fallback.ts. A model wrapper re-sends the same call to the fallback on 404, 408, 429, 5xx, a transport error or an empty answer; a 400 still fails loudly. Each switch prints one line on stderr.
  • Config. The new optional models.fallback, reported by describe. The docs guard understands the :free and :floor ids.

Verification

  • Live: the real MCP server against dummy/web.
    • The old paid defaults (deepseek-v4-flash + qwen3-vl) found the three 404s, the JSON error and the ReferenceError, but no visual issues.
    • The free defaults found everything the old pair found, plus both dead Add to cart buttons, the invisible subtitle and the low-contrast prices.
    • The final run took 96 s, with two logged 429 fallbacks.
    • A forced run on throttled gemma had 9 fallbacks and still produced full findings.
  • src/agent/fallback.test.ts: a fake 429 on the free id is answered by the paid id. It was red without the fallback.
  • bin/check: 1736 pass, 63 skip, 0 fail. Two android gate tests fail only under Claude Code's shell variables; they fail the same way on an untouched clone.
  • To take effect: a release, then bump developerz.ai's .mcp.json pin, which is 1.9.0 today. 1.9.0 rejects the fallback key.

🤖 Generated with Claude Code

…lback

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@developerz-ai

developerz-ai Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

🟡 Reviewed

4 actionable comment(s) · 1 suppressed (anti-noise) · grounded on your code

🤖 developerz.ai — automated review, running on your model and your box.

@developerz-ai developerz-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary — 24 file(s), 4 finding(s).

Critical 0 · Major 0 · Minor 4 · Nit 0

Five behaviours changed in this PR have no test that would fail if the change were reverted: the streaming fallback, the transport-error fallback, the bare binding when a role already uses the fallback model, the boot-time image check on the fallback, and the docs guard for the fallback id.

File Findings
src/agent/fallback.ts 2 minor
src/agent/models.ts 1 minor
src/boot.ts 1 minor

🤖 developerz.ai review — automated, running on your model and your box. What is this?

Comment thread src/agent/fallback.ts
Comment on lines +40 to +50
status === undefined || status === 404 || status === 408 || status === 429 || status >= 500
);
}

/** The fallback-worthy reason for an error, or `null` when it must propagate. */
export function fallbackReason(error: unknown): string | null {
if (!APICallError.isInstance(error)) return null; // abort, validation, our own bugs
if (!isUnavailable(error.statusCode)) return null;
return error.statusCode === undefined
? `transport error: ${error.message}`
: `HTTP ${error.statusCode}`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor · test Every fake failure in fallback.test.ts carries a status code, so the status === undefined transport-error branch is never run. Deleting it would still pass the suite while dropped connections stop falling back. Add a failing() variant that throws APICallError with no statusCode, and assert the fallback answers with a transport error: … reason.

Comment thread src/agent/fallback.ts
Comment on lines +90 to +99
wrapStream: async ({ doStream, params }) => {
try {
return await doStream();
} catch (error) {
const reason = fallbackReason(error);
if (reason === null) throw error;
fellBack(reason);
return backup.doStream(params);
}
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor · test No test calls streamText, so the wrapStream branch never runs.

If it rethrew on a 429, or called backup.doGenerate instead of backup.doStream, fallback.test.ts would still pass. Add a test where the primary's doStream throws a 429 APICallError and assert that the fallback's doStreamCalls has length 1 and one event is recorded.

Comment thread src/agent/models.ts
Comment on lines +41 to +46
const bind = (role: keyof RoleModels, id: string): LanguageModel =>
id === fallbackId
? provider.chat(id)
: withFallback(provider.chat(id), provider.chat(fallbackId), (event) =>
onFallback(role, event),
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor · test Nothing tests the rule that a role whose id equals the fallback is bound bare.

If bind always wrapped, that role would send a failing call to the same model a second time, and no test would notice. In fallback.test.ts, call resolveModels with { driver: X, fallback: X } against the HTTP fake returning 429, and assert requested equals [X] and roles is empty.

Comment thread src/boot.ts
Comment on lines 124 to +128
if (config.models.driver === config.models.vision) {
const { apiKey, baseURL } = resolveProviderConfig(providerOptions);
selfLook = (await supportsImageInput(baseURL, apiKey, config.models.driver)) === true;
const ids = [...new Set([config.models.driver, config.models.fallback])];
const caps = await Promise.all(ids.map((id) => supportsImageInput(baseURL, apiKey, id)));
selfLook = caps.every((cap) => cap === true);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor · test No changed test covers the new rule that self-look also needs the fallback to accept images.

Reverting to checking only config.models.driver would pass, and an image could then reach a text-only fallback. Through the boot seams, set driver equal to vision with a fallback that supportsImageInput reports false for, and assert selfLook ends up false.

@sebyx07
sebyx07 merged commit cb189e2 into main Nov 15, 2026
4 checks passed
@sebyx07
sebyx07 deleted the feat/free-openrouter-models branch November 15, 2026 00:27
@sebyx07 sebyx07 mentioned this pull request Nov 15, 2026
sebyx07 added a commit that referenced this pull request Nov 15, 2026
## What

Release 1.10.0: version bump in `package.json`, `server.json` and
`src/index.ts`, and the changelog cut.

## Why

Ships #193 (free OpenRouter defaults per role with a paid fallback) to
npm, so developerz.ai can bump its `.mcp.json` pin, which is 1.9.0
today. 1.9.0 rejects the new `models.fallback` key.

## Verification

The version-consistency tests run in CI. After merge, publishing a
GitHub release `v1.10.0` triggers `release.yml`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant