fix(pricing): stop reporting an unpriced rate as free - #930
Merged
Conversation
Two ways a cost estimate turned "nobody declared this rate" into a confident number. OpenRouter quoted a completion rate of `-1` — the rate varies by upstream — or omitted it, and `?? 0` priced the output tokens at zero. The estimate then rendered without its `~`, missing the side of the bill most of the spend sits on. `output` now stays unset like `input` and `cached` already do, so the counter is reported unpriced. `resolveModelPricingFromTable` asked the `notTokenBilled` guard about the raw id while the substring walk ran against the prefix-stripped, lower-cased one. Gemini's image matchers are anchored, so `models/gemini-2.5-flash-image-preview` slipped past the guard and took `gemini-2.5-flash`'s per-1M-token card for a model billed per image. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T8DU24G9GWuRUJncc5TJFZ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two independent ways a cost estimate turned "nobody declared this rate" into a confident dollar figure. Same theme, so they travel together.
1. OpenRouter priced an unquoted completion rate at zero
OpenRouterQueuedProvider.modelPricingbuilt its card as:perMillion()deliberately returnsundefinedfor a non-numeric or negative quote — the JSDoc right above it says"-1""means the rate varies by upstream, which is unpriced rather than negative" — and the module's stated rule is that an unset rate means "not declared here", never zero. The?? 0undid that foroutputalone.Concrete failure: a router entry such as
openrouter/autoquotes{ prompt: "0.000003", completion: "-1" }(a partial card omittingcompletionbehaves the same).modelPricingreturned{ input: 3, output: 0 }, soestimateCostpriced a 10k-output-token call at exactly the input cost, setpriced = true, leftunpricedempty, andformatCostrendered a confident$0.0300with no~marker — silently dropping the side of the bill most of the spend sits on.Change: drop the
?? 0.outputnow behaves likecached: unset, soestimateCostlists it inunpricedand the total renders approximate. The card is still returned (theinputrate is real), which is the same tradecachedalready makes.2. The
notTokenBilledguard was asked about the wrong idresolveModelPricingFromTablestrips vendor prefixes and lower-cases the id intoid, runs the substring walk againstid— but called the guard with the rawmodelId.Gemini's matchers are
^-anchored ([/^imagen-/i, /^gemini-.*-image(?:-|$)/i]), so any id carrying one of the three prefixes the function itself declares walked straight past the guard. Reproduced against the shipped Gemini table before the fix:models/…is literally the shape Gemini's ListModels returns, which is whyGemini_ModelSearchstrips it. The result is the exact "fabricated unit" the guard exists to prevent: a per-image model resolvinggemini-2.5-flash's per-1M-token card, and the run reporting a confident wrong-unit figure. (xAI was unaffected only by luck — its matcher is unanchored.)Change: pass the normalized
id, and say so in the JSDoc, since providers are told to pass the same matcher their effort policy uses.Tests
packages/test/.../OpenRouterPricing.test.ts— quoted prompt withcompletion: "-1", and a card omittingcompletion, must leaveoutputunset.packages/ai/.../ModelPricing.test.ts— thenotTokenBilledblock previously only exercised the no-prefix path; a table-driven case now pairsvendorPrefixeswith an anchored matcher, including a mixed-case prefixed id.packages/test/.../ModelSearchPricing.test.ts— the prefixed Gemini forms against the real shipped table, plus the prefixed text sibling still resolving.Verified
bunx vitest run packages/test/src/test/ai-provider-api packages/ai/src examples/cli/src/test/lookupModelPricing.test.ts— 87 files, 985 tests, all pass.bunx turbo run build-types --filter=@workglow/ai --filter=@workglow/openrouter— clean (9 tasks, includes the dependency chain).bunx oxfmt --checkandbunx oxlintover the touched trees — clean.Not verified here: the full repo test suite and the full
bun run lint/bun run build. Only the slices above were run.🤖 Generated with Claude Code
https://claude.ai/code/session_01T8DU24G9GWuRUJncc5TJFZ
Generated by Claude Code