model-picker

Stop overpaying for AI models. Find the cheapest model that's smart enough for the job, and the cheapest provider to run it on, from live Artificial Analysis benchmarks pulled through Firecrawl Alexandria.

$ model-picker pick --min-score 45 --vs claude-fable-5-1

score  $/task  speed     model                         vs Claude Fable 5.1
46.3   $0.13   48 tok/s  MiMo-V2.6-Pro (Xiaomi)         87% of the score, 57x cheaper
47.8   $0.21   55 tok/s  GPT-6.1 Sol (Medium) (OpenAI)  90% of the score, 36x cheaper
50.2   $0.32   57 tok/s  GPT-6.1 Sol (High) (OpenAI)    94% of the score, 24x cheaper
...

Every row is a model nothing else beats on both score and cost. If a model isn't listed, something cheaper is at least as smart.

Install

On macOS or Linux:

curl -fsSL https://raw.githubusercontent.com/firecrawl/model-picker/main/install.sh | sh

This downloads the build for your machine from the latest release, checks it against the release's SHA256SUMS, and puts model-picker (and its short name mp) in ~/.local/bin. It needs nothing else: no Bun, no Node, and Pi comes with it. Run the same line again to update, or end it with | MODEL_PICKER_VERSION=v0.1.0 sh to install a particular release.

At the end it offers to run mp setup, which asks for your API keys, checks them with Firecrawl and OpenRouter, and saves them to ~/.config/model-picker/.env (readable only by you). Run mp setup again any time to change a key. You can also edit that file or export the keys in your shell, which wins over the file:

Key	For
FIRECRAWL_API_KEY	every command; free at firecrawl.dev
OPENROUTER_API_KEY	chat and build; model usage is billed to it
SUBQ_API_KEY	optional: SubQ Snap, a decision model, labels the work instead of a GPT model

mp pick --min-score 45
mp chat                                           # in the project folder you want to work in

model-picker is a command-line tool, a Claude Code skill that drives it, and an experimental chat router for Pi. Both ways of running models use Pi: build runs each step as its own Pi session, and chat runs inside one Pi session, switching models as you go. build can use OpenCode instead with --harness opencode, which must be on your PATH. mp chat --help lists the chat options; see Pi router for how routing works.

To check where a release came from: gh attestation verify model-picker-darwin-arm64.tar.gz --repo firecrawl/model-picker. To uninstall: rm -rf ~/.local/share/model-picker ~/.local/bin/model-picker ~/.local/bin/mp ~/.config/model-picker.

Claude Code

Inside Claude Code:

/plugin marketplace add firecrawl/model-picker
/plugin install model-picker@model-picker

The skill uses the model-picker command, so install it as above first. Then ask things like "which model should I use for my support bot?", "find me a cheaper provider for GLM-5.3" or "build this with the cheapest models".

From source

Needs Bun 1.4+, and Node 22.19+ for chat and build (Pi runs on Node from source):

git clone https://github.com/firecrawl/model-picker
cd model-picker
bun install
bun link                                          # puts `model-picker` and `mp` on your PATH

From source, a .env in the model-picker folder works as well as ~/.config/model-picker/.env.

How it picks

It picks the cheapest model that clears a score floor, with one exception: a model costing at most 10% more wins if it scores higher, since a few percent of price isn't worth several points of intelligence. There's nothing to switch on to make it prefer cheap models. A model at a given reasoning effort counts as its own row (e.g. gpt-6-1-sol-low and gpt-6-1-sol-high), so picking a row also picks the effort level. To steer it:

• --min-score raises or lowers the floor for pick (build uses hard 50, medium 42, easy 30).
• --max-cost caps the benchmark cost per task.
• --open limits it to open-weights models.
• --claude-code limits it to Claude models and effort levels.

--max-cost, --open and --claude-code also apply to build.

Commands

Command	What it does
pick [--min-score N] [--max-cost USD] [--open] [--claude-code] [--vs slug]	Cheapest models at or above a score, optionally compared with the one you use now. --claude-code limits it to the Claude models and effort levels Claude Code runs natively
providers <slug> [--sort cost|speed]	Every provider serving one model, by cost per task or speed
search <name>	Find a model's slug. Also lists models Artificial Analysis hasn't measured a cost for yet
build "<task>" [--dir ./out] [--report dir] [--harness pi|opencode] [--planner-min-score N]	Build something with the cheapest good-enough model per step, through Pi (or OpenCode) and OpenRouter, and write a report of every step's model, time, tokens and cost

Add --json to any command for machine-readable output.

Score is the Artificial Analysis Intelligence Index (0-100; today's best models score about 55-58). $/task is what it costs to run one Intelligence Index task, which accounts for how many tokens a model actually burns, not just its price per token.

Building with the cheapest model per step (experimental)

build needs an OPENROUTER_API_KEY. Each step runs in Pi by default, or in OpenCode with --harness opencode; the report says which.

1. The cheapest model scoring 50+ writes plan.json: as few steps as the job needs (1 for a focused edit, up to 4 for a large build). Each step lists the files it owns and the steps it depends on.
2. Each step is labelled hard, medium or easy and given a kind of work (coding, 3d, general and so on): by SubQ Snap if SUBQ_API_KEY is set, otherwise by the planner, which then labels difficulty only.
3. Each step runs as its own Pi (or OpenCode) session on the model clearing that difficulty's floor (hard 50, medium 42, easy 30) that's best at its kind of work, among models within 10% of the cheapest. Steps whose dependencies are done and whose files don't overlap run at the same time.
4. Only the last step runs the app or checks screenshots; earlier steps just write code.
5. REPORT.md and report.json record the harness and each step's label (and Snap's confidence), model, time, tokens (input, output, reasoning, cache) and cost, from Pi's session usage or OpenCode's session export, plus the build's wall-clock time and each step's session ID.

Both harnesses run tool calls without asking (Pi always does in print mode; OpenCode runs with --auto), so only build in folders you're happy for it to change.

Pi refreshes its model catalog only when asked, so model-picker runs pi update --models at most once a day before build and chat; otherwise models released since the last refresh would be missing.

How build decides which model does what

Any prompt ──► Planner: the cheapest model scoring 50+ (today GPT-6.1 Sol High)
                  │ writes plan.json: a to-do list
                  ▼
               SubQ Snap: labels each item hard / medium / easy and a kind of work
                  │ (without a SubQ key, the planner labels difficulty instead)
                  ▼
               Workers: one per item, picked by a lookup (no AI)
                  hard   (50+) ──► e.g. GPT-6.1 Sol (High)
                  medium (42+) ──► e.g. MiMo-V2.6-Pro or GPT-6.1 Sol (Low)
                  easy   (30+) ──► e.g. GPT-6 Luna (High)

• AI labels the work, a lookup picks the model. The planner splits the job; Snap (or the planner) labels each part. model-picker then takes the leaderboard models clearing that label's score floor that OpenRouter can run, and of those priced within 10% of the cheapest, the one best at that kind of work. No model picks another model.
• The planner is the same for every prompt. By default it's picked by the same rule as a hard step (50+), because a bad split hurts every step after it. --planner-min-score changes its floor without touching the steps' floors, e.g. --planner-min-score 42 to plan with a cheaper model now that Snap does the labelling. Planning cost $0.07-0.17 per build in our tests.
• Models only change when the leaderboard does. The Artificial Analysis leaderboard is fetched at most once a day. Within a build everything is fixed; a new cheaper model scoring 50+ becomes the planner from the next day's fetch. --claude-code, --open and --max-cost filter the candidates (with --claude-code the planner is Claude Opus 5.5 Medium).

Labelling with SubQ Snap (optional)

SubQ Snap (subq-snap-preview) is a decision model: it answers multiple-choice questions rather than writing text, at $0.05 per million input tokens with free output. With SUBQ_API_KEY set, it answers two questions for every build step and chat message: how hard is it, and what kind of work is it? It also gives its confidence; under 60%, the work goes one tier up, so an unsure "medium" runs as hard. The threshold isn't tuned on labelled data yet.

This keeps a GPT model from labelling the work that GPT models then compete for, reads the whole message rather than its first 8,000 characters, and costs a fraction of a cent per build.

Pi router (experimental)

pi/router.ts is a Pi extension that adds a model called model-picker/auto. For each message you send:

1. SubQ Snap (with SUBQ_API_KEY), or else the cheapest easy-tier model, labels it with a difficulty (hard, medium, easy) and a kind of work (coding, tools, vision, long-context, 3d, general).
2. Every benchmarked model on OpenRouter is a candidate. Models that can't take the message are dropped: no image input when you attached an image, or a context window too small for the conversation.
3. Of the rest clearing the difficulty floor (50 / 42 / 30), it takes the one best at that kind of work, ranked on that work's own benchmark (Terminal-Bench and SciCode for coding, τ²-Bench for tools, MMMU-Pro for images, LCR for long context, MineBench for 3D), among models priced within valueMargin of the cheapest. The default is 1.1, i.e. 10%; raise it to pay more for specialists.

It all happens in one session. Tool calls and retries within a turn stay on that turn's model, so the prompt cache survives. Computer use isn't ranked: the benchmark data has no computer-use score.

MineBench isn't on Alexandria, so its leaderboard is scraped with Firecrawl and cached for a day like everything else. It rates a model without its effort level, so every effort level of a model shares one rating, and models it hasn't tested (OpenAI's non-Pro GPT-6 models, for example) rank last for 3D. If the page can't be scraped, the router still works and 3D messages go to the cheapest qualifying model.

model-picker chat            # in any project folder; extra options go to Pi

For one session you can narrow or tune the models with flags, on top of the settings file below:

mp chat --region china          # or us / europe; makers with an unclear home country are left out
mp chat --creators Anthropic,OpenAI
mp chat --open                  # open-weights models only
mp chat --value-margin 2        # pay up to 2x the cheapest for a better-suited model

If a filter leaves no model above a difficulty's floor (no Chinese model scores 50+, for instance), that difficulty goes to the strongest allowed model, marked "best available" in the header and footer.

chat opens Pi's terminal UI with the router loaded and selected. It needs OPENROUTER_API_KEY and FIRECRAWL_API_KEY, from your environment or ~/.config/model-picker/.env.

The footer shows why each message went where it did (e.g. hard · coding → GPT-6.1 Sol high) and a running savings line, e.g. $0.42 spent · ~$3.10 on Claude Opus 5.5 · saved 86%. The comparison prices this session's actual tokens at the baseline model's rates, so it's an estimate: the baseline would have written different responses. /session shows cost per model. To limit the candidates, write ~/.config/model-picker/router.json (or point MODEL_PICKER_ROUTER_SETTINGS at a file):

{ "region": "us", "creators": ["OpenAI", "Anthropic"], "openWeightsOnly": false, "maxCost": 1, "valueMargin": 2, "baseline": "anthropic/claude-opus-5.5" }

Pi's virtual-model API is experimental, so Pi is pinned (1.2.0).

Cost

A pick uses 20 Firecrawl credits and a providers lookup 5. Results are cached for 24 hours, so repeat runs are free.

Contributing

The repo includes agent-harness checks: if you work on it with Claude Code, .claude/settings.json runs formatting, lint and typecheck after each edit and the full checks (.harness/hooks/check-all.sh) before the agent stops. Without Claude Code, run that script yourself before opening a pull request, plus bun test.

To release, bump version in package.json, then push a matching tag (git tag v0.2.0 && git push origin v0.2.0). The Release workflow builds the four platforms with scripts/package.sh, which you can also run locally (e.g. scripts/package.sh darwin-arm64), and publishes them with SHA256SUMS and build attestations. Pi's version comes from the @earendil-works/pi-coding-agent pin in package.json.

License

MIT