Pricing table is a hardcoded 5-model dict: resolve rates from a per-run map or the control plane

[susheem-k]

TokenOps can price five models. Every other model halts the run before its first call, with an error that reads like a budget stop. This proposes replacing the built-in table with a rate map supplied per run or by the control plane.

---

1. Context

TokenOps puts a spend cap on an agent workflow and enforces it before every model call. The unit it governs is a run — one whole workflow, not one request. A research → summarise → review pipeline is a single run with a single budget, even when each step is a separate process.

If you are new to the repo, these are the pieces this issue assumes:

Term	Meaning
run	One whole workflow. Shares a run_id and one ledger.
tokenops_run()	Opens or joins a run, returns a bound handle.
wrap_complete()	Wraps the function that calls the model. The enforcement point.
Ledger	Records spend per run, holds the halt flag.
Policy	A (detect, fix) pair. cost_budget halts after the call that crosses the cap; pre_call_worst_case halts before a call that could cross it.
Governor	Runs the policies. Built per run from governance config.
Halt	Raised to abort a run. Extends BaseException so a framework's broad except Exception cannot swallow the breaker.
Micros	All money is integer micro-USD. $1.00 == 1_000_000. No floats in the cost path.
PriceFn	(provider, model, Usage) -> Micros. Converts token counts into money. Everything above depends on it.

How a price function reaches the policies

One is built per run and handed to the governor, which distributes it to every policy that needs money rather than tokens.

# src/tokenops/control/run.py:221 — inside tokenops_run()
price_fn = price if price is not None else build_price_book()
config   = client_obj.governance_config_for(svc)
gov      = build_governor(config, price_fn, controls, store=store_obj, enforce=enforce)

build_price_book() is the default, and price= is the only way to supply anything else. Note the ordering: the price function is built before the governance config is fetched, which matters for phase 2 below.

Assumes #131. That PR removes the local ledger: ControlPlaneClient.from_env() raises without a URL, TOKENOPS_EMBEDDED is gone, and every live run governs through HttpLedgerBackend. So a control plane is no longer optional infrastructure — every deployment has one, with control/dev_plane.py covering local work and tests passing store= explicitly. This changes which rung below is the realistic default, and is called out where it matters.

When the price function is called

Instrumenting a live run shows price() invoked five times for a single model call. It is on the hot path, and it runs both before and after the provider request.

#	Step	Prices?
1	governed() builds a CallRequest — estimated input tokens, max-output cap (control/integration.py)
2	governor.pre_call(request) — pre_call_worst_case prices projected input and projected output separately	×2 (policies/pre_call_worst_case.py:64)
3	ledger.admit(segment) — reserves the call; refuses immediately if the run is already flagged halted
4	dispatch(provider, model, messages) — the only step that touches the provider
5	Crossing hook → ledger observes actual Usage, bills the call	×1 (ledger.py:309)

Two consequences drive the design. Resolution must be a cheap in-memory lookup, because it runs several times per call. And a price failure surfaces at step 2 — before the provider is ever contacted — which is why the symptom is a run that spends nothing at all.

---

2. Problem

The entire price book is a module-level dict:

# src/tokenops/control/pricing.py:36
#: Approximate list prices (micros per 1M tokens). Swap for exact rates anytime.
DEFAULT_RATES: dict[str, Rate] = {
    "gpt-4o-mini":       Rate(input=  150_000, output=   600_000),
    "gpt-4o":            Rate(input=2_500_000, output=10_000_000),
    "claude-sonnet-4-6": Rate(input=3_000_000, output=15_000_000),
    "claude-haiku-4-5":  Rate(input=  800_000, output= 4_000_000),
    "claude-opus-4-8":   Rate(input=15_000_000, output=75_000_000),
}

Lookup is an exact-string dict.get, and a miss raises:

# src/tokenops/control/pricing.py:53
rate = table.get(model)
if rate is None:
    raise ValueError(
        f"no price for model {model!r} (provider {provider!r}); failing closed"
    )

Failing closed is deliberate and correct — control/core.py:18 states it as an invariant: "an unknown price or unhandled action blocks; it never silently allows." Pricing an unknown model at zero would mean a budget that never trips, which is worse.

The problem is that nothing outside Python can add a rate. TOKENOPS_CONFIG configures budgets and policies but not prices. So anyone using Gemini, Llama, Mistral, DeepSeek, Bedrock, a local model, or an Anthropic model addressed by its dated ID cannot use TokenOps at all without writing a PriceFn.

What it looks like when it happens

Observed while integrating TokenOps into mini-SWE-agent with gemini/gemini-3-flash-preview on SWE-bench Lite. Every run ended like this:

exit_status : TokenOpsHalt
cost        : $0.0000
api_calls   : 1
halt        : run already halted; refusing further calls

messages    : system, user          # no assistant message — the model was never called

That message is the crux of why this cost hours rather than minutes. It says the run was already halted, which is byte-for-byte what a legitimate exhausted budget looks like. Nothing in it mentions pricing. The only tell was arithmetic: a $0.02 budget cannot be exhausted before the first call.

The real cause is one frame further in:

# src/tokenops/control/policies/pre_call_worst_case.py:64
try:
    projected = self._price(request.provider, request.model, Usage(input=...)) \
              + self._price(request.provider, request.model, Usage(output=capped_out))
except Exception as exc:  # unknown price → fail closed (HALT), never assume 0
    return Signal(
        severity=Severity.TRIP,
        reason=f"unknown price for {request.provider}/{request.model}; failing closed ({exc})",
    )

The informative message exists, but it is emitted once, on the TRIP that sets the halt flag. Every subsequent call hits ledger.admit, sees the flag, and reports the generic run already halted — which is what reaches the caller and the logs. The cause is buried under its own consequence.

The same line has a second, smaller defect. It composes f"{request.provider}/{request.model}", so a caller passing provider="gemini" with a litellm-style model="gemini/gemini-3-flash" gets gemini/gemini/gemini-3-flash in the error. Cosmetic, but it sends readers hunting for a prefix bug that does not exist — price() receives the model string exactly as passed, verified by instrumenting a live run.

A second problem in the same table

Seeding current prices found two of the five existing rates to be wrong:

Model	In the table	Actual	Effect
claude-haiku-4-5	800k / 4M	1M / 5M	Under-bills 20%. A budget overruns before it trips — the failure this product exists to prevent.
claude-opus-4-8	15M / 75M	5M / 25M	Over-bills 3×. Runs halt well before their cap, and the spend report is wrong.

Both are corrected on chore/seed-default-rates (commit 2fa671e), which also expands the table to 27 current Anthropic, OpenAI and Gemini models. That branch is self-contained and can merge independently of everything proposed below.

---

3. Proposed approach

DEFAULT_RATES answers two questions with one object: what does a token cost, and how does every agent process come to agree on that. Fusing them is why there is no configuration path — you cannot change the number without changing the distribution mechanism, because they are the same object.

Separating them gives a two-rung chain. The first hit wins; exhausting the chain fails closed exactly as today.

Rung	Source	Notes
1	price= on tokenops_run	The embedder supplies a rate map for this run. Wins over everything, so one workflow can override the fleet default. (phase 1)
2	Control-plane rates	The same JSON, served per agent through the existing governance config endpoint. The normal path once #131 lands, since every run already talks to a plane. (phase 2)
3	Built-in DEFAULT_RATES	Retained so that with nothing configured, behaviour is exactly what it is today.
✕	Fail fast	Raise, naming the model and which sources were consulted.

Why a map rather than a function

A single run may call several models — a cheap model to plan, an expensive one to write, a small one to summarise. A PriceFn bound to one model cannot express that, so the shape is a map keyed by model identifier.

{
  "rates": {
    "gemini/gemini-3-flash-preview": {
      "unit":        "per_token",
      "cost_input":  "0.0000005",
      "cost_output": "0.000003",
      "cost_cached": "0.00000005"
    },
    "anthropic/claude-sonnet-4-5-20250929": {
      "unit":        "per_million",   // same file, different unit
      "cost_input":  "3.00",
      "cost_output": "15.00",
      "cost_cached": "0.30"
      // cost_reasoning omitted -> defaults to cost_output
    }
  }
}

Keys are whatever string the caller passes as model. Lookup stays exact, so a full provider-qualified name works as a key with no special handling.

unit is per entry and required. Providers publish in different units — Anthropic and OpenAI quote per million, litellm's table is per token — so a single file-wide unit would force whoever writes it to convert by hand. Per entry moves that conversion into the loader, where it is tested once. Required rather than defaulted, because the two readings differ by a factor of a million and a map that omits it is better rejected than guessed at.

Costs are strings. 0.0000005 parsed as a float carries binary representation error into money, against the repo's own no-float-drift invariant. Parse as decimal, multiply by the token count, convert to micros once, and round up — a spend cap should only ever err toward stopping slightly early.

cost_reasoning is optional and defaults to cost_output, matching how Rate.reasoning already behaves. In practice it is inert today: ModelResponse carries only content, input_tokens and output_tokens, so Usage.reasoning is always zero through wrap_complete. The hazard to record for whenever that changes is overlap, not rate — providers report reasoning tokens as already included in the output total, so billing both categories double-charges.

Merging is per key, not whole-map. A run supplying a rate for one experimental model must not lose the plane's rates for the other models it also calls.

Worked example — one measured mini-SWE-agent call

Category	Tokens	Per token	Micros
Input	1,500	0.0000005	750
Output	300	0.000003	900
Cached	0	0.00000005	0
Call total			1,650

Worth building the test fixture from a real trajectory rather than round numbers, so it catches a units mistake instead of confirming one.

Phases

Phase 1 — rate map on the run. Rungs 1 and 3, and the whole JSON contract. Unblocks every provider immediately; no schema, no migration, no server change.

• price= accepts a mapping as well as a callable
• Decimal parse, per-entry unit, micros conversion, round up
• cost_reasoning defaults to cost_output
• Error names the model and the sources consulted
• Fix the provider/model message composition

Phase 2 — plane-served rates. Rung 2. Same JSON, distributed per agent instead of per caller.

• Rates in the governance config payload
• Store table and _assemble_governance_config
• Cache invalidation on rate writes
• Reorder run.py:221 — fetch config before building the price fn
• Per-key merge with rung 1

Both phases are needed; only the order is a choice. An earlier draft of this issue argued phase 1 alone would close it "for single-process users". After #131 there is no such user — every run is plane-backed, so a rate supplied only through price= has to be passed by every caller in every process, which is exactly the drift a control plane exists to remove. Phase 1 remains worth doing first because it is small, additive, and defines the JSON contract phase 2 then distributes; it is the fastest unblock and the per-run override, not the destination.

The natural home for rung 2 is the governance: block, because load_governance_yaml returns only that block and it is already served per agent over GET /v1/governance/{agent} with an in-process cache. Nesting there inherits distribution for free; a sibling block would need new loader, store and endpoint work.

---

4. Tradeoffs and non-scope

Tradeoffs accepted

• Exact matching only. No prefix stripping, no date-suffix inference, no alias table. The cost is that a caller must list every model string they might use. The benefit is that the design cannot produce a wrong price. Fuzzy matching fails silently and permanently: an under-priced model means the ledger under-bills, the budget never trips, and TokenOps reports a healthy run while spend goes uncapped. A loud failure is strictly better.
• Two rungs, not four. An earlier draft added a TOKENOPS_PRICES local file. Dropped: a per-host file is the rung most likely to drift between machines, which is the problem a control plane exists to solve. refactor: rename RunState to LocalRunState; add PolicyInstance.data_scope #131 settles it — with the local ledger gone, a local rate file would be the only remaining piece of per-host governance state.
• DEFAULT_RATES stays as a silent third rung. Removing it would turn an additive feature into a breaking change for anyone pricing gpt-4o today with no configuration.

Explicitly out of scope

• Failure-mode postures → Failure-mode groups and per-group behaviour flags (plane unavailable, pricing unavailable, policy error) #129. Fail-fast behaviour is unchanged here. Whether an unpriced model should be able to fail open, and how that differs from a control plane being unreachable or a policy raising, is a separate design with its own taxonomy.
• Gateways that select a model at runtime → Gateways that pick a model at request time: the model is unknown before the call and unreported after it #133. This design assumes the model identifier is known before the call. For stacks that route through a gateway choosing a model at request time, it is not, and pre_call_worst_case cannot price a worst case for a model nobody has picked yet. That is a different mechanism, not a rate-table change.
• Refreshing rates over time. A hand-edited table drifts silently and the spend numbers still look plausible — which is exactly how the two wrong entries above survived. A scheduled job that diffs the table against a maintained source and opens a PR would help, but it is separable from the resolution mechanism.

---

5. Acceptance

Check	Phase
With nothing configured, gpt-4o prices as it does today and an unknown model raises	1
A rate can be supplied without writing Python	1
Two models used inside one run price independently and correctly	1
Two entries in one map with different unit values both price correctly	1
A missing or unsupported unit is rejected, not guessed	1
The full string gemini/gemini-3-flash-preview resolves as a key	1
No float reaches the ledger; rounding is up, proven by a fixture from a real run	1
An omitted cost_reasoning bills at cost_output, not zero	1
The failure names the model and the sources consulted, not run already halted	1
Rung 1 overrides rung 2 per key, without dropping other keys	2
Two processes against one plane price identically	2
Editing a rate invalidates the governance cache	2

---

Related: #129 (failure-mode postures), #133 (gateway model selection), #57 (output-token prediction, same price path), #114 and #118 (governance config distribution), #131 (removes the local ledger — see the phasing note).

Code references and the live probes — exact-match lookup, five price() calls per model call, Usage.reasoning never populated — verified against main at v0.2.1.

[susheem-k]

Related: #137 — Usage.cached is never populated, so cached tokens bill at the full input rate (measured 3× over-billing on a real agent run). Relevant here because the `cost_cached` field proposed in this issue can be configured but never applied until that lands.