Hold a 402, report a discarded api_url, and sweep gateway drift

[anassg-lago]

Follow-up to @ancorcruz's review on the JS side (#33, #34). His findings were against the JS port, but four of the five are in this repo too and its stack is already merged, so they need their own PR here rather than an edit to an open branch. Every finding was verified live against real infrastructure before writing any code, and re-verified after.

The JS half is on fix/gateway-and-gemini-attribution and will follow.

The one that loses money

A 402 from anything in front of Lago silently discarded every billable event for the whole outage. 402 was in _PERMANENT_STATUSES, which routes a batch to _send_individually; every isolated send then 402s too and is logged and dropped for good.

Measured against a server returning 402 — 5 events in, 6 HTTP calls out, 0 recoverable, one on_error for the lot:

--- before ---                         --- after ---
request shapes : [a,b,c,d,e]           request shapes : [a,b,c,d,e]
                 [a] [b] [c] [d] [e]                    [a,b,c,d,e]
                                                        [a,b,c,d,e]
peak buffered  : 0                     peak buffered  : 5
outcome        : all 5 lost            outcome        : all 5 delivered

The test for the permanent set is "is what makes this fail a property of the batch?" — true for 413 (size) and 415 (media type), constant across retries; false for 402, where payment required is a property of the account and stops being true the moment someone pays. That is the same shape as the 429 which was deliberately carved out.

Ancor argued this from the semantics of the status. Probing a real Lago gives a harder argument: Lago cannot emit 402 at all.

probe against a real instance	status
valid batch	200
duplicate transaction_id (replay)	422 value_already_exist
duplicate inside one batch	422
bad api key	401
~20k-event batch (4.2 MB)	422 too_many_events
Content-Type: text/plain	422
malformed JSON	400

Its surface is 400/401/403/404/405/422/429. So the only possible sender of a 402 is an intermediary, and every plausible intermediary emits it for account reasons that resolve out-of-band. Two side effects worth recording: an oversized batch answers 422 too_many_events, which already routes to the split path (the batch-splitting capability 413 was added for was already reachable), and a duplicate transaction_id answers 422, not the 409 the comment claimed as its own exemplar. 413/415 stay for an intermediary — nginx's client_max_body_size really does answer 413 before the request reaches Rails — and the comments now say which reader they are for.

The one the review found by accident, which had already happened

usage_metadata from the Cloudflare gateway got no drift sweep. extras was a fixed three-key dict, so any counter the adapter does not map vanished with no error and no on_error — the one place violating the contract test_drift.py enforces for the native adapters.

A live Logs API pull over 50 real entries:

input_tokens                 50    -> (top-level tokens_in)
output_tokens                50    -> (top-level tokens_out)
total_tokens                 50    -> (derivable)
input_cached_tokens          50    -> cache_read
input_cache_creation_tokens  25    -> cache_write
neurons                      25    -> DROPPED
units                        25    -> DROPPED

neurons is Cloudflare's Workers AI billing unit and units is a cost quantity — and units appears in none of the 14 captured fixtures, so the hand-maintained enumeration in the module docstring had already drifted past reality. That is the money-relevant drift the review only hypothesised, present today. (Ancor named input_text_tokens; neurons was unmapped too and matters more.)

Unmapped keys now sweep into extras["usage_metadata"] against an explicit _MAPPED_USAGE_KEYS set, so a ninth spelling surfaces on its own instead of needing another 14-fixture audit. The docstring now says the enumeration is illustrative, not exhaustive.

Closes #16, which tracked exactly this. One deviation from that issue's suggestion: it proposed a dotted prefix (extras["usage_metadata.neurons"]), and this nests the whole dict instead (extras["usage_metadata"]["neurons"]). Nesting matches the convention already pinned by test_drift.py::test_invoke_openai_compat_prompt_tokens_details_lands_in_extras, which asserts extras["prompt_tokens_details"] == {...} — the nested dict under its own key. It also removes a shadowing hazard: the poller reads extras["cached"] to decide whether to skip billing a request Cloudflare served for free, so a future usage_metadata key called cached or step must not be able to overwrite it.

Also in here

• An explicitly-passed falsy api_url silently resolved to PRODUCTION. Preferring the config value over "" is right — requests raises MissingSchema, not a LagoApiError, so the queue retried forever and all billing stopped. But LagoConfig's default is the production URL, so api_url=os.environ.get("LAGO_API_URL", "") with the var unset resolved to production with no on_error and no log (verified live). For a CI job or a developer holding a real production key that writes live billing data, and ingested events cannot be un-ingested. The fallback is unchanged; it is now reported under config.api_url.
• prime() re-downloaded the ~400-model OpenRouter catalogue regardless of the TTL. It set _openrouter_stale unconditionally, and it is called from _auto_prime_pricing_for on a matching wrap() and from warm_pricing() — both of which a server can run per request — so pricing_ttl_seconds never applied there at all. Against the live catalogue with the shipped 1-hour TTL: 4 cycles → 4 full downloads where 1 was correct; now 1.
• A failed pricing fetch retried every tick, forever, with no backoff, ahead of the drain. Only the success path cleared a source's stale flag. With a bad Cloudflare token: 5 ticks → 5 real requests and 5 on_error reports; now 1, and it still recovers once the window expires (checked explicitly — this is a backoff, not a permanent give-up). Per-source 1→2→4→…→60s, matching the queue's own send backoff, so one bad credential cannot delay the three healthy tables.

A deliberate divergence from JS

The four pricing fetches stay sequential here. This refresh runs on a blocking daemon thread, where a thread pool for four fetches is a larger change than the problem warrants; the per-source backoff already removes the harm. JS additionally parallelises them under Promise.allSettled, which its event loop makes free. Recorded in the changelog as language-inherent, like os.register_at_fork against AsyncLocalStorage.

Verification

Every fix has a test that fails when the fix is reverted. Green at this commit: ruff, ruff format, mypy strict, 552 unit tests (was 545).

Each finding was driven against live infrastructure before and after — a real local Lago, the real OpenRouter catalogue, the real Cloudflare model and Logs APIs — rather than only against mocks:

	before	after
402, batch of 5	6 calls, 0 recoverable	3 calls, all 5 delivered
api_url falsy	production, 0 reports	production, reported
prime() TTL	4 cycles → 4 downloads	→ 1
failed source	5 ticks → 5 requests	→ 1, recovers after the window
gateway drift	neurons + units dropped	both preserved, 0 dropped