fix(metering): alarm when a billable meter emit is swallowed - #484
Conversation
Both metering helpers caught every exception and logged a warning. That is
lost billable usage, and the loss is undetectable: the caller ignores the
helper's return value, and `_worker_loop`'s own `search.metering.job_failed`
capture can never fire because these handlers swallow first. Downstream, the
enterprise Sentry integration is wired at `event_level=logging.ERROR` with
Sentry Logs off, so a `logger.warning` produces no event at all -- searching
Sentry for one returns a false clean whether or not the drop is happening.
The asymmetry is what makes it dangerous. `_meter_applied_learnings` performs
a config lookup before emitting; `_meter_search_request` does not. A
config-store blip therefore drops `learning_applied` while `search_request`
keeps flowing, and the two meters diverge with no signal anywhere -- a shape
that is easily misread as a retrieval bug.
Adds `capture_anomaly("search.metering.emit_failed", level="error", ...)` to
both handlers, tagged with the meter so the two are distinguishable.
The existing failure test asserted the drop happened and commented that it was
silent; that comment is now false, so it asserts the alarm instead.
Mutation-verified: removing the applied-learnings capture fails with
"assert [] == ['search.metering.emit_failed']"; restored and confirmed by
SHA-256. 24 passed in that file; ruff clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Essentials Run ID: 📒 Files selected for processing (2)
Included review availability: 1 review is currently available. Your included PR review attempts over the past 7 days set your current allowance at 4 reviews per hour. 📝 WalkthroughWalkthroughMetering helpers now report emission failures as ChangesMetering anomaly reporting
Priority: ⬇️ Low Estimated code review effort: 3 (Moderate) | ~20 minutes Merge Risk: ⚪ Minimal · up to Metering emission failures now produce error-level anomaly events while preserving successful search responses and the existing best-effort metering behavior. No merge-blocking risk remains. Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
Problem
Both metering helpers in
routes/_metering.pycaught every exception and logged awarning. That is lost billable usage, and the loss is undetectable:
Falsegoes nowhere;_worker_loop's ownsearch.metering.job_failedcapture can never fire, becausethese handlers swallow the exception first;
event_level=logging.ERRORwith Sentry Logs off, so alogger.warningproducesno event at all. Searching Sentry for one returns a false clean whether or not
the drop is happening.
The asymmetry between the two helpers is what makes it dangerous.
_meter_applied_learningsperforms a config lookup before emitting;_meter_search_requestdoes not. A config-store blip therefore dropslearning_appliedwhilesearch_requestkeeps flowing, and the two meters divergewith no signal anywhere — a shape that is easily misread as a retrieval bug rather
than a metering one.
Change
capture_anomaly("search.metering.emit_failed", level="error", ...)in bothhandlers, tagged with the meter so the two are distinguishable. The
logger.warningstays; this adds the alarm rather than replacing the log.
Tests
test_metering_failure_does_not_break_search_responsealready drove this exactfailure path and asserted the drop happened, with a comment that it was silent.
That comment is now false, so the test asserts the alarm instead of the silence.
Mutation-verified rather than assumed: removing the applied-learnings capture fails
with
assert [] == ['search.metering.emit_failed']. File restored afterwards andconfirmed by SHA-256.
tests/server/api_endpoints/test_applied_learnings_metering.py— 24 passedruff check/ruff format --checkclean on both changed filesScope
Observability only — no metering behaviour changes, and a failed emit still returns
Falseand still leaves the search response a 200.Summary by CodeRabbit
Bug Fixes
Tests