Repository navigation
fix(ingest): exact WAL backlog and org queue counters, no metering for refused batches - #2440
Merged
Merged
Maple Review Bot / Maple / review
succeeded
Nov 10, 2026 in 2m 34s
Confidence 10/10 · No issues found
🟢 Confidence 10/10 · safe to merge. Backlog and org-byte arithmetic verified by reading the lane sync/delivery ordering; the Rust test job has not finished at this head.
no findings · tests covered · risk high
Fixes five WAL/metering defects in the ingest gateway plus two encoder mismatches in the Go collector exporter, and drops the unused INGEST_TINYBIRD_CONCURRENCY_PER_SHARD. The counter and billing changes hold up; safe to merge.
while_caller_waitssubtracts a skipped frame's bytes fromcommitted_bytes, so backlog returns to zeroexport_and_markreleases org queue bytes beforemarked?, covering a failed cursor writebatch_was_storedskips metering and key use when a relay was refused and nothing was storedread_wal_framegrows the payload buffer instead of allocating the declared length up front
Before merge
- Infra · Check every environment has what this config now expects ·
apps/ingest/Dockerfile.prebuilt - Infra · Check every environment has what this config now expects ·
apps/ingest/alchemy.run.ts - Manual · Drop
INGEST_TINYBIRD_CONCURRENCY_PER_SHARDfrom the prd and dev deploy configs; nothing reads it at this head
What was checked
- Skip subtraction matches the add exactly:
end - startequals theaddedaccumulated intoSyncBatch.bytes(telemetry.rs:1754,:1854) mpsc::channel(0)check sits invalidate_for_pipeline, which both write-mode branches call at boot (main.rs:463-467)uses_native_pipeline_foris the same predicate that gates the native accept inprocess_decoded_payload(main.rs:7955), so the dual-mode predicate cannot over- or under-count
4434067 · Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.
Loading