Skip to content

Report batches whose processing transaction fails (onBatchFailed / okapi.batch.failed) #150

Open

Description

@rawo

Problem

Since #137 (2.0.0) OutboxProcessorListener events are published only after the processing transaction commits, and a batch whose commit fails emits nothing. That keeps the counters correct (okapi.entries.delivered / failed / retry.scheduled count only committed status changes), but now a failed batch is invisible to metrics: a commit that keeps failing (lost connection, serialization failure, timeout, or a delivery that throws inside the transaction) shows up only as log lines and a growing PENDING lag, and nothing tells the user why.

Proposal

  • Add a callback to OutboxProcessorListener, e.g. fun onBatchFailed(error: Throwable, duration: Duration) {} (default no-op), called by OutboxScheduler when runInTransaction { processBatch(...) } throws, which covers both a failing processBatch and a failing commit.
  • In MicrometerOutboxListener, record an okapi.batch.failed counter (optionally tagged with the exception class) and include failed batches in okapi.batch.duration with an outcome tag, or record them under a separate timer.
  • Document it next to the existing counters in the README, and say that entries of a failed batch are not counted individually, because their status changes were rolled back.

Compatibility

Additive: a new interface method with a default implementation and a new meter. Fits a 2.x minor release.

Split out of #137 (see the review discussion on #147).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions