Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 36 additions & 11 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,44 @@ Until `1.0.0`, breaking changes may appear in any release and are flagged with *
not send the `OutboxId`; that is no longer the case, and the "Deduplicating on `x-outbox-id`"
section documents the recipe and both caveats. (KOJAK-78)

### Fixed

- **MySQL: the processor no longer blocks `publish()` for the duration of delivery.** Under MySQL's
default `REPEATABLE READ`, the claim's `FOR UPDATE SKIP LOCKED` took gap locks on the claim index;
with a backlog smaller than the batch, the gap ran to the end of the claimed type's range, exactly
where every `publish()` of that type inserts, so business transactions waited for the whole
HTTP/Kafka delivery and could fail with `Lock wait timeout exceeded`. On MySQL and MariaDB the scheduler
transaction now runs at `READ COMMITTED`, which takes no gap locks:
- Spring Boot: the autoconfigured `TransactionRunner` pins `READ COMMITTED` when the store is
`MysqlOutboxStore` and its `TransactionTemplate` leaves isolation at `DEFAULT`. It uses a copy,
so a user's or Boot's template bean is not mutated and its timeout/propagation/read-only
settings carry over; a template with an explicit isolation level is still used as-is. Pinning is
skipped, with a startup WARN, for PTMs that reject custom isolation levels — a
`JtaTransactionManager`, or a `JpaTransactionManager` without the Hibernate/EclipseLink
`JpaDialect` — since every tick would otherwise throw `InvalidIsolationLevelException`.
- `ExposedTransactionRunner(database)` uses `READ COMMITTED` when the database's dialect is MySQL or
MariaDB. A new `ExposedTransactionRunner(database, transactionIsolation)` constructor sets the
level explicitly (`null` = the database default).

Other stores and databases are unchanged: Postgres already defaults to `READ COMMITTED`. MySQL at
`READ COMMITTED` needs `binlog_format=ROW` or `MIXED` (see the README). (#129)
- **A worker no longer marks rows it cannot deliver as `FAILED`.** Previously every worker claimed
all `PENDING` rows, so a worker without a deliverer for a row's type turned it into a permanent
failure. Each worker now claims only its registered delivery types, in at most two queries per
batch. Workers on an older version still claim every type, so publish a newly introduced type only
after all old processors are upgraded or stopped. Rows already marked `FAILED` are not retried
automatically. Cross-type claim order is no longer globally oldest first: the processor rotates
the starting type, and any leftover batch capacity goes to the other types alphabetically.
([#116](https://github.com/softwaremill/okapi/issues/116))

### Changed

- **`ExposedTransactionRunner` on MySQL/MariaDB now overrides `DatabaseConfig.defaultIsolationLevel`.**
Scheduler transactions run at `READ COMMITTED` even when the application configured another default
isolation (e.g. `SERIALIZABLE`) for its `Database`; business transactions keep that default. Pass
the level you want to `ExposedTransactionRunner(database, transactionIsolation)`, or `null` to keep
the database default, but note that `REPEATABLE READ` or stricter brings back the `publish()`
blocking described under *Fixed*. (#129)
- **`CompositeMessageDeliverer.deliverBatch`** now dispatches transport groups concurrently — one
virtual thread per group, with the last group run on the calling thread — instead of blocking on
each group in turn. A batch spanning N transports costs ~`max(Tᵢ)` rather than `sum(Tᵢ)`; a batch
Expand Down Expand Up @@ -84,17 +120,6 @@ Until `1.0.0`, breaking changes may appear in any release and are flagged with *
without it, and the PostgreSQL script must run outside a transaction (`CREATE INDEX CONCURRENTLY`).
([#120](https://github.com/softwaremill/okapi/pull/120))

### Fixed

- **A worker no longer marks rows it cannot deliver as `FAILED`.** Previously every worker claimed
all `PENDING` rows, so a worker without a deliverer for a row's type turned it into a permanent
failure. Each worker now claims only its registered delivery types, in at most two queries per
batch. Workers on an older version still claim every type, so publish a newly introduced type only
after all old processors are upgraded or stopped. Rows already marked `FAILED` are not retried
automatically. Cross-type claim order is no longer globally oldest first: the processor rotates
the starting type, and any leftover batch capacity goes to the other types alphabetically.
([#116](https://github.com/softwaremill/okapi/issues/116))

## [1.0.0] — 2026-07-28

First stable release. The public API now follows semantic versioning — breaking changes will
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@ Runnable, self-contained applications live in [okapi-examples](https://github.co
- **Different workers can handle different delivery types.** Each worker claims only its registered types, using at most two claim queries per batch regardless of the number of types. A custom `OutboxStore` used for processing must implement `RouteAwareOutboxStore`; otherwise processing fails before an unsafe claim. The older `claimPending(limit)` method remains available for callers that need it, but Okapi's processor does not use it. Apply the new index migration before starting updated workers if you manage the schema yourself.
- **Rolling upgrades need sequencing.** Workers running an older Okapi version can still claim rows of a newly introduced delivery type. Upgrade or stop all old processors before publishing that type. An upgrade does not automatically retry rows already marked `FAILED` by older workers.
- **Failure classification is the transport's job.** Each deliverer decides what is retriable. HTTP: 5xx, 429, 408 and connection errors are retriable; other responses and TLS errors are permanent. Kafka: broker-side retriable exceptions are retried; authorization and configuration errors are not.
- **On MySQL, scheduler transactions run at `READ COMMITTED`.** The autoconfigured Spring `TransactionRunner` (with `MysqlOutboxStore`) and `ExposedTransactionRunner` (on a MySQL/MariaDB dialect) both use it. At MySQL's default `REPEATABLE READ`, the claim's gap locks would block every concurrent `publish()` until delivery finishes. If you supply your own `TransactionRunner`, an `OutboxStore` of your own, or a `TransactionTemplate` with an explicit isolation level, keep it at `READ COMMITTED` on MySQL. With a `JtaTransactionManager`, or a `JpaTransactionManager` without the Hibernate/EclipseLink `JpaDialect`, okapi cannot set it (startup logs a WARN): configure `transaction_isolation = 'READ-COMMITTED'` on the server or in the JDBC URL instead. `READ COMMITTED` requires `binlog_format=ROW` (the default since MySQL 5.7.7) or `MIXED`; with `STATEMENT`, InnoDB rejects the scheduler's writes with `ERROR 1665`.
- **Retry budget.** `okapi.processor.max-retries` (default 5) counts retries *after* the first attempt — six attempts in total before a row becomes `FAILED`. `FAILED` is terminal. Retriable messages become eligible again on the next processor poll; there is no per-message backoff.

### Deduplicating on `x-outbox-id`
Expand Down
3 changes: 3 additions & 0 deletions gradle/libs.versions.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ slf4j = "2.0.20"
logback = "1.6.5"
assertj = "3.27.7"
h2 = "2.5.252"
jakartaTransaction = "2.0.1"
micrometer = "1.17.2"
jmh = "1.37"
jmhGradlePlugin = "0.7.3"
Expand Down Expand Up @@ -66,6 +67,8 @@ slf4jApi = { module = "org.slf4j:slf4j-api", version.ref = "slf4j" }
slf4jSimple = { module = "org.slf4j:slf4j-simple", version.ref = "slf4j" }
logbackClassic = { module = "ch.qos.logback:logback-classic", version.ref = "logback" }
h2 = { module = "com.h2database:h2", version.ref = "h2" }
# Test-only: a fake UserTransaction for JtaTransactionManager in okapi-spring-boot tests.
jakartaTransactionApi = { module = "jakarta.transaction:jakarta.transaction-api", version.ref = "jakartaTransaction" }
jmhCore = { module = "org.openjdk.jmh:jmh-core", version.ref = "jmh" }
jmhGeneratorAnnprocess = { module = "org.openjdk.jmh:jmh-generator-annprocess", version.ref = "jmh" }

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
package com.softwaremill.okapi.exposed

import com.softwaremill.okapi.core.TransactionRunner
import org.jetbrains.exposed.v1.core.vendors.MysqlDialect
import org.jetbrains.exposed.v1.jdbc.Database
import org.jetbrains.exposed.v1.jdbc.transactions.transaction
import java.sql.Connection

/**
* Exposed implementation of [TransactionRunner].
Expand All @@ -11,10 +13,36 @@ import org.jetbrains.exposed.v1.jdbc.transactions.transaction
* Used by the outbox scheduler/processor when running outside of
* an existing transactional context (e.g., background processing thread).
*
* On MySQL and MariaDB it runs at READ COMMITTED by default (issue #129). Under MySQL's default
* REPEATABLE READ the claim's `FOR UPDATE SKIP LOCKED` takes gap locks at the end of each claimed
* delivery type's range in the claim index -- where every `publish()` of that type inserts -- so publishers block until the
* claiming transaction, and with it delivery, finishes. This overrides a
* `DatabaseConfig.defaultIsolationLevel` set for the rest of the application. Other databases keep
* their default (Postgres already defaults to READ COMMITTED).
*
* @param database The [Database] instance where the outbox table resides.
*/
class ExposedTransactionRunner(
class ExposedTransactionRunner private constructor(
private val database: Database,
explicitIsolation: Int?,
isolationIsExplicit: Boolean,
) : TransactionRunner {
override fun <T> runInTransaction(block: () -> T): T = transaction(database) { block() }
constructor(database: Database) : this(database, null, isolationIsExplicit = false)

/**
* @param transactionIsolation a `java.sql.Connection.TRANSACTION_*` level, or `null` for the
* [database]'s default isolation -- on every database, MySQL included.
*/
constructor(database: Database, transactionIsolation: Int?) : this(database, transactionIsolation, isolationIsExplicit = true)

// Lazy: resolving the dialect may open a connection, which the constructor shouldn't do.
private val transactionIsolation: Int? by lazy {
when {
isolationIsExplicit -> explicitIsolation
database.dialect is MysqlDialect -> Connection.TRANSACTION_READ_COMMITTED // MariaDBDialect extends it
else -> null
}
}

override fun <T> runInTransaction(block: () -> T): T = transaction(database, transactionIsolation = transactionIsolation) { block() }
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
package com.softwaremill.okapi.exposed

import io.kotest.core.spec.style.FunSpec
import io.kotest.matchers.shouldBe
import org.jetbrains.exposed.v1.core.DatabaseConfig
import org.jetbrains.exposed.v1.jdbc.Database
import org.jetbrains.exposed.v1.jdbc.transactions.TransactionManager
import org.jetbrains.exposed.v1.jdbc.transactions.transactionManager
import java.sql.Connection

class ExposedTransactionRunnerTest : FunSpec({

val db = Database.connect(
"jdbc:h2:mem:runner_test_${System.nanoTime()};DB_CLOSE_DELAY=-1",
driver = "org.h2.Driver",
)

fun isolationSeenBy(runner: ExposedTransactionRunner): Int = runner.runInTransaction {
(TransactionManager.current().connection.connection as Connection).transactionIsolation
}

// The MySQL default (READ COMMITTED, #129) is covered against a real MySQL in
// okapi-integration-tests' MysqlClaimDoesNotBlockPublishersTest.
test("keeps the database's default isolation on a non-MySQL database") {
// H2's own default is already READ COMMITTED, so configure another one to tell them apart.
val serializableDb = Database.connect(
"jdbc:h2:mem:runner_test_serializable_${System.nanoTime()};DB_CLOSE_DELAY=-1",
driver = "org.h2.Driver",
databaseConfig = DatabaseConfig { defaultIsolationLevel = Connection.TRANSACTION_SERIALIZABLE },
)
isolationSeenBy(ExposedTransactionRunner(serializableDb)) shouldBe Connection.TRANSACTION_SERIALIZABLE
}

test("honours an explicit READ COMMITTED") {
isolationSeenBy(ExposedTransactionRunner(db, Connection.TRANSACTION_READ_COMMITTED)) shouldBe
Connection.TRANSACTION_READ_COMMITTED
}

test("honours an explicit isolation level") {
isolationSeenBy(ExposedTransactionRunner(db, Connection.TRANSACTION_SERIALIZABLE)) shouldBe
Connection.TRANSACTION_SERIALIZABLE
}

test("null falls back to the database's default isolation") {
isolationSeenBy(ExposedTransactionRunner(db, null)) shouldBe db.transactionManager.defaultIsolationLevel
}
})
1 change: 1 addition & 0 deletions okapi-integration-tests/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ dependencies {
testImplementation(project(":okapi-http"))
testImplementation(project(":okapi-spring-boot"))
testImplementation(project(":okapi-micrometer"))
testImplementation(project(":okapi-exposed"))

// Test framework
testImplementation(libs.kotestRunnerJunit5)
Expand Down
Loading