Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,7 @@
<module>tolerant-reader</module>
<module>trampoline</module>
<module>transaction-script</module>
<module>transactional-outbox</module>
<module>twin</module>
<module>type-object</module>
<module>unit-of-work</module>
Expand Down
240 changes: 240 additions & 0 deletions transactional-outbox/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,240 @@
---
title: "Transactional Outbox Pattern in Java: Ensuring Reliable Event Publishing"
shortTitle: Transactional Outbox
description: "Learn how to implement the Transactional Outbox pattern in Java using Spring Boot and H2. Master reliable event publishing and eliminate dual-write inconsistencies in microservices."
category: Architectural
language: en
tag:
- Spring Boot
- Microservices
- Event-Driven
- Messaging
- Persistence
---

## Also known as

* Outbox Pattern
* Application Event Outbox
* Transactional Event Outbox

## Intent of Transactional Outbox Pattern

The Transactional Outbox pattern reliably publishes events in microservices architectures without requiring distributed transactions (XA/2PC). By persisting business data and event notifications in the same database transaction, it guarantees that message publishing always stays consistent with database changes.

## Detailed Explanation of the Pattern with Real-World Examples

### Real-world analogy

> Imagine writing an important contract and placing the outgoing notice into a postal outbox tray located right next to your desk in a single action. Even if the mail courier arrives later, the document is securely staged in the outbox tray and cannot be lost. A dedicated mail clerk periodically inspects the outbox tray and delivers the letters to the post office.

### In plain words

> Instead of updating the database and publishing a message directly to a message broker in two separate network calls, a service writes both the business entity and an outbox event into the database within a single database transaction. A separate background process periodically reads pending outbox events and publishes them to the message broker.

### Architecture Flow

```
+-------------------------------------------------------------+
| Service Boundary |
| |
| +--------------------+ +------------------------+ |
| | Order Service | | Outbox Publisher | |
| +--------------------+ +------------------------+ |
| | | (Polls) |
| (Atomic Transaction) v |
| | +------------------------+ |
| +------------------> | Outbox Table (Pending) | |
| | +------------------------+ |
| v | (Publishes) |
| +--------------------+ v |
| | Orders Table | +------------------------+ |
| +--------------------+ | Message Broker | |
| +------------------------+ |
+-------------------------------------------------------------+
```

## Programmatic Example (Spring Boot)

### Order & Outbox Entities

The `Order` entity represents business data, while `OutboxEvent` represents the event payload staged for asynchronous publishing.

```java
@Entity
@Table(name = "orders")
@Data
@Builder
public class Order {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private String customerName;
private String productName;
private double amount;
@Enumerated(EnumType.STRING)
private OrderStatus status;
}

@Entity
@Table(name = "outbox_events")
@Data
@Builder
public class OutboxEvent {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private String aggregateType;
private String aggregateId;
private String eventType;
private String payload;
@Enumerated(EnumType.STRING)
private EventStatus status;
private LocalDateTime createdAt;
private LocalDateTime processedAt;
}
```

### Atomic Transactional Write (`OrderService`)

The service saves the order entity and creates an outbox event in the same transactional context using Spring's `@Transactional`.

```java
@Service
@RequiredArgsConstructor
public class OrderService {

private final OrderRepository orderRepository;
private final OutboxRepository outboxRepository;

@Transactional
public Order createOrder(String customerName, String productName, double amount) {
var order = Order.builder()
.customerName(customerName)
.productName(productName)
.amount(amount)
.status(OrderStatus.CREATED)
.createdAt(LocalDateTime.now())
.build();

var savedOrder = orderRepository.save(order);

var outboxEvent = OutboxEvent.builder()
.aggregateType("Order")
.aggregateId(String.valueOf(savedOrder.getId()))
.eventType("ORDER_CREATED")
.payload(String.format("{\"orderId\":%d,\"amount\":%.2f}", savedOrder.getId(), amount))
.status(EventStatus.PENDING)
.createdAt(LocalDateTime.now())
.build();

outboxRepository.save(outboxEvent);
return savedOrder;
}
}
```

### Background Polling Publisher (`OutboxPublisher`)

A scheduled background process polls `PENDING` outbox events, publishes them to the message broker, and marks their status as `PROCESSED`.

```java
@Component
@RequiredArgsConstructor
public class OutboxPublisher {

private final OutboxRepository outboxRepository;
private final MessageBroker messageBroker;

@Scheduled(fixedDelay = 5000)
@Transactional
public void processOutboxEvents() {
List<OutboxEvent> pendingEvents = outboxRepository.findByStatus(EventStatus.PENDING);
for (OutboxEvent event : pendingEvents) {
messageBroker.publish("order-events", event.getPayload());
event.setStatus(EventStatus.PROCESSED);
event.setProcessedAt(LocalDateTime.now());
outboxRepository.save(event);
}
}
}
```

## Class Diagram

```mermaid
classDiagram
class Order {
+Long id
+String customerName
+String productName
+double amount
+OrderStatus status
}
class OutboxEvent {
+Long id
+String aggregateType
+String aggregateId
+String eventType
+String payload
+EventStatus status
+LocalDateTime createdAt
+LocalDateTime processedAt
}
class OrderService {
+createOrder(customerName, productName, amount) Order
}
class OutboxPublisher {
+processOutboxEvents() List~OutboxEvent~
}
class MessageBroker {
<<interface>>
+publish(topic, payload)
}

OrderService ..> Order : creates
OrderService ..> OutboxEvent : creates
OutboxPublisher ..> OutboxEvent : polls & updates
OutboxPublisher --> MessageBroker : dispatches
```

## When to Use the Transactional Outbox Pattern

Use this pattern when:

* You need to update a database and publish messages to an event broker without data loss or inconsistent dual-writes.
* Distributed transactions (XA 2-phase commit) are not supported, perform poorly, or add unwanted complexity.
* You are building event-driven microservices requiring **at-least-once** event delivery guarantees.

## Real-World Applications

* E-commerce checkout systems emitting order creation events for billing and fulfillment services.
* Financial transaction processing services issuing audit log events alongside database updates.
* Microservices using Change Data Capture (CDC) like Debezium for database log mining outbox patterns.

## Benefits and Trade-offs

### Benefits

* **No Dual-Write Inconsistency**: Prevents lost messages or phantom events caused by network/broker outages.
* **At-Least-Once Delivery**: Guarantees event delivery to message consumers.
* **No Distributed Transactions**: Avoids expensive and fragile XA/2PC transactions across services.

### Trade-Offs

* **Near Real-time Latency**: Polling intervals add slight delay before events are dispatched.
* **Duplicate Message Handling**: Consumers must implement idempotent processing to handle potential message redeliveries.
* **Outbox Table Cleanup**: Outbox entries must be periodically archived or purged to prevent uncontrolled table growth.

## Related Java Design Patterns

* [Polling Publisher](https://java-design-patterns.com/patterns/polling-publisher/)
* [Saga Pattern](https://java-design-patterns.com/patterns/saga/)
* [Idempotent Consumer](https://java-design-patterns.com/patterns/microservices-idempotent-consumer/)
* [Event-Driven Architecture](https://java-design-patterns.com/patterns/event-driven-architecture/)

## References and Credits

* [Microservices.io - Pattern: Transactional Outbox](https://microservices.io/patterns/data/transactional-outbox.html)
* [Debezium - Reliable Microservices Data Exchange With the Outbox Pattern](https://debezium.io/blog/2019/02/19/reliable-microservices-data-exchange-with-outbox-pattern/)
* [Designing Data-Intensive Applications - Martin Kleppmann](https://dataintensive.net/)
116 changes: 116 additions & 0 deletions transactional-outbox/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--

This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).

The MIT License
Copyright © 2014-2022 Ilkka Seppälä

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.iluwatar</groupId>
<artifactId>java-design-patterns</artifactId>
<version>1.26.0-SNAPSHOT</version>
</parent>

<artifactId>transactional-outbox</artifactId>

<dependencies>
<!-- Spring Boot Starter Data JPA -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>

<!-- Spring Boot Starter Web -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>

<!-- Lombok -->
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>${lombok.version}</version>
<scope>provided</scope>
</dependency>

<!-- H2 Database -->
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>runtime</scope>
</dependency>

<!-- Spring Boot Starter Test -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>

<!-- JUnit Jupiter Engine -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<scope>test</scope>
</dependency>

<!-- Mockito Core -->
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>com.iluwatar.transactionaloutbox.App</mainClass>
</manifest>
</archive>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Loading
Loading