Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ history, monolith dropped). The full agreed plan is in [`docs/PLAN.md`](docs/PLA
- Name prefix: `ms12p` (parent uses `flaskms`). Teardown deletes the whole resource group, so a
shared name would delete the other project.

## Current state (complete: 12 of 12 live, plus demo 13; one Azure run done and torn down, #11)
## Current state (complete: 12 of 12 live, plus demos 13 and 14; one Azure run done and torn down, #11)
- Folders: `services/{user,catalog,order,payment,inventory}/` (Flask, one DB each), `gateway/`
(FastAPI), `registry/` (FastAPI), `shared/` (`shop_common`), `contracts/`, `frontend/`, `infra/`,
`scripts/`, `tests/`.
Expand All @@ -49,6 +49,14 @@ history, monolith dropped). The full agreed plan is in [`docs/PLAN.md`](docs/PLA
are returned as they are. `saga.settle_charge`: APPROVED confirms, DECLINED 402, VOIDED 503 (both
release), PROCESSING/no answer writes `pending_payment` (checkout and add-item answer 409) and returns 202;
`reconcile.py` settles pending rows every 5 s (deleting and committing the row is the claim). Page `/patterns/slow-payment`.
- Demo 14 (compensation, from a LinkedIn comment): inventory switch `POST /api/inventory/chaos`
(`healthy|slow_reserve|slow_release`, 6 s, past order's 3 s inventory timeout). `stock.release` of an
unknown hold records a RELEASED row (order_id `NEVER_HELD` = 0, no items, logged), so a late reserve
returns it and holds nothing; release never 404s now (ids must be 8-36 chars). `releases_received`
counts release copies (old DBs gain it via `add_missing_columns`). `consumer.counts` has
`duplicates_skipped` and `not_handled` (in `/api/inventory/events`). Demo switches use
`shop_common.flask_support.DemoSwitch` and `switches_off()`. `POST /api/order/sagas/<id>/replay` (chaos, own
checkout) clears `published_at` so the relay resends the same ids. Page `/patterns/compensation`.
- Events (pattern 7): `shop_common/events.py` (Redis Stream `shop.events`, consumer groups, at least
once). Order writes `outbox_event` in the same transaction; `outbox.py` relays every 1 s; inventory's
`consumer.py` commits the hold on OrderConfirmed and dedupes via `consumed_event`. `POST /debug/bus`
Expand Down
28 changes: 24 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ can click.

> **Status: complete.** All 12 patterns are live, each with a working demo, locally and in one
> real Azure run: provision, deploy through GitHub Actions with OIDC, smoke tests, screenshots,
> teardown. Demo 13, a slow payment in the middle of checkout, came from a reader's comment on
> the project's LinkedIn post. The foundation is copied from
> teardown. Demos 13 (a slow payment in the middle of checkout) and 14 (the saga's undo under
> timeouts and duplicate events) came from readers' comments on the project's LinkedIn post. The foundation is copied from
> [Azure-Ecommerce-Monolith-Microservices-Migration](https://github.com/MAOFILHO/Azure-Ecommerce-Monolith-Microservices-Migration).

## Project summary
Expand All @@ -27,15 +27,16 @@ React frontend with one page per pattern. Every demo calls the real services: no
| M3a: async and packaging | 7, 8, 9 | [#4](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/4) |
| M3b: delivery and the Azure run | 3 | [#12](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/12), [#13](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/13), [#14](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/14) |
| Demo 13: a slow payment mid-checkout | 4 and 8, under a timeout | [#16](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/16) |
| Demo 14: compensation under fire | 7 and 8, under timeouts and duplicates | [#17](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/17) |

M3b was planned as 7 GitHub issues ([#5 to #11](https://github.com/MAOFILHO/Microservices-12-Patterns/issues?q=is%3Aissue)),
all closed.

**How it was checked.**
- **181 unit tests** across 8 suites, each in its own virtualenv, built test-first from M3a on.
- **194 unit tests** across 8 suites, each in its own virtualenv, built test-first from M3a on.
- **Pact contract tests**: the order service's contracts with payment, catalog and inventory,
verified against the real providers in CI.
- **25 local smoke tests** against the running stack, and **4 Azure smoke tests** against the real
- **28 local smoke tests** against the running stack, and **4 Azure smoke tests** against the real
deployment.
- **CI on every pull request** runs only what the change touches; every image is built for
linux/amd64, started, and rejected if it runs as root. `main` requires the "CI ok" check.
Expand All @@ -62,6 +63,7 @@ or the cheapest SKUs, with a $25/month budget alert, then torn down and verified
| 11 | [Contract Testing](#11-contract-testing) | Checks that APIs stay compatible across services | M2 |
| 12 | [Service Discovery](#12-service-discovery) | Finds service instances automatically at runtime | M2 |
| 13 | [Saga under a slow payment](#13-saga-under-a-slow-payment) | Follows the money and the stock when payment times out mid-checkout | Demo 13 |
| 14 | [Compensation under fire](#14-compensation-under-fire) | Times out and duplicates the saga's undo, then checks the stock ends where it began | Demo 14 |

## Architecture

Expand Down Expand Up @@ -213,6 +215,24 @@ payment still said PROCESSING, so the order waited; 5 s later it settled as 1 ch

<img width="100%" alt="Demo 13: payment slowed mid-checkout; the order waits as pending, then settles as 1 charge, 1 order, 2 mugs sold" src="docs/screenshots/p13-slow-payment.jpg" />

### 14. Compensation under fire
An undo is a new action, not a rewind, and it crosses the same unreliable network. A second
reader asked for duplicate events and timeouts during compensation. Three cases:

- **A release that beats its hold.** Inventory holds slowly (6 s); the saga gives up after 3 s and
sends a release as an event, which arrives first (about 4 s). Inventory records the release, so
the late hold holds nothing. Before this demo, that late hold kept the stock held for good.
- **A release that times out but goes through.** The saga also sends it as an event. The event copy
lands first, the slow direct call second. Release only acts on a hold still RESERVED, so the
stock comes back once; the page shows both copies arrived.
- **Duplicate events.** A replay button sends a checkout's events again with the same ids.
Inventory acts on 2 event types: one it has acted on before is skipped and counted. The other
types are not its business, so it ignores them (also counted).

Here the hold arrived late: the release came first, the stock went 473 to 473, nothing held, 1 release received.

<img width="100%" alt="Demo 14: inventory holds slowly; the saga's release arrives first, the late hold holds nothing, and the stock ends where it began" src="docs/screenshots/p14-compensation.jpg" />

## Run locally (no Docker)

```bash
Expand Down
Binary file added docs/screenshots/p14-compensation.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
35 changes: 35 additions & 0 deletions frontend/src/api/gatewayClient.ts
Original file line number Diff line number Diff line change
Expand Up @@ -349,3 +349,38 @@ export interface DeliveryView {
export const deliveryApi = {
view: () => request<DeliveryView>("/api/patterns/delivery"),
};

// --- Demo 14: compensation under timeouts and duplicate events -----------------------------------

export interface HoldRow {
reservation_id: string;
order_id: number;
status: "RESERVED" | "SOLD" | "RELEASED" | string;
items: { product_id: number; quantity: number }[];
releases_received: number;
}

export interface HandledEvent {
event_id: string;
type: string;
reservation_id: string | null;
outcome: string;
}

export const compensationApi = {
inventoryMode: () => request<{ mode: string; enabled: boolean }>("/api/shop/inventory/chaos"),
setInventoryMode: (mode: string) =>
getAny<{ mode?: string; message?: string }>("/api/shop/inventory/chaos", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ mode }),
}),
holds: () => request<{ results: HoldRow[] }>("/api/shop/inventory/reservations"),
handled: () =>
request<{ results: HandledEvent[]; duplicates_skipped: number; not_handled: number }>("/api/shop/inventory/events"),
replay: (token: string, sagaId: string) =>
getAny<{ replayed?: number; message?: string }>(`/api/shop/order/sagas/${sagaId}/replay`, {
method: "POST",
headers: authHeaders(token),
}),
};
2 changes: 1 addition & 1 deletion frontend/src/pages/Home.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ export default function Home() {
</Link>
<Link to="/patterns" className="card" style={{ textDecoration: "none", color: "inherit" }}>
<h3>Patterns</h3>
<p className="muted">The 12 patterns. All 12 are live, each with a working demo, plus demo 13: a slow payment in the middle of checkout.</p>
<p className="muted">The 12 patterns. All 12 are live, each with a working demo, plus demo 13 (a slow payment in the middle of checkout) and demo 14 (the saga's undo under timeouts and duplicate events).</p>
</Link>
</div>

Expand Down
2 changes: 2 additions & 0 deletions frontend/src/pages/PatternPage.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import ObservabilityDemo from "../patterns/demos/ObservabilityDemo";
import SagaDemo from "../patterns/demos/SagaDemo";
import ServiceDiscoveryDemo from "../patterns/demos/ServiceDiscoveryDemo";
import SlowPaymentDemo from "../patterns/demos/SlowPaymentDemo";
import CompensationDemo from "../patterns/demos/CompensationDemo";
import StatelessDemo from "../patterns/demos/StatelessDemo";
import { PATTERNS } from "../patterns/content";

Expand All @@ -29,6 +30,7 @@ const DEMOS: Record<string, () => ReactNode> = {
containerization: () => <ContainerizationDemo />,
"ci-cd-pipeline": () => <CiCdDemo />,
"slow-payment": () => <SlowPaymentDemo />,
compensation: () => <CompensationDemo />,
};

export default function PatternPage() {
Expand Down
6 changes: 3 additions & 3 deletions frontend/src/pages/Patterns.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@ import { Link } from "react-router-dom";
import { PATTERNS } from "../patterns/content";

export default function Patterns() {
const patterns = PATTERNS.filter((p) => p.n <= 12); // demo 13 is a bonus, not a 13th pattern
const patterns = PATTERNS.filter((p) => p.n <= 12); // demos 13 and 14 are bonuses, not patterns
const liveCount = patterns.filter((p) => p.live).length;
return (
<div className="grid" style={{ gap: "1.5rem" }}>
<section className="card">
<h1>The 12 patterns, plus demo 13</h1>
<h1>The 12 patterns, plus demos 13 and 14</h1>
<p className="muted">
Each pattern has a page that explains it, says when not to use it, and runs a live demo against the real
services. {liveCount} of {patterns.length} are live. Demo 13 puts patterns 4 and 8 under a slow payment.
services. {liveCount} of {patterns.length} are live. Demo 13 puts patterns 4 and 8 under a slow payment; demo 14 puts the saga's undo under timeouts and duplicate events.
</p>
</section>
<div className="grid cols-3">
Expand Down
23 changes: 23 additions & 0 deletions frontend/src/patterns/content.ts
Original file line number Diff line number Diff line change
Expand Up @@ -303,6 +303,29 @@ export const PATTERNS: PatternInfo[] = [
],
source: { label: "Stripe — Idempotent requests", url: "https://docs.stripe.com/api/idempotent_requests" },
},
{
n: 14,
slug: "compensation",
name: "Compensation under fire",
what: "Times out and duplicates the saga's undo, then checks the stock ends where it began.",
milestone: "M3",
live: true,
problem:
"An undo is a new action, not a rewind, and it travels over the same unreliable network. A release can time out yet go through, so it is sent twice. A release can overtake the slow hold it undoes, so the hold lands afterwards with nothing left to cancel it. And at-least-once delivery sends any event twice.",
inThisProject: [
"Inventory has a demo switch: hold slowly (6 s, past the saga's 3 s wait) or release slowly.",
"When the hold times out, the saga sends its release as an event; it reaches inventory in about 4 s, before the 6 s hold. Inventory records a release for a hold it never saw as RELEASED (and logs it). When the late hold arrives, it finds that record and holds nothing.",
"A release that times out is also sent as an event. The event copy lands first, the slow direct call second. Release only acts on a hold that is still RESERVED, so the second copy changes nothing; the page counts both copies.",
"Inventory acts on 2 event types, OrderConfirmed and StockReleaseRequested. It remembers every id it acted on and skips one it has seen; other types are not its business and it ignores them. The replay button sends a checkout's events again with the same ids, and the page counts what was skipped and what was ignored.",
"Raised by a reader of the project's LinkedIn post, after demo 13.",
],
whenNotToUse: [
"Recording releases for unknown holds keeps rows that never held anything. At scale they need a retention window, longer than any request can be late.",
"An undo that touches the outside world (an email sent, money refunded) cannot be made invisible: the customer sees both actions. Here the undo only moves stock inside one service.",
"Event-id dedupe needs the ids kept as long as a duplicate can arrive. A bus with a short redelivery window can keep fewer.",
],
source: { label: "Microsoft — Compensating Transaction pattern", url: "https://learn.microsoft.com/azure/architecture/patterns/compensating-transaction" },
},
];

export const SERVICE_NAMES = ["gateway", "user", "catalog", "order", "payment", "inventory"] as const;
Loading
Loading