Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ history, monolith dropped). The full agreed plan is in [`docs/PLAN.md`](docs/PLA
- Name prefix: `ms12p` (parent uses `flaskms`). Teardown deletes the whole resource group, so a
shared name would delete the other project.

## Current state (complete: 12 of 12 live, plus demos 13 and 14; one Azure run done and torn down, #11)
## Current state (complete: 12 of 12 live, plus demos 13, 14 and 15; one Azure run done and torn down, #11)
- Folders: `services/{user,catalog,order,payment,inventory}/` (Flask, one DB each), `gateway/`
(FastAPI), `registry/` (FastAPI), `shared/` (`shop_common`), `contracts/`, `frontend/`, `infra/`,
`scripts/`, `tests/`.
Expand Down Expand Up @@ -57,6 +57,14 @@ history, monolith dropped). The full agreed plan is in [`docs/PLAN.md`](docs/PLA
`duplicates_skipped` and `not_handled` (in `/api/inventory/events`). Demo switches use
`shop_common.flask_support.DemoSwitch` and `switches_off()`. `POST /api/order/sagas/<id>/replay` (chaos, own
checkout) clears `published_at` so the relay resends the same ids. Page `/patterns/compensation`.
- Demo 15 (autoscaling, from a LinkedIn comment): `scripts/flash_sale.py`: `sign_up` buyers (not timed), then
`sell`: each buyer buys 1 item; a failed checkout retries the SAME cart (no cart growth); `summarize` counts only
200 as completed. `top_up` brings free stock to a level. `make flash-sale CAP=3` / `CAP=8` (`infra/autoscale_demo.py`,
Azure only) prints the max cost and asks first, sets every service's `--max-replicas`, sets `CHAOS_ENABLED=1` on
inventory only to top up to 2,500 (removed in `finally`), waits for every app's latest revision, runs the sale, waits
out the 5-min cooldown, reads Azure Monitor `Replicas`/`Requests` split by revision, prices sale replicas active and
cooldown replicas idle plus MySQL (`PRICES`, checked 2026-11-18), resets the cap to 3 in `finally`, and writes
`frontend/src/patterns/demos/flashSaleResults.json`. The page `/patterns/autoscaling` shows that file.
- Events (pattern 7): `shop_common/events.py` (Redis Stream `shop.events`, consumer groups, at least
once). Order writes `outbox_event` in the same transaction; `outbox.py` relays every 1 s; inventory's
`consumer.py` commits the hold on OrderConfirmed and dedupes via `consumed_event`. `POST /debug/bus`
Expand Down
6 changes: 5 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: setup run github-identity provision pause resume teardown verify test contracts smoke clean
.PHONY: setup run github-identity provision pause resume teardown verify test contracts smoke flash-sale clean

setup:
python3.12 scripts/setup.py
Expand Down Expand Up @@ -44,6 +44,10 @@ contracts:
smoke:
.venv/bin/python -m pytest tests/smoke -v

# Demo 15, Azure only: the same flash sale under a replica cap. Run CAP=3, then CAP=8.
flash-sale:
.venv/bin/python infra/autoscale_demo.py --cap $(CAP) $(ARGS)

clean:
find . -name "instance" -type d -exec rm -rf {} + 2>/dev/null || true
find . -name "__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true
Expand Down
35 changes: 31 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ can click.

> **Status: complete.** All 12 patterns are live, each with a working demo, locally and in one
> real Azure run: provision, deploy through GitHub Actions with OIDC, smoke tests, screenshots,
> teardown. Demos 13 (a slow payment in the middle of checkout) and 14 (the saga's undo under
> timeouts and duplicate events) came from readers' comments on the project's LinkedIn post. The foundation is copied from
> teardown. Demos 13 (a slow payment in the middle of checkout), 14 (the saga's undo under
> timeouts and duplicate events) and 15 (a flash sale under a replica cap of 3 and of 8, run in
> Azure) came from readers' comments on the project's LinkedIn post. The foundation is copied from
> [Azure-Ecommerce-Monolith-Microservices-Migration](https://github.com/MAOFILHO/Azure-Ecommerce-Monolith-Microservices-Migration).

## Project summary
Expand All @@ -28,15 +29,16 @@ React frontend with one page per pattern. Every demo calls the real services: no
| M3b: delivery and the Azure run | 3 | [#12](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/12), [#13](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/13), [#14](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/14) |
| Demo 13: a slow payment mid-checkout | 4 and 8, under a timeout | [#16](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/16) |
| Demo 14: compensation under fire | 7 and 8, under timeouts and duplicates | [#17](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/17) |
| Demo 15: autoscaling under a cap | 9 and 10, under a flash sale in Azure | [#18](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/18) |

M3b was planned as 7 GitHub issues ([#5 to #11](https://github.com/MAOFILHO/Microservices-12-Patterns/issues?q=is%3Aissue)),
all closed.

**How it was checked.**
- **194 unit tests** across 8 suites, each in its own virtualenv, built test-first from M3a on.
- **206 unit tests** across 8 suites, each in its own virtualenv, built test-first from M3a on.
- **Pact contract tests**: the order service's contracts with payment, catalog and inventory,
verified against the real providers in CI.
- **28 local smoke tests** against the running stack, and **4 Azure smoke tests** against the real
- **29 local smoke tests** against the running stack, and **4 Azure smoke tests** against the real
deployment.
- **CI on every pull request** runs only what the change touches; every image is built for
linux/amd64, started, and rejected if it runs as root. `main` requires the "CI ok" check.
Expand Down Expand Up @@ -64,6 +66,7 @@ or the cheapest SKUs, with a $25/month budget alert, then torn down and verified
| 12 | [Service Discovery](#12-service-discovery) | Finds service instances automatically at runtime | M2 |
| 13 | [Saga under a slow payment](#13-saga-under-a-slow-payment) | Follows the money and the stock when payment times out mid-checkout | Demo 13 |
| 14 | [Compensation under fire](#14-compensation-under-fire) | Times out and duplicates the saga's undo, then checks the stock ends where it began | Demo 14 |
| 15 | [Autoscaling under a cap](#15-autoscaling-under-a-cap) | Runs the same flash sale with a replica cap of 3 and of 8, and compares success rate and cost per 1,000 orders | Demo 15 |

## Architecture

Expand Down Expand Up @@ -233,6 +236,30 @@ Here the hold arrived late: the release came first, the stock went 473 to 473, n

<img width="100%" alt="Demo 14: inventory holds slowly; the saga's release arrives first, the late hold holds nothing, and the stock ends where it began" src="docs/screenshots/p14-compensation.jpg" />

### 15. Autoscaling under a cap
A third reader asked how a node cap changes cloud cost: run the same flash sale with a low and a
high cap, and compare checkout success rate and cost per 1,000 completed orders. Container Apps
caps replicas, not nodes, so `make flash-sale CAP=3` and `CAP=8` cap every service at that many
replicas. 80 buyers each buy 1 item, over and over, for 4 minutes, from the same starting stock. A
failed checkout is retried with the same cart, so every order is 1 item. Cost is read from Azure
Monitor and priced at list price: replicas during the sale and the 5-minute cooldown after it,
requests, and the MySQL server for the same time.

| | Cap 3 | Cap 8 |
|---|---|---|
| Checkout success rate | 50.3% (1,289 of 2,563) | 99.2% (5,193 of 5,235) |
| Completed orders per minute | 322 | 1,298 |
| Median checkout time | 8.5 s | 2.5 s |
| Cost of the run | $0.032 | $0.067 |
| **Cost per 1,000 completed orders** | **$0.025** | **$0.013** |

The higher cap cost 2.1 times as much and sold 4 times as many orders, so each order cost about
half as much. The single small MySQL server was not the limit at this load; the gateway and order
service, capped at 3 replicas, were. One run each, 19 November 2026, East US: a trend, not a
benchmark.

<img width="100%" alt="Demo 15: the same flash sale at a cap of 3 and of 8 replicas; cap 8 completes 99% of checkouts and halves the cost per 1,000 orders" src="docs/screenshots/p15-autoscaling.jpg" />

## Run locally (no Docker)

```bash
Expand Down
Binary file added docs/screenshots/p15-autoscaling.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion frontend/src/pages/Home.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ export default function Home() {
</Link>
<Link to="/patterns" className="card" style={{ textDecoration: "none", color: "inherit" }}>
<h3>Patterns</h3>
<p className="muted">The 12 patterns. All 12 are live, each with a working demo, plus demo 13 (a slow payment in the middle of checkout) and demo 14 (the saga's undo under timeouts and duplicate events).</p>
<p className="muted">The 12 patterns. All 12 are live, each with a working demo, plus demo 13 (a slow payment in the middle of checkout), demo 14 (the saga's undo under timeouts and duplicate events) and demo 15 (a flash sale under a replica cap of 3 and of 8).</p>
</Link>
</div>

Expand Down
2 changes: 2 additions & 0 deletions frontend/src/pages/PatternPage.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import SagaDemo from "../patterns/demos/SagaDemo";
import ServiceDiscoveryDemo from "../patterns/demos/ServiceDiscoveryDemo";
import SlowPaymentDemo from "../patterns/demos/SlowPaymentDemo";
import CompensationDemo from "../patterns/demos/CompensationDemo";
import AutoscalingDemo from "../patterns/demos/AutoscalingDemo";
import StatelessDemo from "../patterns/demos/StatelessDemo";
import { PATTERNS } from "../patterns/content";

Expand All @@ -31,6 +32,7 @@ const DEMOS: Record<string, () => ReactNode> = {
"ci-cd-pipeline": () => <CiCdDemo />,
"slow-payment": () => <SlowPaymentDemo />,
compensation: () => <CompensationDemo />,
autoscaling: () => <AutoscalingDemo />,
};

export default function PatternPage() {
Expand Down
6 changes: 3 additions & 3 deletions frontend/src/pages/Patterns.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@ import { Link } from "react-router-dom";
import { PATTERNS } from "../patterns/content";

export default function Patterns() {
const patterns = PATTERNS.filter((p) => p.n <= 12); // demos 13 and 14 are bonuses, not patterns
const patterns = PATTERNS.filter((p) => p.n <= 12); // demos 13-15 are bonuses, not patterns
const liveCount = patterns.filter((p) => p.live).length;
return (
<div className="grid" style={{ gap: "1.5rem" }}>
<section className="card">
<h1>The 12 patterns, plus demos 13 and 14</h1>
<h1>The 12 patterns, plus demos 13, 14 and 15</h1>
<p className="muted">
Each pattern has a page that explains it, says when not to use it, and runs a live demo against the real
services. {liveCount} of {patterns.length} are live. Demo 13 puts patterns 4 and 8 under a slow payment; demo 14 puts the saga's undo under timeouts and duplicate events.
services. {liveCount} of {patterns.length} are live. Demo 13 puts patterns 4 and 8 under a slow payment; demo 14 puts the saga's undo under timeouts and duplicate events; demo 15 runs a flash sale under a replica cap of 3 and of 8.
</p>
</section>
<div className="grid cols-3">
Expand Down
23 changes: 23 additions & 0 deletions frontend/src/patterns/content.ts
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,29 @@ export const PATTERNS: PatternInfo[] = [
],
source: { label: "Microsoft — Compensating Transaction pattern", url: "https://learn.microsoft.com/azure/architecture/patterns/compensating-transaction" },
},
{
n: 15,
slug: "autoscaling",
name: "Autoscaling under a cap",
what: "Runs the same flash sale with a replica cap of 3 and of 8, and compares success rate and cost per 1,000 orders.",
milestone: "M3",
live: true,
problem:
"Autoscaling adds copies of a service when load rises, but only up to a cap. A low cap keeps the bill small and loses sales at the peak. A high cap keeps sales and costs more, unless something else, such as the database, is the real limit. You only know which by measuring.",
inThisProject: [
"Every service is a Container App on the Consumption plan: 0.25 vCPU and 0.5 GiB per replica, scaled on HTTP load (10 requests at once per replica), billed per second.",
"make flash-sale CAP=3 sets every service's cap, tops stock up to 2,500 of each product, and runs scripts/flash_sale.py: 80 signed-in buyers each buy 1 item, over and over, for 4 minutes. A failed checkout is retried with the same cart, so every order is 1 item. Then CAP=8 runs the same sale.",
"Afterwards it reads Azure Monitor: replicas per minute and requests per service, during the sale and through the 5-minute cooldown. It prices them at list price (idle rate for the cooldown) and adds the MySQL server for the same time.",
"Kubernetes caps nodes, Container Apps caps replicas: the question is the same, what the cap costs and what it saves.",
"Raised by a reader of the project's LinkedIn post, after demo 14.",
],
whenNotToUse: [
"A shop with steady traffic gains little from autoscaling: a fixed size is cheaper to reason about.",
"Raising the cap does not help when a shared piece (here 1 small MySQL server) is the limit: more replicas only queue up at it.",
"Scale-out takes tens of seconds: a sale that peaks in seconds needs capacity warmed up beforehand (min replicas), not a higher cap.",
],
source: { label: "Microsoft — Scaling in Azure Container Apps", url: "https://learn.microsoft.com/azure/container-apps/scale-app" },
},
];

export const SERVICE_NAMES = ["gateway", "user", "catalog", "order", "payment", "inventory"] as const;
170 changes: 170 additions & 0 deletions frontend/src/patterns/demos/AutoscalingDemo.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
import { SERVICE_NAMES } from "../content";
import results from "./flashSaleResults.json";

interface AppUsage {
replica_seconds: number;
cooldown_replica_seconds: number;
peak_replicas: number;
requests: number;
}

interface FlashSaleRun {
cap: number;
buyers: number;
seconds: number;
region: string;
ran_at: string;
attempts: number;
completed: number;
success_rate: number;
by_status: Record<string, number>;
p50_ms: number;
p95_ms: number;
orders_per_minute: number;
usage: Record<string, AppUsage>;
cost_usd: { replicas: number; requests: number; mysql: number; total: number };
cost_per_1k_usd: number | null;
}

const RUNS = results.runs as FlashSaleRun[];
const PRICES = results.prices;

function replicaMinutes(run: FlashSaleRun, key: "replica_seconds" | "cooldown_replica_seconds") {
return Math.round(SERVICE_NAMES.reduce((sum, n) => sum + (run.usage[n]?.[key] ?? 0), 0) / 60);
}

function pct(share: number) {
return `${(share * 100).toFixed(1)}%`;
}

function usd(value: number | null) {
return value === null ? "–" : `$${value.toFixed(4)}`;
}

const STATUS_LABEL: Record<string, string> = {
"202": "payment pending",
"404": "no cart left: an earlier try went through after a timeout",
"409": "sold out, pending, or a checkout already running",
"503": "a service was down or its breaker open",
"no answer": "no answer in 30 s",
};

function failures(run: FlashSaleRun) {
const lost = Object.entries(run.by_status).filter(([status]) => status !== "200");
return lost.length === 0
? "none"
: lost.map(([status, n]) => `${n} × ${status} (${STATUS_LABEL[status] ?? "error"})`).join(", ");
}

/** Demo 15: the same flash sale twice in Azure, with a replica cap of 3 and of 8. Recorded, not live. */
export default function AutoscalingDemo() {
return (
<div className="grid" style={{ gap: "1.25rem" }}>
<div>
<h3>The experiment</h3>
<p className="muted">
Many buyers sign in, then each adds 1 item and checks out, again and again, for the same time. Azure Container
Apps adds a replica when each replica of a service is handling more than 10 requests at once, up to the cap. Run 1 caps every service at
3 replicas, run 2 at 8. Nothing else changes: the same 0.25 vCPU replicas, the same single MySQL server.
</p>
<p className="muted" style={{ fontSize: "0.85rem" }}>
Recorded in Azure with <code>make flash-sale CAP=3</code> and <code>CAP=8</code>, then torn down: the stack does
not stay up, so this page shows the saved numbers.
</p>
</div>

{RUNS.length === 0 ? (
<p>No run recorded yet.</p>
) : (
<>
<div className="table-scroll">
<table className="data-table">
<thead>
<tr>
<th></th>
{RUNS.map((r) => (
<th key={r.cap}>Cap {r.cap}</th>
))}
</tr>
</thead>
<tbody>
<tr>
<td>Checkout success rate</td>
{RUNS.map((r) => (
<td key={r.cap}>
<strong>{pct(r.success_rate)}</strong> ({r.completed} of {r.attempts})
</td>
))}
</tr>
<tr>
<td>Completed orders per minute</td>
{RUNS.map((r) => (
<td key={r.cap}>{r.orders_per_minute}</td>
))}
</tr>
<tr>
<td>Checkout time, median / 95th percentile</td>
{RUNS.map((r) => (
<td key={r.cap}>
{Math.round(r.p50_ms)} ms / {Math.round(r.p95_ms)} ms
</td>
))}
</tr>
<tr>
<td>Lost sales (HTTP status)</td>
{RUNS.map((r) => (
<td key={r.cap}>{failures(r)}</td>
))}
</tr>
<tr>
<td>Peak replicas (gateway, order)</td>
{RUNS.map((r) => (
<td key={r.cap}>
{r.usage.gateway?.peak_replicas ?? "?"}, {r.usage.order?.peak_replicas ?? "?"}
</td>
))}
</tr>
<tr>
<td>Replica time, all 6 services: sale / cooldown</td>
{RUNS.map((r) => (
<td key={r.cap}>
{replicaMinutes(r, "replica_seconds")} / {replicaMinutes(r, "cooldown_replica_seconds")} replica-minutes
</td>
))}
</tr>
<tr>
<td>Cost of the run: replicas + requests + MySQL</td>
{RUNS.map((r) => (
<td key={r.cap}>
{usd(r.cost_usd.total)} = {usd(r.cost_usd.replicas)} + {usd(r.cost_usd.requests)} +{" "}
{usd(r.cost_usd.mysql)}
</td>
))}
</tr>
<tr>
<td>
<strong>Cost per 1,000 completed orders</strong>
</td>
{RUNS.map((r) => (
<td key={r.cap}>
<strong>{usd(r.cost_per_1k_usd)}</strong>
</td>
))}
</tr>
</tbody>
</table>
</div>
<p className="muted" style={{ fontSize: "0.85rem" }}>
{RUNS.map((r) => `Cap ${r.cap}: ${r.buyers} buyers for ${r.seconds} s in ${r.region}, ${r.ran_at}.`).join(" ")}{" "}
Both runs start from 2,500 free units of each product. A buyer whose checkout fails pays for the same 1-item
cart again, so every completed order is 1 item. Azure's list prices before the monthly free grant: replicas
during the sale at the active rate (${PRICES.vcpu_second} per vCPU-second, ${PRICES.gib_second} per
GiB-second), replicas in the 5-minute cooldown after it at the idle rate (${PRICES.idle_vcpu_second} per
vCPU-second), ${PRICES.per_million_requests} per million requests, and the MySQL server at $
{PRICES.mysql_hour} an hour for the same time.
</p>
</>
)}
</div>
);
}
Loading
Loading