diff --git a/CLAUDE.md b/CLAUDE.md index 4ebd105..a1dec94 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -27,7 +27,7 @@ history, monolith dropped). The full agreed plan is in [`docs/PLAN.md`](docs/PLA - Name prefix: `ms12p` (parent uses `flaskms`). Teardown deletes the whole resource group, so a shared name would delete the other project. -## Current state (complete: 12 of 12 live, plus demos 13 and 14; one Azure run done and torn down, #11) +## Current state (complete: 12 of 12 live, plus demos 13, 14 and 15; one Azure run done and torn down, #11) - Folders: `services/{user,catalog,order,payment,inventory}/` (Flask, one DB each), `gateway/` (FastAPI), `registry/` (FastAPI), `shared/` (`shop_common`), `contracts/`, `frontend/`, `infra/`, `scripts/`, `tests/`. @@ -57,6 +57,14 @@ history, monolith dropped). The full agreed plan is in [`docs/PLAN.md`](docs/PLA `duplicates_skipped` and `not_handled` (in `/api/inventory/events`). Demo switches use `shop_common.flask_support.DemoSwitch` and `switches_off()`. `POST /api/order/sagas//replay` (chaos, own checkout) clears `published_at` so the relay resends the same ids. Page `/patterns/compensation`. +- Demo 15 (autoscaling, from a LinkedIn comment): `scripts/flash_sale.py`: `sign_up` buyers (not timed), then + `sell`: each buyer buys 1 item; a failed checkout retries the SAME cart (no cart growth); `summarize` counts only + 200 as completed. `top_up` brings free stock to a level. `make flash-sale CAP=3` / `CAP=8` (`infra/autoscale_demo.py`, + Azure only) prints the max cost and asks first, sets every service's `--max-replicas`, sets `CHAOS_ENABLED=1` on + inventory only to top up to 2,500 (removed in `finally`), waits for every app's latest revision, runs the sale, waits + out the 5-min cooldown, reads Azure Monitor `Replicas`/`Requests` split by revision, prices sale replicas active and + cooldown replicas idle plus MySQL (`PRICES`, checked 2026-11-18), resets the cap to 3 in `finally`, and writes + `frontend/src/patterns/demos/flashSaleResults.json`. The page `/patterns/autoscaling` shows that file. - Events (pattern 7): `shop_common/events.py` (Redis Stream `shop.events`, consumer groups, at least once). Order writes `outbox_event` in the same transaction; `outbox.py` relays every 1 s; inventory's `consumer.py` commits the hold on OrderConfirmed and dedupes via `consumed_event`. `POST /debug/bus` diff --git a/Makefile b/Makefile index 952236c..ee114fd 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: setup run github-identity provision pause resume teardown verify test contracts smoke clean +.PHONY: setup run github-identity provision pause resume teardown verify test contracts smoke flash-sale clean setup: python3.12 scripts/setup.py @@ -44,6 +44,10 @@ contracts: smoke: .venv/bin/python -m pytest tests/smoke -v +# Demo 15, Azure only: the same flash sale under a replica cap. Run CAP=3, then CAP=8. +flash-sale: + .venv/bin/python infra/autoscale_demo.py --cap $(CAP) $(ARGS) + clean: find . -name "instance" -type d -exec rm -rf {} + 2>/dev/null || true find . -name "__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true diff --git a/README.md b/README.md index a03671e..a382964 100644 --- a/README.md +++ b/README.md @@ -6,8 +6,9 @@ can click. > **Status: complete.** All 12 patterns are live, each with a working demo, locally and in one > real Azure run: provision, deploy through GitHub Actions with OIDC, smoke tests, screenshots, -> teardown. Demos 13 (a slow payment in the middle of checkout) and 14 (the saga's undo under -> timeouts and duplicate events) came from readers' comments on the project's LinkedIn post. The foundation is copied from +> teardown. Demos 13 (a slow payment in the middle of checkout), 14 (the saga's undo under +> timeouts and duplicate events) and 15 (a flash sale under a replica cap of 3 and of 8, run in +> Azure) came from readers' comments on the project's LinkedIn post. The foundation is copied from > [Azure-Ecommerce-Monolith-Microservices-Migration](https://github.com/MAOFILHO/Azure-Ecommerce-Monolith-Microservices-Migration). ## Project summary @@ -28,15 +29,16 @@ React frontend with one page per pattern. Every demo calls the real services: no | M3b: delivery and the Azure run | 3 | [#12](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/12), [#13](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/13), [#14](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/14) | | Demo 13: a slow payment mid-checkout | 4 and 8, under a timeout | [#16](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/16) | | Demo 14: compensation under fire | 7 and 8, under timeouts and duplicates | [#17](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/17) | +| Demo 15: autoscaling under a cap | 9 and 10, under a flash sale in Azure | [#18](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/18) | M3b was planned as 7 GitHub issues ([#5 to #11](https://github.com/MAOFILHO/Microservices-12-Patterns/issues?q=is%3Aissue)), all closed. **How it was checked.** -- **194 unit tests** across 8 suites, each in its own virtualenv, built test-first from M3a on. +- **206 unit tests** across 8 suites, each in its own virtualenv, built test-first from M3a on. - **Pact contract tests**: the order service's contracts with payment, catalog and inventory, verified against the real providers in CI. -- **28 local smoke tests** against the running stack, and **4 Azure smoke tests** against the real +- **29 local smoke tests** against the running stack, and **4 Azure smoke tests** against the real deployment. - **CI on every pull request** runs only what the change touches; every image is built for linux/amd64, started, and rejected if it runs as root. `main` requires the "CI ok" check. @@ -64,6 +66,7 @@ or the cheapest SKUs, with a $25/month budget alert, then torn down and verified | 12 | [Service Discovery](#12-service-discovery) | Finds service instances automatically at runtime | M2 | | 13 | [Saga under a slow payment](#13-saga-under-a-slow-payment) | Follows the money and the stock when payment times out mid-checkout | Demo 13 | | 14 | [Compensation under fire](#14-compensation-under-fire) | Times out and duplicates the saga's undo, then checks the stock ends where it began | Demo 14 | +| 15 | [Autoscaling under a cap](#15-autoscaling-under-a-cap) | Runs the same flash sale with a replica cap of 3 and of 8, and compares success rate and cost per 1,000 orders | Demo 15 | ## Architecture @@ -233,6 +236,30 @@ Here the hold arrived late: the release came first, the stock went 473 to 473, n Demo 14: inventory holds slowly; the saga's release arrives first, the late hold holds nothing, and the stock ends where it began +### 15. Autoscaling under a cap +A third reader asked how a node cap changes cloud cost: run the same flash sale with a low and a +high cap, and compare checkout success rate and cost per 1,000 completed orders. Container Apps +caps replicas, not nodes, so `make flash-sale CAP=3` and `CAP=8` cap every service at that many +replicas. 80 buyers each buy 1 item, over and over, for 4 minutes, from the same starting stock. A +failed checkout is retried with the same cart, so every order is 1 item. Cost is read from Azure +Monitor and priced at list price: replicas during the sale and the 5-minute cooldown after it, +requests, and the MySQL server for the same time. + +| | Cap 3 | Cap 8 | +|---|---|---| +| Checkout success rate | 50.3% (1,289 of 2,563) | 99.2% (5,193 of 5,235) | +| Completed orders per minute | 322 | 1,298 | +| Median checkout time | 8.5 s | 2.5 s | +| Cost of the run | $0.032 | $0.067 | +| **Cost per 1,000 completed orders** | **$0.025** | **$0.013** | + +The higher cap cost 2.1 times as much and sold 4 times as many orders, so each order cost about +half as much. The single small MySQL server was not the limit at this load; the gateway and order +service, capped at 3 replicas, were. One run each, 19 November 2026, East US: a trend, not a +benchmark. + +Demo 15: the same flash sale at a cap of 3 and of 8 replicas; cap 8 completes 99% of checkouts and halves the cost per 1,000 orders + ## Run locally (no Docker) ```bash diff --git a/docs/screenshots/p15-autoscaling.jpg b/docs/screenshots/p15-autoscaling.jpg new file mode 100644 index 0000000..d90a596 Binary files /dev/null and b/docs/screenshots/p15-autoscaling.jpg differ diff --git a/frontend/src/pages/Home.tsx b/frontend/src/pages/Home.tsx index 96032c0..82ef439 100644 --- a/frontend/src/pages/Home.tsx +++ b/frontend/src/pages/Home.tsx @@ -18,7 +18,7 @@ export default function Home() {

Patterns

-

The 12 patterns. All 12 are live, each with a working demo, plus demo 13 (a slow payment in the middle of checkout) and demo 14 (the saga's undo under timeouts and duplicate events).

+

The 12 patterns. All 12 are live, each with a working demo, plus demo 13 (a slow payment in the middle of checkout), demo 14 (the saga's undo under timeouts and duplicate events) and demo 15 (a flash sale under a replica cap of 3 and of 8).

diff --git a/frontend/src/pages/PatternPage.tsx b/frontend/src/pages/PatternPage.tsx index a30abee..2e1f1cd 100644 --- a/frontend/src/pages/PatternPage.tsx +++ b/frontend/src/pages/PatternPage.tsx @@ -13,6 +13,7 @@ import SagaDemo from "../patterns/demos/SagaDemo"; import ServiceDiscoveryDemo from "../patterns/demos/ServiceDiscoveryDemo"; import SlowPaymentDemo from "../patterns/demos/SlowPaymentDemo"; import CompensationDemo from "../patterns/demos/CompensationDemo"; +import AutoscalingDemo from "../patterns/demos/AutoscalingDemo"; import StatelessDemo from "../patterns/demos/StatelessDemo"; import { PATTERNS } from "../patterns/content"; @@ -31,6 +32,7 @@ const DEMOS: Record ReactNode> = { "ci-cd-pipeline": () => , "slow-payment": () => , compensation: () => , + autoscaling: () => , }; export default function PatternPage() { diff --git a/frontend/src/pages/Patterns.tsx b/frontend/src/pages/Patterns.tsx index 0764e8a..023275c 100644 --- a/frontend/src/pages/Patterns.tsx +++ b/frontend/src/pages/Patterns.tsx @@ -2,15 +2,15 @@ import { Link } from "react-router-dom"; import { PATTERNS } from "../patterns/content"; export default function Patterns() { - const patterns = PATTERNS.filter((p) => p.n <= 12); // demos 13 and 14 are bonuses, not patterns + const patterns = PATTERNS.filter((p) => p.n <= 12); // demos 13-15 are bonuses, not patterns const liveCount = patterns.filter((p) => p.live).length; return (
-

The 12 patterns, plus demos 13 and 14

+

The 12 patterns, plus demos 13, 14 and 15

Each pattern has a page that explains it, says when not to use it, and runs a live demo against the real - services. {liveCount} of {patterns.length} are live. Demo 13 puts patterns 4 and 8 under a slow payment; demo 14 puts the saga's undo under timeouts and duplicate events. + services. {liveCount} of {patterns.length} are live. Demo 13 puts patterns 4 and 8 under a slow payment; demo 14 puts the saga's undo under timeouts and duplicate events; demo 15 runs a flash sale under a replica cap of 3 and of 8.

diff --git a/frontend/src/patterns/content.ts b/frontend/src/patterns/content.ts index ffce541..d3b420d 100644 --- a/frontend/src/patterns/content.ts +++ b/frontend/src/patterns/content.ts @@ -326,6 +326,29 @@ export const PATTERNS: PatternInfo[] = [ ], source: { label: "Microsoft — Compensating Transaction pattern", url: "https://learn.microsoft.com/azure/architecture/patterns/compensating-transaction" }, }, + { + n: 15, + slug: "autoscaling", + name: "Autoscaling under a cap", + what: "Runs the same flash sale with a replica cap of 3 and of 8, and compares success rate and cost per 1,000 orders.", + milestone: "M3", + live: true, + problem: + "Autoscaling adds copies of a service when load rises, but only up to a cap. A low cap keeps the bill small and loses sales at the peak. A high cap keeps sales and costs more, unless something else, such as the database, is the real limit. You only know which by measuring.", + inThisProject: [ + "Every service is a Container App on the Consumption plan: 0.25 vCPU and 0.5 GiB per replica, scaled on HTTP load (10 requests at once per replica), billed per second.", + "make flash-sale CAP=3 sets every service's cap, tops stock up to 2,500 of each product, and runs scripts/flash_sale.py: 80 signed-in buyers each buy 1 item, over and over, for 4 minutes. A failed checkout is retried with the same cart, so every order is 1 item. Then CAP=8 runs the same sale.", + "Afterwards it reads Azure Monitor: replicas per minute and requests per service, during the sale and through the 5-minute cooldown. It prices them at list price (idle rate for the cooldown) and adds the MySQL server for the same time.", + "Kubernetes caps nodes, Container Apps caps replicas: the question is the same, what the cap costs and what it saves.", + "Raised by a reader of the project's LinkedIn post, after demo 14.", + ], + whenNotToUse: [ + "A shop with steady traffic gains little from autoscaling: a fixed size is cheaper to reason about.", + "Raising the cap does not help when a shared piece (here 1 small MySQL server) is the limit: more replicas only queue up at it.", + "Scale-out takes tens of seconds: a sale that peaks in seconds needs capacity warmed up beforehand (min replicas), not a higher cap.", + ], + source: { label: "Microsoft — Scaling in Azure Container Apps", url: "https://learn.microsoft.com/azure/container-apps/scale-app" }, + }, ]; export const SERVICE_NAMES = ["gateway", "user", "catalog", "order", "payment", "inventory"] as const; diff --git a/frontend/src/patterns/demos/AutoscalingDemo.tsx b/frontend/src/patterns/demos/AutoscalingDemo.tsx new file mode 100644 index 0000000..347a264 --- /dev/null +++ b/frontend/src/patterns/demos/AutoscalingDemo.tsx @@ -0,0 +1,170 @@ +import { SERVICE_NAMES } from "../content"; +import results from "./flashSaleResults.json"; + +interface AppUsage { + replica_seconds: number; + cooldown_replica_seconds: number; + peak_replicas: number; + requests: number; +} + +interface FlashSaleRun { + cap: number; + buyers: number; + seconds: number; + region: string; + ran_at: string; + attempts: number; + completed: number; + success_rate: number; + by_status: Record; + p50_ms: number; + p95_ms: number; + orders_per_minute: number; + usage: Record; + cost_usd: { replicas: number; requests: number; mysql: number; total: number }; + cost_per_1k_usd: number | null; +} + +const RUNS = results.runs as FlashSaleRun[]; +const PRICES = results.prices; + +function replicaMinutes(run: FlashSaleRun, key: "replica_seconds" | "cooldown_replica_seconds") { + return Math.round(SERVICE_NAMES.reduce((sum, n) => sum + (run.usage[n]?.[key] ?? 0), 0) / 60); +} + +function pct(share: number) { + return `${(share * 100).toFixed(1)}%`; +} + +function usd(value: number | null) { + return value === null ? "–" : `$${value.toFixed(4)}`; +} + +const STATUS_LABEL: Record = { + "202": "payment pending", + "404": "no cart left: an earlier try went through after a timeout", + "409": "sold out, pending, or a checkout already running", + "503": "a service was down or its breaker open", + "no answer": "no answer in 30 s", +}; + +function failures(run: FlashSaleRun) { + const lost = Object.entries(run.by_status).filter(([status]) => status !== "200"); + return lost.length === 0 + ? "none" + : lost.map(([status, n]) => `${n} × ${status} (${STATUS_LABEL[status] ?? "error"})`).join(", "); +} + +/** Demo 15: the same flash sale twice in Azure, with a replica cap of 3 and of 8. Recorded, not live. */ +export default function AutoscalingDemo() { + return ( +
+
+

The experiment

+

+ Many buyers sign in, then each adds 1 item and checks out, again and again, for the same time. Azure Container + Apps adds a replica when each replica of a service is handling more than 10 requests at once, up to the cap. Run 1 caps every service at + 3 replicas, run 2 at 8. Nothing else changes: the same 0.25 vCPU replicas, the same single MySQL server. +

+

+ Recorded in Azure with make flash-sale CAP=3 and CAP=8, then torn down: the stack does + not stay up, so this page shows the saved numbers. +

+
+ + {RUNS.length === 0 ? ( +

No run recorded yet.

+ ) : ( + <> +
+ + + + + {RUNS.map((r) => ( + + ))} + + + + + + {RUNS.map((r) => ( + + ))} + + + + {RUNS.map((r) => ( + + ))} + + + + {RUNS.map((r) => ( + + ))} + + + + {RUNS.map((r) => ( + + ))} + + + + {RUNS.map((r) => ( + + ))} + + + + {RUNS.map((r) => ( + + ))} + + + + {RUNS.map((r) => ( + + ))} + + + + {RUNS.map((r) => ( + + ))} + + +
Cap {r.cap}
Checkout success rate + {pct(r.success_rate)} ({r.completed} of {r.attempts}) +
Completed orders per minute{r.orders_per_minute}
Checkout time, median / 95th percentile + {Math.round(r.p50_ms)} ms / {Math.round(r.p95_ms)} ms +
Lost sales (HTTP status){failures(r)}
Peak replicas (gateway, order) + {r.usage.gateway?.peak_replicas ?? "?"}, {r.usage.order?.peak_replicas ?? "?"} +
Replica time, all 6 services: sale / cooldown + {replicaMinutes(r, "replica_seconds")} / {replicaMinutes(r, "cooldown_replica_seconds")} replica-minutes +
Cost of the run: replicas + requests + MySQL + {usd(r.cost_usd.total)} = {usd(r.cost_usd.replicas)} + {usd(r.cost_usd.requests)} +{" "} + {usd(r.cost_usd.mysql)} +
+ Cost per 1,000 completed orders + + {usd(r.cost_per_1k_usd)} +
+
+

+ {RUNS.map((r) => `Cap ${r.cap}: ${r.buyers} buyers for ${r.seconds} s in ${r.region}, ${r.ran_at}.`).join(" ")}{" "} + Both runs start from 2,500 free units of each product. A buyer whose checkout fails pays for the same 1-item + cart again, so every completed order is 1 item. Azure's list prices before the monthly free grant: replicas + during the sale at the active rate (${PRICES.vcpu_second} per vCPU-second, ${PRICES.gib_second} per + GiB-second), replicas in the 5-minute cooldown after it at the idle rate (${PRICES.idle_vcpu_second} per + vCPU-second), ${PRICES.per_million_requests} per million requests, and the MySQL server at $ + {PRICES.mysql_hour} an hour for the same time. +

+ + )} +
+ ); +} diff --git a/frontend/src/patterns/demos/flashSaleResults.json b/frontend/src/patterns/demos/flashSaleResults.json new file mode 100644 index 0000000..6076464 --- /dev/null +++ b/frontend/src/patterns/demos/flashSaleResults.json @@ -0,0 +1,139 @@ +{ + "prices": { + "vcpu_second": 2.4e-05, + "idle_vcpu_second": 3e-06, + "gib_second": 3e-06, + "per_million_requests": 0.4, + "mysql_hour": 0.017 + }, + "runs": [ + { + "cap": 3, + "buyers": 80, + "seconds": 240, + "region": "eastus", + "ran_at": "2026-11-19T03:25:26+00:00", + "attempts": 2563, + "completed": 1289, + "success_rate": 0.5029, + "by_status": { + "503": 1076, + "409": 95, + "202": 1, + "200": 1289, + "404": 102 + }, + "p50_ms": 8523.6, + "p95_ms": 14713.3, + "orders_per_minute": 322.2, + "usage": { + "user": { + "replica_seconds": 240.0, + "peak_replicas": 1, + "requests": 96, + "cooldown_replica_seconds": 210.0 + }, + "catalog": { + "replica_seconds": 210.0, + "peak_replicas": 2, + "requests": 1980, + "cooldown_replica_seconds": 600.0 + }, + "order": { + "replica_seconds": 450.0, + "peak_replicas": 3, + "requests": 3892, + "cooldown_replica_seconds": 900.0 + }, + "payment": { + "replica_seconds": 90.0, + "peak_replicas": 1, + "requests": 1406, + "cooldown_replica_seconds": 600.0 + }, + "inventory": { + "replica_seconds": 330.0, + "peak_replicas": 3, + "requests": 2586, + "cooldown_replica_seconds": 1080.0 + }, + "gateway": { + "replica_seconds": 540.0, + "peak_replicas": 3, + "requests": 4162, + "cooldown_replica_seconds": 900.0 + } + }, + "cost_usd": { + "replicas": 0.0236, + "requests": 0.0056, + "mysql": 0.0026, + "total": 0.0318 + }, + "cost_per_1k_usd": 0.0247 + }, + { + "cap": 8, + "buyers": 80, + "seconds": 240, + "region": "eastus", + "ran_at": "2026-11-19T03:43:17+00:00", + "attempts": 5235, + "completed": 5193, + "success_rate": 0.992, + "by_status": { + "200": 5193, + "503": 22, + "404": 20 + }, + "p50_ms": 2543.1, + "p95_ms": 9626.2, + "orders_per_minute": 1298.2, + "usage": { + "user": { + "replica_seconds": 240.0, + "peak_replicas": 1, + "requests": 119, + "cooldown_replica_seconds": 240.0 + }, + "catalog": { + "replica_seconds": 420.0, + "peak_replicas": 3, + "requests": 5213, + "cooldown_replica_seconds": 900.0 + }, + "order": { + "replica_seconds": 1290.0, + "peak_replicas": 8, + "requests": 10448, + "cooldown_replica_seconds": 2400.0 + }, + "payment": { + "replica_seconds": 390.0, + "peak_replicas": 3, + "requests": 5213, + "cooldown_replica_seconds": 900.0 + }, + "inventory": { + "replica_seconds": 480.0, + "peak_replicas": 4, + "requests": 5213, + "cooldown_replica_seconds": 1680.0 + }, + "gateway": { + "replica_seconds": 1290.0, + "peak_replicas": 8, + "requests": 10551, + "cooldown_replica_seconds": 2400.0 + } + }, + "cost_usd": { + "replicas": 0.05, + "requests": 0.0147, + "mysql": 0.0026, + "total": 0.0673 + }, + "cost_per_1k_usd": 0.013 + } + ] +} diff --git a/infra/autoscale_demo.py b/infra/autoscale_demo.py new file mode 100644 index 0000000..131e8b5 --- /dev/null +++ b/infra/autoscale_demo.py @@ -0,0 +1,225 @@ +#!/usr/bin/env python3 +"""make flash-sale CAP=3 (then CAP=8) — demo 15, against a provisioned Azure stack. + +Caps every service at CAP replicas, runs the same flash sale (scripts/flash_sale.py), then reads +how many replicas Azure ran and how many requests it served from Azure Monitor, and prices them. +The result goes into frontend/src/patterns/demos/flashSaleResults.json for the demo page. + +Every run starts from the same stock (2,500 free units of each product), and a buyer whose checkout +fails pays for the same 1-item cart again, so both runs sell orders of the same size. +Restocking needs inventory's demo switches, which are off in Azure: CHAOS_ENABLED=1 is set on +inventory for the run only and removed afterwards, even when the run fails. The cap goes back to 3 +afterwards, also when the run fails. It prints the most the run can cost and asks before starting. +""" +import argparse +import json +import subprocess +import sys +import time +from datetime import datetime, timedelta, timezone +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(REPO_ROOT / "scripts")) +import httpx # noqa: E402 + +import flash_sale # noqa: E402 +from resume import MAX_REPLICAS_DEFAULT # noqa: E402 +from service_table import AZURE_SERVICES # noqa: E402 + +DEFAULT_CAP = int(MAX_REPLICAS_DEFAULT) # what provision and make resume set + +STATE_FILE = REPO_ROOT / "infra" / ".state.json" +RESULTS_FILE = REPO_ROOT / "frontend" / "src" / "patterns" / "demos" / "flashSaleResults.json" + +# Every app runs 0.25 vCPU and 0.5 GiB per replica (infra/bicep/modules/container-app.bicep). +REPLICA_VCPU = 0.25 +REPLICA_GIB = 0.5 +# List prices, East US, from https://prices.azure.com/api/retail/prices (Azure Container Apps, +# Consumption; Azure Database for MySQL, B1MS compute), checked 2026-11-18. The monthly free grant +# is ignored: it would make both runs read $0. +PRICES = { + "vcpu_second": 0.000024, "idle_vcpu_second": 0.000003, "gib_second": 0.000003, + "per_million_requests": 0.40, "mysql_hour": 0.017, +} +COOLDOWN_SECONDS = 300 # Container Apps keeps extra replicas this long after the load ends +METRICS_DELAY_SECONDS = 180 # Azure Monitor shows a minute's numbers a few minutes later +SESSION_HOURS = 0.5 # MySQL bills from make provision to make teardown: about this long for one run +STOCK_PER_PRODUCT = 2500 # free stock every run starts from, so no run sells out + + +def _value(point: dict, key: str) -> float: + return point.get(key) or 0.0 # Azure Monitor sends null for a minute with no data + + +def replica_usage(metrics: dict) -> dict: + """One app's `az monitor metrics list` answer, 1-minute points, one series per revision -> + replica-seconds, peak replicas, requests. Revisions running side by side are both billed.""" + series = {m["name"]["value"]: m["timeseries"] for m in metrics["value"]} + per_minute: dict = {} + seconds = 0.0 + for revision in series.get("Replicas", []): + for i, point in enumerate(revision["data"]): + seconds += _value(point, "average") * 60 + minute = point.get("timeStamp", i) + per_minute[minute] = per_minute.get(minute, 0.0) + _value(point, "maximum") + requests = sum(_value(p, "total") for revision in series.get("Requests", []) for p in revision["data"]) + return {"replica_seconds": seconds, "peak_replicas": int(max(per_minute.values(), default=0)), + "requests": int(requests)} + + +def _active_replica_second() -> float: + return REPLICA_VCPU * PRICES["vcpu_second"] + REPLICA_GIB * PRICES["gib_second"] + + +def cost_usd(usage: dict[str, dict], mysql_seconds: float) -> dict: + """Replicas during the sale at the active rate, through the cooldown at the idle rate (they + serve nothing then), requests, and the MySQL server for the same time.""" + idle_replica_second = REPLICA_VCPU * PRICES["idle_vcpu_second"] + REPLICA_GIB * PRICES["gib_second"] + replicas = sum(u["replica_seconds"] * _active_replica_second() + + u["cooldown_replica_seconds"] * idle_replica_second for u in usage.values()) + requests = sum(u["requests"] for u in usage.values()) / 1_000_000 * PRICES["per_million_requests"] + mysql = mysql_seconds / 3600 * PRICES["mysql_hour"] + return {"replicas": replicas, "requests": requests, "mysql": mysql, "total": replicas + requests + mysql} + + +def estimate_max_usd(cap: int, seconds: float) -> float: + """The most one run can cost: every service at the cap for the sale and the cooldown, all at the + active rate, plus MySQL for the whole session.""" + return (len(AZURE_SERVICES) * cap * (seconds + COOLDOWN_SECONDS) * _active_replica_second() + + SESSION_HOURS * PRICES["mysql_hour"]) + + +def per_thousand(cost: float, completed: int) -> float | None: + return round(cost / completed * 1000, 4) if completed else None + + +def merge_run(results: dict, run: dict) -> dict: + """A rerun with the same cap replaces the old one; runs stay sorted by cap.""" + runs = [r for r in results.get("runs", []) if r["cap"] != run["cap"]] + [run] + return {**results, "runs": sorted(runs, key=lambda r: r["cap"])} + + +def _az(args: list[str]) -> str: + print(f"$ az {' '.join(args)}") + return subprocess.run(["az", *args, "--output", "json"], capture_output=True, text=True, check=True).stdout + + +def _set_cap(resource_group: str, cap: int) -> None: + for svc in AZURE_SERVICES: + _az(["containerapp", "update", "--name", svc.name, "--resource-group", resource_group, + "--max-replicas", str(cap)]) + + +def _inventory_switches(resource_group: str, on: bool) -> None: + change = ["--set-env-vars", "CHAOS_ENABLED=1"] if on else ["--remove-env-vars", "CHAOS_ENABLED"] + _az(["containerapp", "update", "--name", "inventory", "--resource-group", resource_group, *change]) + + +def _wait_until_ready(resource_group: str, timeout: float = 600) -> None: + """Every update starts a new revision. The sale starts only once every app runs its latest one.""" + deadline = time.monotonic() + timeout + for svc in AZURE_SERVICES: + while True: + app = json.loads(_az(["containerapp", "show", "--name", svc.name, "--resource-group", resource_group])) + props = app["properties"] + if props.get("latestReadyRevisionName") == props.get("latestRevisionName"): + break + if time.monotonic() > deadline: + raise SystemExit(f"{svc.name} never got its new revision ready") + time.sleep(10) + + +def _top_up_when_ready(gateway: str, timeout: float = 300) -> None: + """Restock answers 403 until inventory's new revision (with CHAOS_ENABLED=1) takes the traffic.""" + deadline = time.monotonic() + timeout + while True: + try: + flash_sale.top_up(gateway, STOCK_PER_PRODUCT) + return + except httpx.HTTPError as err: + if time.monotonic() > deadline: + raise SystemExit(f"Restock never worked: {err}") + time.sleep(10) + + +def _metrics(subscription: str, resource_group: str, app: str, start: datetime, end: datetime) -> dict: + resource = f"/subscriptions/{subscription}/resourceGroups/{resource_group}/providers/Microsoft.App/containerApps/{app}" + query = ["monitor", "metrics", "list", "--resource", resource, "--metrics", "Replicas", "Requests", + "--interval", "PT1M", "--aggregation", "Average", "Maximum", "Total", + "--start-time", start.isoformat(), "--end-time", end.isoformat()] + try: # one series per revision, so revisions running side by side are added up + return json.loads(_az([*query, "--filter", "revisionName eq '*'"])) + except subprocess.CalledProcessError: + return json.loads(_az(query)) # the metric has no revision split: Azure adds them up itself + + +def _usage(subscription: str, resource_group: str, start: datetime, end: datetime, cooldown_end: datetime) -> dict: + usage = {} + for svc in AZURE_SERVICES: + sale = replica_usage(_metrics(subscription, resource_group, svc.name, start, end)) + cooldown = replica_usage(_metrics(subscription, resource_group, svc.name, end, cooldown_end)) + usage[svc.name] = {**sale, "cooldown_replica_seconds": cooldown["replica_seconds"], + "requests": sale["requests"] + cooldown["requests"]} + return usage + + +def main() -> int: + parser = argparse.ArgumentParser(description="Demo 15: one flash sale under a replica cap.") + parser.add_argument("--cap", type=int, required=True, help="max replicas per service, e.g. 3 or 8") + parser.add_argument("--buyers", type=int, default=80, help="buyers checking out at the same time") + parser.add_argument("--seconds", type=float, default=240) + parser.add_argument("--yes", action="store_true", help="skip the cost confirmation") + args = parser.parse_args() + if not STATE_FILE.exists(): + print("No infra/.state.json: run make provision first.") + return 1 + state = json.loads(STATE_FILE.read_text()) + resource_group, subscription = state["resource_group"], state["subscription_id"] + gateway = f"https://{state['outputs']['gatewayFqdn']}" + + estimate = estimate_max_usd(args.cap, args.seconds) + print(f"Cap {args.cap}, {args.buyers} buyers for {args.seconds:.0f} s. This run costs at most ${estimate:.2f} " + f"(every service at {args.cap} replicas the whole time, plus MySQL).") + if not args.yes and input("Go ahead? [y/N] ").strip().lower() != "y": + return 1 + + try: + _set_cap(resource_group, args.cap) + _inventory_switches(resource_group, on=True) + try: + _wait_until_ready(resource_group) + _top_up_when_ready(gateway) + buyers = flash_sale.sign_up(gateway, args.buyers) # not timed, not measured + start = datetime.now(timezone.utc) + summary = flash_sale.sell(gateway, buyers, args.seconds) + end = datetime.now(timezone.utc) + finally: + _inventory_switches(resource_group, on=False) + print(json.dumps(summary, indent=2)) + + # The replicas still running through the cooldown are billed too: wait them out, then read. + cooldown_end = end + timedelta(seconds=COOLDOWN_SECONDS) + wait = (cooldown_end - datetime.now(timezone.utc)).total_seconds() + METRICS_DELAY_SECONDS + print(f"Waiting {wait / 60:.0f} min for the cooldown and for Azure Monitor to catch up...") + time.sleep(max(0.0, wait)) + usage = _usage(subscription, resource_group, start, end, cooldown_end) + finally: + if args.cap != DEFAULT_CAP: + _set_cap(resource_group, DEFAULT_CAP) # always, so a failed run never leaves the higher cap + + cost = cost_usd(usage, mysql_seconds=(cooldown_end - start).total_seconds()) + run = {"cap": args.cap, "buyers": args.buyers, "seconds": args.seconds, "region": state["location"], + "ran_at": start.isoformat(timespec="seconds"), **summary, "usage": usage, + "cost_usd": {k: round(v, 4) for k, v in cost.items()}, + "cost_per_1k_usd": per_thousand(cost["total"], summary["completed"])} + results = json.loads(RESULTS_FILE.read_text()) if RESULTS_FILE.exists() else {} + RESULTS_FILE.write_text(json.dumps(merge_run({**results, "prices": PRICES}, run), indent=2) + "\n") + per_1k = "no completed orders" if run["cost_per_1k_usd"] is None else f"${run['cost_per_1k_usd']} per 1,000 orders" + print(f"Cap {args.cap}: {summary['completed']} orders, {summary['success_rate']:.0%} succeeded, " + f"${cost['total']:.4f}, {per_1k}. Saved to {RESULTS_FILE.relative_to(REPO_ROOT)}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/flash_sale.py b/scripts/flash_sale.py new file mode 100644 index 0000000..4871617 --- /dev/null +++ b/scripts/flash_sale.py @@ -0,0 +1,171 @@ +"""Demo 15: a flash sale. Many signed-in buyers each add 1 item and check out, again and again, +for a fixed time. Works against the local gateway and the Azure one: + + .venv/bin/python scripts/flash_sale.py --gateway http://127.0.0.1:8100 --buyers 5 --seconds 10 --stock 200 + +Prints a JSON summary. --stock needs CHAOS_ENABLED=1 on inventory (on locally under make run). +""" +import argparse +import json +import math +import threading +import time +import uuid +from collections import Counter +from concurrent.futures import ThreadPoolExecutor +from dataclasses import dataclass + +import httpx + +PRODUCTS = range(1, 25) # the catalog's 24 sample products +TIMEOUT = httpx.Timeout(30.0) +RESTOCK_MAX = 1000 # inventory's restock accepts at most this many per call +PENDING_WAIT_SECONDS = 6 # the reconciler settles pending payments every 5 s +SIGN_UP_TRIES = 6 +SIGN_UP_RETRY_SECONDS = 3 + + +@dataclass(frozen=True) +class Attempt: + status: int | None # None: no answer (timeout or refused connection) + ms: float + + +def _percentile(values: list[float], share: float) -> float: + ordered = sorted(values) + return ordered[max(0, math.ceil(share * len(ordered)) - 1)] + + +def summarize(attempts: list[Attempt], seconds: float) -> dict: + """Only a 200 is a completed order: 202 is still pending, everything else is a lost sale.""" + completed = sum(1 for a in attempts if a.status == 200) + times = [a.ms for a in attempts] + return { + "attempts": len(attempts), + "completed": completed, + "success_rate": round(completed / len(attempts), 4) if attempts else 0.0, + "by_status": dict(Counter("no answer" if a.status is None else str(a.status) for a in attempts)), + "p50_ms": round(_percentile(times, 0.50), 1) if times else 0.0, + "p95_ms": round(_percentile(times, 0.95), 1) if times else 0.0, + "orders_per_minute": round(completed / seconds * 60, 1) if seconds else 0.0, + } + + +def top_up_amounts(stock: list[dict], target: int) -> dict[int, list[int]]: + """Restock calls that bring each product's free stock to `target` (the API takes 1,000 at most).""" + plan = {} + for row in stock: + missing = target - (row["on_hand"] - row["reserved"]) + if missing > 0: + plan[row["product_id"]] = [RESTOCK_MAX] * (missing // RESTOCK_MAX) + ([missing % RESTOCK_MAX] if missing % RESTOCK_MAX else []) + return plan + + +def top_up(gateway: str, target: int) -> None: + """Every run starts from the same stock, so one run's sales never decide the next run's result.""" + shop = _shop(gateway) + stock = httpx.get(f"{shop}/inventory", timeout=TIMEOUT).raise_for_status().json()["results"] + for product_id, amounts in top_up_amounts([r for r in stock if r["product_id"] in PRODUCTS], target).items(): + for quantity in amounts: + httpx.post(f"{shop}/inventory/{product_id}/restock", json={"quantity": quantity}, + timeout=TIMEOUT).raise_for_status() + + +def _shop(gateway: str) -> str: + return f"{gateway.rstrip('/')}/api/shop" + + +def sign_up_one(client: httpx.Client) -> dict: + """Creates and logs in one buyer. Retried: a busy user service (password hashing is slow on a + 0.25 vCPU replica) may answer 503 while it scales up, and signing up is not part of the sale.""" + tag = uuid.uuid4().hex[:10] + user = {"username": f"fs{tag}", "password": "flashsale1"} + for attempt in range(SIGN_UP_TRIES): + try: + client.post("/user/create", data={**user, "email": f"fs{tag}@example.com"}) + r = client.post("/user/login", data=user) + if r.status_code == 200: + return {"Authorization": f"Bearer {r.json()['token']}"} + except httpx.HTTPError: + pass + time.sleep(SIGN_UP_RETRY_SECONDS * (attempt + 1)) + raise SystemExit(f"Could not sign up a buyer after {SIGN_UP_TRIES} tries") + + +def _cart_has_items(client: httpx.Client) -> bool: + try: + return bool(client.get("/order").json().get("result", {}).get("items")) + except (httpx.HTTPError, ValueError): + return True # unknown: retrying the checkout is safe, adding an item is not + + +def buy(client: httpx.Client, first_product: int, until: float, out: list[Attempt]) -> None: + """One buyer: 1 item per order. A failed checkout keeps the cart, so the buyer pays for the same + cart again instead of adding a second item (which would make that order bigger).""" + product, cart_full = first_product, False + while time.monotonic() < until: + started = time.perf_counter() + try: + status = 200 if cart_full else client.post("/order/add-item", data={"product_id": product, "qty": 1}).status_code + added = status == 200 + if added: + status = client.post("/order/checkout").status_code + except httpx.HTTPError: + status, added = None, False + out.append(Attempt(status, (time.perf_counter() - started) * 1000)) # list.append is thread-safe + if status == 200: + cart_full, product = False, product % len(PRODUCTS) + 1 + elif status == 404 and added: # no open order: an earlier checkout went through after all + cart_full = False + elif status == 202: + time.sleep(PENDING_WAIT_SECONDS) + cart_full = _cart_has_items(client) + elif status is None or not added: + cart_full = _cart_has_items(client) + else: + cart_full = True + if status == 409: + time.sleep(1) # sold out, or still pending: do not hammer the order service + + +def sign_up(gateway: str, buyers: int) -> list[dict]: + with httpx.Client(base_url=_shop(gateway), timeout=TIMEOUT) as client, ThreadPoolExecutor(max_workers=4) as pool: + return list(pool.map(lambda _: sign_up_one(client), range(buyers))) + + +def sell(gateway: str, buyers: list[dict], seconds: float) -> dict: + """Every buyer buys at once for `seconds`; returns the summary.""" + attempts: list[Attempt] = [] + until = time.monotonic() + seconds + clients = [httpx.Client(base_url=_shop(gateway), headers=h, timeout=TIMEOUT) for h in buyers] + threads = [threading.Thread(target=buy, args=(c, i % len(PRODUCTS) + 1, until, attempts)) + for i, c in enumerate(clients)] + for t in threads: + t.start() + for t in threads: + t.join() + for c in clients: + c.close() + return summarize(attempts, seconds) + + +def run(gateway: str, buyers: int, seconds: float) -> dict: + """Signs the buyers up first (not timed), then lets them all buy at once.""" + return sell(gateway, sign_up(gateway, buyers), seconds) + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument("--gateway", default="http://127.0.0.1:8100") + parser.add_argument("--buyers", type=int, default=5) + parser.add_argument("--seconds", type=float, default=10) + parser.add_argument("--stock", type=int, default=0, help="first top up each product's free stock to this") + args = parser.parse_args() + if args.stock: + top_up(args.gateway, args.stock) + print(json.dumps(run(args.gateway, args.buyers, args.seconds), indent=2)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/smoke/test_post_run.py b/tests/smoke/test_post_run.py index 97997e7..1f1074e 100644 --- a/tests/smoke/test_post_run.py +++ b/tests/smoke/test_post_run.py @@ -337,3 +337,15 @@ def test_every_process_reports_how_it_runs(): processes = httpx.get(f"{GATEWAY_URL}/api/patterns/runtime").json()["processes"] assert len(processes) == 7 # gateway + 5 services, order twice assert all("uid" in p for p in processes) + + +# --- Demo 15: the flash sale script, locally (the replica caps only exist in Azure) --------------- + +def test_demo_15_the_flash_sale_completes_orders_through_the_gateway(): + import flash_sale + + _require_stack() + flash_sale.top_up(GATEWAY_URL, 50) # a sold-out product answers 409 and stays in the cart + summary = flash_sale.run(GATEWAY_URL, buyers=3, seconds=5) + assert summary["completed"] >= 3 + assert summary["success_rate"] >= 0.9, summary["by_status"] diff --git a/tests/unit/test_autoscale_demo.py b/tests/unit/test_autoscale_demo.py new file mode 100644 index 0000000..39ab938 --- /dev/null +++ b/tests/unit/test_autoscale_demo.py @@ -0,0 +1,121 @@ +"""Demo 15: the same flash sale under a replica cap of 3 and of 8. These cover the numbers the page +shows: success rate, replica-seconds read from Azure Monitor, and cost per 1,000 completed orders.""" +import time + +import httpx + +import autoscale_demo +import flash_sale +from flash_sale import Attempt + + +def test_summary_counts_only_200s_as_completed_and_reports_latency(): + attempts = [Attempt(200, 100.0)] * 6 + [Attempt(503, 50.0)] * 2 + [Attempt(202, 900.0), Attempt(None, 3000.0)] + summary = flash_sale.summarize(attempts, seconds=60) + assert summary["attempts"] == 10 + assert summary["completed"] == 6 + assert summary["success_rate"] == 0.6 + assert summary["by_status"] == {"200": 6, "503": 2, "202": 1, "no answer": 1} + assert summary["p50_ms"] == 100.0 + assert summary["p95_ms"] == 3000.0 + assert summary["orders_per_minute"] == 6.0 + + +def test_summary_of_no_attempts_is_zero_not_a_crash(): + assert flash_sale.summarize([], seconds=60)["success_rate"] == 0.0 + + +def _metric(name, points): + return {"name": {"value": name}, "timeseries": [{"data": points}]} + + +def test_replica_usage_turns_one_minute_averages_into_seconds(): + metrics = {"value": [ + _metric("Replicas", [{"average": 1.0, "maximum": 1.0}, {"average": 2.5, "maximum": 3.0}, {}]), + _metric("Requests", [{"total": 400.0}, {"total": 600.0}]), + ]} + assert autoscale_demo.replica_usage(metrics) == {"replica_seconds": 210.0, "peak_replicas": 3, "requests": 1000} + + +def test_replica_usage_adds_up_revisions_and_survives_empty_minutes(): + """A cap change starts a new revision: both run side by side for a while and both are billed.""" + old = {"data": [{"timeStamp": "t1", "average": 2.0, "maximum": 2.0}, {"timeStamp": "t2", "average": None}]} + new = {"data": [{"timeStamp": "t1", "average": 1.0, "maximum": 2.0}, {"timeStamp": "t2", "average": 3.0, "maximum": 3.0}]} + metrics = {"value": [{"name": {"value": "Replicas"}, "timeseries": [old, new]}, + {"name": {"value": "Requests"}, "timeseries": []}]} + assert autoscale_demo.replica_usage(metrics) == {"replica_seconds": 360.0, "peak_replicas": 4, "requests": 0} + + +def test_cost_prices_the_sale_active_the_cooldown_idle_and_adds_mysql(): + usage = {"gateway": {"replica_seconds": 1000.0, "cooldown_replica_seconds": 1000.0, "peak_replicas": 3, + "requests": 1_000_000}, + "order": {"replica_seconds": 1000.0, "cooldown_replica_seconds": 0.0, "peak_replicas": 3, "requests": 0}} + cost = autoscale_demo.cost_usd(usage, mysql_seconds=3600) + # sale: 2000 s x (0.25 x $0.000024 + 0.5 x $0.000003) = $0.015 + # cooldown: 1000 s x (0.25 x $0.000003 + 0.5 x $0.000003) = $0.00225 + # requests: 1M x $0.40/M = $0.40; MySQL B1ms: 1 hour x $0.017 + assert {k: round(v, 6) for k, v in cost.items()} == { + "replicas": 0.01725, "requests": 0.4, "mysql": 0.017, "total": 0.43425} + + +def test_the_estimate_before_a_run_assumes_every_service_sits_at_the_cap(): + # 6 apps x 8 replicas x (240 s + 300 s cooldown) at the active rate, plus MySQL for 30 minutes + assert round(autoscale_demo.estimate_max_usd(cap=8, seconds=240), 4) == round( + 6 * 8 * 540 * 0.0000075 + 0.5 * 0.017, 4) + + +def test_top_up_brings_every_product_to_the_same_level_in_steps_the_api_accepts(): + rows = [{"product_id": 1, "on_hand": 30, "reserved": 5}, {"product_id": 2, "on_hand": 2600, "reserved": 0}, + {"product_id": 3, "on_hand": 100, "reserved": 0}] + assert flash_sale.top_up_amounts(rows, target=2500) == {1: [1000, 1000, 475], 3: [1000, 1000, 400]} + + +def test_cost_per_thousand_orders(): + assert autoscale_demo.per_thousand(0.5, completed=2000) == 0.25 + assert autoscale_demo.per_thousand(0.5, completed=0) is None + + +def test_a_rerun_replaces_the_run_with_the_same_cap(): + results = {"runs": [{"cap": 8, "completed": 1}, {"cap": 3, "completed": 1}]} + merged = autoscale_demo.merge_run(results, {"cap": 3, "completed": 2}) + assert merged["runs"] == [{"cap": 3, "completed": 2}, {"cap": 8, "completed": 1}] + + +def _shop(checkout_statuses: list[int], calls: list[str]) -> httpx.Client: + def handler(request: httpx.Request) -> httpx.Response: + calls.append(request.url.path) + if request.url.path.endswith("/checkout"): + return httpx.Response(checkout_statuses.pop(0) if checkout_statuses else 200) + return httpx.Response(200, json={"result": {"items": []}}) + return httpx.Client(transport=httpx.MockTransport(handler), base_url="http://shop") + + +def test_a_buyer_retries_the_same_cart_after_a_failed_checkout(): + """A failed checkout keeps the cart: adding another item would make the next order bigger.""" + calls: list[str] = [] + out: list[Attempt] = [] + flash_sale.buy(_shop([503, 503, 200], calls), first_product=1, until=time.monotonic() + 0.1, out=out) + assert calls[:5] == ["/order/add-item", "/order/checkout", "/order/checkout", "/order/checkout", "/order/add-item"] + assert [a.status for a in out[:3]] == [503, 503, 200] + + +def test_sign_up_waits_out_a_busy_user_service(monkeypatch): + """Signing up is not timed: a 503 from a user service still scaling up is retried, not fatal.""" + logins = [503, 503, 200] + + def handler(request: httpx.Request) -> httpx.Response: + if request.url.path.endswith("/login"): + status = logins.pop(0) + return httpx.Response(status, json={"token": "t"} if status == 200 else {}) + return httpx.Response(201) + + monkeypatch.setattr(flash_sale, "SIGN_UP_RETRY_SECONDS", 0) + with httpx.Client(transport=httpx.MockTransport(handler), base_url="http://shop") as client: + assert flash_sale.sign_up_one(client) == {"Authorization": "Bearer t"} + + +def test_a_buyer_whose_order_went_through_after_all_starts_a_new_cart(): + """A checkout can time out at the gateway yet complete: the retry then finds no open order (404).""" + calls: list[str] = [] + flash_sale.buy(_shop([503, 404, 200], calls), first_product=1, until=time.monotonic() + 0.1, out=[]) + assert calls[:5] == ["/order/add-item", "/order/checkout", "/order/checkout", "/order/add-item", "/order/checkout"]