From 80972bd540a7f6d78cf153bdf34a8fcc8f1a47fd Mon Sep 17 00:00:00 2001 From: Marcos Oliveira Date: Sun, 15 Nov 2026 20:57:48 -0500 Subject: [PATCH] =?UTF-8?q?docs:=20final=20README=20=E2=80=94=20project=20?= =?UTF-8?q?summary,=20patterns=201-12=20in=20order=20with=20their=20screen?= =?UTF-8?q?shots?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - New project summary: what was built, milestones with their PRs, how it was checked, cost. - The pattern table links to each demo; demos now run 1 to 12 (they were in build order). - Azure screenshots sit inside the pattern they prove (3, 7, 9). - Stale lines fixed: 'Still to come (M3b)' removed; contract testing names all 3 providers. Co-Authored-By: Claude Opus 5.5 --- README.md | 226 +++++++++++++++++++++++++++++++++--------------------- 1 file changed, 140 insertions(+), 86 deletions(-) diff --git a/README.md b/README.md index 584e9b6..31c9a1d 100644 --- a/README.md +++ b/README.md @@ -5,29 +5,62 @@ Each pattern gets its own page with an explanation, when *not* to use it, and a can click. > **Status: complete.** All 12 patterns are live, each with a working demo, locally and in one -> real Azure run (issue #11): provision, Deploy through GitHub Actions with OIDC, smoke tests, -> screenshots, teardown. The foundation is copied from +> real Azure run: provision, deploy through GitHub Actions with OIDC, smoke tests, screenshots, +> teardown. The foundation is copied from > [Azure-Ecommerce-Monolith-Microservices-Migration](https://github.com/MAOFILHO/Azure-Ecommerce-Monolith-Microservices-Migration). -> See [docs/PLAN.md](docs/PLAN.md) and the [M3b issues](https://github.com/MAOFILHO/Microservices-12-Patterns/issues). + +## Project summary + +**What was built.** 5 Flask services (user, catalog, order, payment, inventory), each owning its +own database, behind one FastAPI gateway; a small service registry; Redis as the event bus; and a +React frontend with one page per pattern. Every demo calls the real services: nothing is mocked. + +**How it was delivered.** In milestones, one reviewable pull request each, following +[docs/PLAN.md](docs/PLAN.md): + +| Milestone | Patterns | Pull requests | +|---|---|---| +| M0, M1a: bootstrap and layout | (foundation) | [#1](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/1) | +| M1: foundation | 1, 2, 5, 6 | [#2](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/2) | +| M2: resilience and independence | 4, 10, 11, 12 | [#3](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/3) | +| M3a: async and packaging | 7, 8, 9 | [#4](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/4) | +| M3b: delivery and the Azure run | 3 | [#12](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/12), [#13](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/13), [#14](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/14) | + +M3b was planned as 7 GitHub issues ([#5 to #11](https://github.com/MAOFILHO/Microservices-12-Patterns/issues?q=is%3Aissue)), +all closed. + +**How it was checked.** +- **155 unit tests** across 8 suites, each in its own virtualenv, built test-first from M3a on. +- **Pact contract tests**: the order service's contracts with payment, catalog and inventory, + verified against the real providers in CI. +- **23 local smoke tests** against the running stack, and **4 Azure smoke tests** against the real + deployment. +- **CI on every pull request** runs only what the change touches; every image is built for + linux/amd64, started, and rejected if it runs as root. `main` requires the "CI ok" check. +- A two-axis code review (coding standards, and the spec) before every milestone PR. + +**What it cost.** Locally: $0, no Docker. In Azure: one session of about 1.5 hours, on free tiers +or the cheapest SKUs, with a $25/month budget alert, then torn down and verified empty. See +[cost_estimate.md](cost_estimate.md). ## The 12 patterns -| # | Pattern | What it does | Status | +| # | Pattern | What it does | Milestone | |---|---|---|---| -| 1 | Database per Service | Gives each service its own database | **Live (M1)** | -| 2 | Bounded Context | Splits one large domain into clear business domains | **Live (M1)** | -| 3 | CI-CD Pipeline | Builds, tests and deploys every service automatically | **Live (M3b)** | -| 4 | Circuit Breaker | Stops calls to a failing service to prevent more failures | **Live (M2)** | -| 5 | API Gateway | Routes client requests to the right service | **Live (M1)** | -| 6 | Observability | Collects logs, metrics and traces from every service | **Live (M1)** | -| 7 | Event-Driven Architecture | Services communicate by publishing and consuming events | **Live (M3a)** | -| 8 | Saga Pattern | Coordinates distributed transactions and rolls back failures safely | **Live (M3a)** | -| 9 | Containerization | Packages each service with all its dependencies | **Live (M3a)** | -| 10 | Stateless Services | Stores session data outside the service | **Live (M2)** | -| 11 | Contract Testing | Checks that APIs stay compatible across services | **Live (M2)** | -| 12 | Service Discovery | Finds service instances automatically at runtime | **Live (M2)** | - -## Architecture today +| 1 | [Database per Service](#1-database-per-service) | Gives each service its own database | M1 | +| 2 | [Bounded Context](#2-bounded-context) | Splits one large domain into clear business domains | M1 | +| 3 | [CI-CD Pipeline](#3-ci-cd-pipeline) | Builds, tests and deploys every service automatically | M3b | +| 4 | [Circuit Breaker](#4-circuit-breaker) | Stops calls to a failing service to prevent more failures | M2 | +| 5 | [API Gateway](#5-api-gateway) | Routes client requests to the right service | M1 | +| 6 | [Observability](#6-observability) | Collects logs, metrics and traces from every service | M1 | +| 7 | [Event-Driven Architecture](#7-event-driven-architecture) | Services communicate by publishing and consuming events | M3a | +| 8 | [Saga Pattern](#8-saga-pattern) | Coordinates distributed transactions and rolls back failures safely | M3a | +| 9 | [Containerization](#9-containerization) | Packages each service with all its dependencies | M3a | +| 10 | [Stateless Services](#10-stateless-services) | Stores session data outside the service | M2 | +| 11 | [Contract Testing](#11-contract-testing) | Checks that APIs stay compatible across services | M2 | +| 12 | [Service Discovery](#12-service-discovery) | Finds service instances automatically at runtime | M2 | + +## Architecture ``` Browser → API gateway ──┬──► user → user_db (signs login tokens) @@ -46,15 +79,17 @@ Every service heartbeats into the registry (:8500); callers look each other up t - **FastAPI gateway** (`gateway/`): the only thing the browser talks to. Routes `/api/shop/*` to the owning service, adds request IDs, aggregates `/api/order-summary`, serves the demo endpoints. - **`shop_common`** (`shared/`): OpenTelemetry tracing, JSON logs with trace IDs, Prometheus - metrics, the `/health/db`, `/debug/spans`, `/metrics` endpoints, token checks and the discovery client. + metrics, the `/health/db`, `/debug/spans`, `/metrics` endpoints, token checks, the discovery + client and the event bus helpers. - **Service registry** (`registry/`): about 90 lines of FastAPI. Local only; Azure uses Container Apps DNS. -- **Contracts** (`contracts/pacts/`): the Pact files the order service needs payment, catalog and inventory to honour. -- **Redis** (Homebrew `redis-server`, in memory, port 6390): the event bus. `make run` starts and stops it. +- **Contracts** (`contracts/pacts/`): the Pact files the order service needs payment, catalog and + inventory to honour. +- **Redis** (Homebrew `redis-server` locally, an internal Container App in Azure): the event bus. - **Dockerfiles**: one per service plus gateway and registry, multi-stage, non-root, built only in the cloud or CI for `linux/amd64`. - **React + TypeScript** frontend (`frontend/`) with a page per pattern. - -Still to come (M3b): path-filtered CI with a manual Deploy workflow (OIDC), and a real Azure run. +- **One service table** (`scripts/service_table.py`): every process, folder and port. Setup, the + local launcher, CI, provision and the Deploy workflow all read it. ## The live demos @@ -70,6 +105,33 @@ its name and price, Inventory knows how many are on hand. Bounded Context demo: context map and one product seen by Catalog and by Inventory +### 3. CI-CD Pipeline +Which commit each running service was built from, compared with the latest commit on main, and +the latest CI and Deploy runs, read from GitHub. A service shows "behind main" only when main +changed its own code since its build, the same rule the Deploy button uses. + +- **CI on every pull request** runs only what the change touches (`scripts/ci_changes.py`, driven + by the service table): a change to one service runs its tests and its image; a change to shared + code, CI or the service table runs everything. Pushes to main run everything. +- **Deploy** is a manual workflow on main. It signs in to Azure with OIDC (no password stored), + rebuilds only services whose code changed since the commit they run, tags each image with the + commit, and waits until each service answers with the new commit. + +Locally, every process is a "local build": + +CI-CD demo locally: each service is a local build, with the latest CI run on main + +In Azure, every service runs the latest commit of main: + +CI-CD page in Azure: every service on the latest commit of main + +### 4. Circuit Breaker +Switch payment to slow and send 10 calls. Each order replica's breaker opens after 3 failures. +A real failure took 2,215 ms; a call the open breaker stopped took 0 ms. After 15 s with payment +healthy again, both breakers close. + +Circuit Breaker demo: both order replicas' breakers open; failed calls take 2.2 s, short-circuited calls 0 ms + ### 5. API Gateway One browser call to `/api/order-summary` becomes 4 service calls. The page shows each call, its status and timing, and the one request ID forwarded to all of them. @@ -82,19 +144,33 @@ database queries, drawn as a waterfall. Metrics per service come from each `/met Observability demo: a trace waterfall across gateway, user, order, catalog and inventory -### 3. CI-CD Pipeline -Which commit each running service was built from, compared with the latest commit on main, and -the latest CI and Deploy runs, read from GitHub. Locally every process is a "local build"; in Azure -each image carries its commit, so a service that is behind main shows up at once. +### 7. Event-Driven Architecture +Disconnect the bus and buy a mug: its 3 events (StockReserved, PaymentCharged, OrderConfirmed) +wait in the order service's outbox, and the mug stays held in inventory. Reconnect: the events go +out and inventory turns the hold into a sale. -CI-CD demo: each service's build commit compared with main, and the latest CI and Deploy runs +Event-Driven demo: bus disconnected, 3 events waiting in the outbox, one mug held -### 4. Circuit Breaker -Switch payment to slow and send 10 calls. Each order replica's breaker opens after 3 failures. -A real failure took 2,215 ms; a call the open breaker stopped took 0 ms. After 15 s with payment -healthy again, both breakers close. +In Azure, the same 3 events were published within a second through the Redis Container App, and +inventory sold the reservation: -Circuit Breaker demo: both order replicas' breakers open; failed calls take 2.2 s, short-circuited calls 0 ms +Event-Driven page in Azure: 3 events published within a second; inventory sold the reservation + +### 8. Saga Pattern +Buy 2 mugs with "make the bank decline the card" ticked. The saga prices the cart, holds the stock, +the charge fails, and it releases the stock. Stock ends exactly where it started; the cart is kept. + +Saga demo: a declined card, with the stock hold released by the compensation step + +### 9. Containerization +How each process runs: uid, container or not, platform, build and image size. Locally they are +plain processes; CI builds and runs every image for linux/amd64 and fails any that runs as root. + +Containerization demo locally: every process, its uid and platform, and the Dockerfile shape + +In Azure, all 6 run in containers as uid 10001 on linux/x86_64, built from one commit, 67-69 MB each: + +Containerization page in Azure: 6 containers, uid 10001, linux/x86_64, built from one commit, 67-69 MB ### 10. Stateless Services Your login is a signed token, readable by anyone, signed only by the user service. Ask for your @@ -104,8 +180,8 @@ logged in. Stateless Services demo: one order replica stopped, all 6 cart requests still succeed on the other ### 11. Contract Testing -The order service's contracts with payment and catalog, read from the committed Pact files, and -the real CI output when payment renames one field the order service relies on. +The order service's contracts with payment, catalog and inventory, read from the committed Pact +files, and the real CI output when payment renames one field the order service relies on. Contract Testing demo: the contract break output and how CI runs consumer and provider checks @@ -115,75 +191,53 @@ The live registry. Stop an instance: its heartbeat stops, the registry expires i Service Discovery demo: registry table and an expired then re-registered order replica -### 7. Event-Driven Architecture -Disconnect the bus and buy a mug: OrderConfirmed waits in the order service's outbox, and the mug -stays held in inventory. Reconnect: the event goes out and inventory turns the hold into a sale. - -Event-Driven demo: bus disconnected, one OrderConfirmed waiting in the outbox, one mug held - -### 8. Saga -Buy 2 mugs with "make the bank decline the card" ticked. The saga prices the cart, holds the stock, -the charge fails, and it releases the stock. Stock ends exactly where it started; the cart is kept. - -Saga demo: a declined card, with the stock hold released by the compensation step - -### 9. Containerization -How each process runs (uid, container or not, platform, build). Locally they are plain processes; -CI builds and runs every image for linux/amd64 and fails any that runs as root. - -Containerization demo: every process, its uid and platform, and the Dockerfile shape - ## Run locally (no Docker) ```bash brew install redis # once: the event bus (patterns 7 and 8) -make setup # one venv per service + gateway, shop_common, frontend packages -make run # frontend :5273, gateway :8100, registry :8500, Redis :6390 -make test # unit tests, each suite in its own venv -make contracts # Pact: order writes its contracts, payment and catalog verify them -make smoke # end-to-end checks against the running stack +make setup # one venv per service + gateway, shop_common, frontend packages +make run # frontend :5273, gateway :8100, registry :8500, Redis :6390 +make test # unit tests, each suite in its own venv +make contracts # Pact: order writes its contracts; payment, catalog and inventory verify them +make smoke # end-to-end checks against the running stack ``` Optional: set `OTEL_EXPORTER_OTLP_ENDPOINT` to also send spans to an OpenTelemetry viewer or collector. Without it, traces stay in each process's memory and the Observability page draws them. -## Cloud +## Run in Azure Azure Container Apps (one app per service plus the gateway, all scale-to-zero; Redis as an -always-on internal bus) + MySQL Flexible Server. About $0.21 for an 8-hour session; see -[cost_estimate.md](cost_estimate.md). In Azure, discovery uses Container Apps DNS (no registry), and -the demo switches stay off. Redis is reachable only from apps inside the environment and has no -password, which is fine for a demo where every app is ours. +always-on internal bus) + MySQL Flexible Server, on free tiers or the cheapest SKUs. About $0.21 +for an 8-hour session; see [cost_estimate.md](cost_estimate.md). In Azure, discovery uses Container +Apps DNS (no registry), and the demo switches stay off. Redis is reachable only from apps inside the +environment and has no password, which is fine for a demo where every app is ours. + +```bash +make github-identity # once: the Azure identity the Deploy workflow uses (OIDC, no secret) +make provision BUDGET_EMAIL=you@example.com # everything, images tagged with the current commit +# then GitHub → Actions → Deploy → Run workflow: rebuilds only the services that changed +make teardown && make verify # remove everything, then prove nothing is left +``` -### The Azure run (#11) +### The Azure run `make provision` built 6 images in the cloud (67-69 MB each, tagged with the commit), deployed them plus Redis, and created the $25 budget alert. The Azure smoke tests (`tests/smoke/test_post_provision_azure.py`) passed 4 of 4: a sale settled through Redis, demo switches answered 403, every container ran as uid 10001 on the deployed commit, and the CI-CD page saw every service on main. The Deploy button then signed in with OIDC and correctly rebuilt -nothing. Everything was removed with `make teardown` the same evening. - -Containerization page in Azure: 6 containers, uid 10001, linux/x86_64, built from one commit, 67-69 MB - -CI-CD page in Azure: every service on the latest commit of main - -Event-Driven page in Azure: 3 events published within a second; inventory sold the reservation +nothing. Everything was removed with `make teardown` the same evening, and `make verify` passed. +The screenshots from Azure are in patterns [3](#3-ci-cd-pipeline), [7](#7-event-driven-architecture) +and [9](#9-containerization) above. Details: [issue #11](https://github.com/MAOFILHO/Microservices-12-Patterns/issues/11). Lessons from the run: GitHub now presents an OIDC subject with account and repo ids, so the -identity trusts both forms (PR #14); `make verify` can time out while Azure removes an empty -Container Apps environment, which takes longer than everything else and bills nothing. +identity trusts both forms ([PR #14](https://github.com/MAOFILHO/Microservices-12-Patterns/pull/14)); +`make verify` can time out while Azure removes an empty Container Apps environment, which takes +longer than everything else and bills nothing. -```bash -make github-identity # once: the Azure identity the Deploy workflow uses (OIDC, no secret) -make provision BUDGET_EMAIL=you@example.com # everything, images tagged with the current commit -# then GitHub → Actions → Deploy → Run workflow: rebuilds only the services that changed -make teardown && make verify # remove everything, then prove nothing is left -``` +## Project docs -### CI-CD (pattern 3) -- **CI on every pull request** runs only what the change touches (`scripts/ci_changes.py`, driven - by the service table): a change to one service runs its tests and its image; a change to shared - code, CI or the service table runs everything. Pushes to main run everything. -- **Deploy** is a manual workflow on main. It signs in to Azure with OIDC (no password stored), - rebuilds only services whose code changed since the commit they run, tags each image with the - commit, and records each image's size. +- [docs/PLAN.md](docs/PLAN.md): the agreed plan, the M3 review decisions and the code review outcomes. +- [cost_estimate.md](cost_estimate.md): every Azure resource, its SKU and its cost. +- [CLAUDE.md](CLAUDE.md): the project's working notes, ground rules and gotchas. +- [Issues](https://github.com/MAOFILHO/Microservices-12-Patterns/issues?q=is%3Aissue): the M3b tickets, all closed.