From f8874b0a834fbb42ce448239c4075adfb4f376a6 Mon Sep 17 00:00:00 2001 From: Ernest Provo Date: Fri, 17 Jul 2026 05:09:49 -0400 Subject: [PATCH] docs: sync v1.1 transport support --- CHANGELOG.md | 17 +++++++++++------ DOCUMENTATION_INDEX.md | 3 ++- README.md | 26 ++++++++++++++++---------- SYSTEM_CONTEXT_DIAGRAM.md | 38 +++++++++++++++++++++++--------------- 4 files changed, 52 insertions(+), 32 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 43bb58e..f49622c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,15 +13,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## Scope (v1) -mcp-warden v1 verifies the **declared surface** of **stdio-transport** MCP servers — -the `(name, description, inputSchema)` metadata returned by `tools/list`, -`resources/list`, and `prompts/list` — pinned into a signed `warden.lock` and gated in -CI. The v0.3 `guard` proxy adds deterministic runtime *result* inspection -(ANSI/control escapes, echoed secrets, exfil domains) with default-block. +mcp-warden v1 verifies an MCP server's **declared surface** — the `(name, description, +inputSchema)` metadata returned by `tools/list`, `resources/list`, and `prompts/list` — +pinned into a signed `warden.lock` and gated in CI. `pin` and `check` support stdio and +Streamable HTTP; the v0.3 `guard` proxy adds deterministic runtime *result* inspection +(ANSI/control escapes, echoed secrets, exfil domains) with default-block for stdio. **Explicitly out of scope in v1 (documented post-1.0 roadmap):** -- **HTTP/SSE transport** — v1 is stdio-only; HTTP/SSE is the headline v1.x item (#9). +- **Runtime proxying over HTTP/SSE** — `guard` remains stdio-only; HTTP/SSE support in + v1.1 applies to definition capture by `pin` and `check` (#74). - **Prompt-injection default-block** — stays opt-in / MONITOR until field false-positive data justifies blocking by default. - Behavioral-attack defense (`T-BEHAVE`), full agent-firewall mediation, and any @@ -33,6 +34,10 @@ CI. The v0.3 `guard` proxy adds deterministic runtime *result* inspection ### Added +- **Streamable HTTP definition capture (DSE-57, #74).** `pin` and `check` now accept + `--url ` as a mutually exclusive alternative to a stdio server command, + connecting to an already-running MCP server over the SDK's Streamable HTTP transport. + Runtime `guard` proxying remains stdio-only. - **Injection-phrase FP-instrumentation (Refs #12).** Shippable, non-default-changing groundwork for eventually promoting `WRD-RES-INJECT-PHRASE` to default-block once field false-positive (FP) data justifies it. **No default posture changed** — the fuzzy tier diff --git a/DOCUMENTATION_INDEX.md b/DOCUMENTATION_INDEX.md index a5c6a27..6402ca1 100644 --- a/DOCUMENTATION_INDEX.md +++ b/DOCUMENTATION_INDEX.md @@ -124,7 +124,7 @@ scope-honesty box and makes no compliance/regulatory claim. |--------|----------------|-------------| | `src/mcp_warden/hashing.py` | `canon()` (RFC 8785) + `hash()` + field hashes | WARDEN_LOCK_SCHEMA §3 | | `src/mcp_warden/tokenizer.py` | Shared tokenizer + capability derivation (single source of truth) | CHECKS §3 / WARDEN_LOCK_SCHEMA §5.4 | -| `src/mcp_warden/capture.py` | MCP stdio capture client (argv array, no shell; timeouts/errors) | THREAT_MODEL §3.3 / WARDEN_LOCK_SCHEMA §4.1 | +| `src/mcp_warden/capture.py` | Declared-surface capture over stdio (argv array, no shell) or Streamable HTTP (`--url`); shared list normalization, timeouts, and errors | THREAT_MODEL §3.3 / WARDEN_LOCK_SCHEMA §4.1 | | `src/mcp_warden/models.py` | Pydantic models for captured surface + lock (incl. `Pinner`/`Attestation` provenance) | WARDEN_LOCK_SCHEMA §2–§8 | | `src/mcp_warden/lockfile.py` | Lock builder + reader/writer + overall digest | WARDEN_LOCK_SCHEMA §5–§6, §9 | | `src/mcp_warden/check_core.py` | **(#22)** Shared check verdict core (`run_check` / `run_check_full`): read_lock→capture→checks→build_lock(in-memory)→compute_drift. Single source of truth for `cli.py:check` AND the pre-commit wrapper | WARDEN_LOCK_SCHEMA §6.2 | @@ -170,6 +170,7 @@ scope-honesty box and makes no compliance/regulatory claim. | `tests/test_policy.py` | Lint (incl. unknown-key error) + eval (allow/deny/SSRF/fail-closed) | | `tests/test_emitters.py` | SARIF shape + level mapping + JSONL records | | `tests/test_e2e_pin_check.py` | **Headline:** real stdio pin→mutate→check round-trip | +| `tests/test_capture_http.py` | **(#74, DSE-57)** Async/sync Streamable HTTP capture, protocol/list normalization, timeout handling, and connection errors | | `tests/test_diff.py` | **(v0.3)** `warden diff` renderer: identical→"no differences", tool add/remove + schema change rows, **redaction-leak guard** (secret in `server.args` absent from human/`--json`/`--sarif` incl. parsed-JSONL `detail`), provenance-only section vs empty integrity drift, `--exit-code` (1 on integrity drift / 0 on provenance-only), `--no-provenance` M6 message, fail-closed on missing/invalid lock | | `tests/test_result_inspection.py` | **(v0.2)** `WRD-RES-*`: ANSI codepoint match (incl. extended/binary-ok), secret-echo reuse + redaction, exfil host/subdomain boundary + path-qualified, injection exact-phrase (no broad-regex FP), URL/uninspectable notes | | `tests/test_inspection_policy.py` | **(v0.2)** §11 per-tool policy fail-safe defaults, byte-identical-to-v0.1 digest when absent, inspection-policy drift, pin-time validation, reader fallback + LOCK-INVALID | diff --git a/README.md b/README.md index 60d8336..7026640 100644 --- a/README.md +++ b/README.md @@ -6,10 +6,9 @@ [![GitHub Action](https://img.shields.io/badge/GitHub%20Action-mcp--warden-2088FF?logo=githubactions&logoColor=white)](https://github.com/DataScience-EngineeringExperts/mcp-warden/blob/main/action.yml) [![Latest release](https://img.shields.io/github/v/release/DataScience-EngineeringExperts/mcp-warden?display_name=tag&sort=semver)](https://github.com/DataScience-EngineeringExperts/mcp-warden/releases) -**mcp-warden is the lockfile and CI gate for stdio-transport MCP servers: it pins -an MCP server's declared tool/resource/prompt surface into a signed `warden.lock`, -then fails CI when that surface drifts from the approved baseline.** v1 covers -**stdio-transport** servers; HTTP/SSE transport is a documented v1.x roadmap item. +**mcp-warden is the lockfile and CI gate for MCP servers: it pins a server's declared +tool/resource/prompt surface into a signed `warden.lock`, then fails CI when that surface +drifts.** `pin` and `check` support stdio and Streamable HTTP; `guard` is stdio-only. > ⚠️ **Install `mcp-warden-cli`, not `mcp-warden`.** The PyPI name `mcp-warden` is > an **unrelated package by a different author** — it is not this project. The @@ -65,8 +64,14 @@ uv pip install --python .venv/bin/python -e ".[dev]" .venv/bin/mcp-warden check python tests/fixtures/mutated_server.py --lock warden.lock ``` -Then wire it into CI with the official GitHub Action (point `server-cmd` at *your* -server's launch argv, commit `warden.lock`): +For an already-running Streamable HTTP server, use `--url` instead of a server command: + +```bash +.venv/bin/mcp-warden pin --url https://example.com/mcp --approve --approver you@example.com --lock warden.lock +.venv/bin/mcp-warden check --url https://example.com/mcp --lock warden.lock +``` + +Then wire it into CI with the official GitHub Action (point `server-cmd` at *your* server's launch argv, commit `warden.lock`): ```yaml # .github/workflows/mcp-integrity.yml @@ -377,8 +382,8 @@ run the gate only on push: | Command | Purpose | Exit code | |---------|---------|-----------| -| `mcp-warden pin [--approve --approver ] [--sign [--identity-token T]] [--sarif F] [--json]` | Capture + write `warden.lock` (TOFU baseline). **(#16)** `--sign` Sigstore-signs `overall_digest` (out-of-digest; needs `mcp-warden[sigstore]`) | 0 on success, 2 on capture/IO error, **1 on signing failure (fail closed, no partial sidecar)** | -| `mcp-warden check [--lock F] [--sarif F] [--json]` | Re-capture + diff vs lock | **non-zero on drift**, 2 on error | +| `mcp-warden pin \| --url URL [--approve --approver ] [--sign [--identity-token T]] [--sarif F] [--json]` | Capture over stdio or Streamable HTTP + write `warden.lock` (TOFU baseline). **(#16)** `--sign` Sigstore-signs `overall_digest` (out-of-digest; needs `mcp-warden[sigstore]`) | 0 on success, 2 on capture/IO error, **1 on signing failure (fail closed, no partial sidecar)** | +| `mcp-warden check \| --url URL [--lock F] [--sarif F] [--json]` | Re-capture over stdio or Streamable HTTP + diff vs lock | **non-zero on drift**, 2 on error | | `mcp-warden check --verify --certificate-identity ID --certificate-oidc-issuer ISS [--lock F] [--offline-bundle P]` | **(#16)** Verify the lock's Sigstore signature against a fixed sidecar (`.sigstore` next to the lock); no server spawn. See [`docs/SIGNING.md`](docs/SIGNING.md) | **0 only on clean verify**; non-zero on any failure (fail closed) | | `mcp-warden policy lint [--lock F]` | Lint a policy file (fail closed) | non-zero on lint error | | `mcp-warden policy eval [--lock F]` | Evaluate one sample call | **non-zero on a deny verdict** (CI assertion) | @@ -388,8 +393,9 @@ run the gate only on push: | `mcp-warden diff [--json] [--sarif F] [--no-provenance] [--exit-code]` | **(v0.3)** Offline, **redacted** viewer over the drift engine: renders integrity drift between two existing locks (A=baseline, B=current) + a separate informational provenance section. Never re-captures and never prints raw `server.command`/`args` (secret-safe) | 0 (viewer); with `--exit-code`, 1 on **integrity** drift only; 2 on missing/invalid lock | | `mcp-warden-precommit [--lock F] [--timeout N] [--strict] -- ` | **(v0.3)** pre-commit hook entry point (see [pre-commit hook](#pre-commit-hook--the-local-pre-ci-gate)). Runs the same check verdict path; check-only (never pins, never writes the lock) | 0 clean / **1 drift** / 2 config error; server-unavailable → 0+warning (non-strict) or 2 (`--strict`) | -`` is passed to the OS as an **argv array, never through a shell.** -Set `WARDEN_LOG_LEVEL=INFO` for diagnostic logging. +For stdio, `` is passed to the OS as an **argv array, never through a +shell.** `--url` instead connects to an already-running Streamable HTTP endpoint and +is mutually exclusive with a server command. Set `WARDEN_LOG_LEVEL=INFO` for diagnostics. ### Runtime result inspection (v0.3 — blocks by default) diff --git a/SYSTEM_CONTEXT_DIAGRAM.md b/SYSTEM_CONTEXT_DIAGRAM.md index ad71697..4121330 100644 --- a/SYSTEM_CONTEXT_DIAGRAM.md +++ b/SYSTEM_CONTEXT_DIAGRAM.md @@ -1,9 +1,11 @@ # mcp-warden — System Context Diagram -Where mcp-warden sits, what it talks to, and where its outputs go. The **v0.1** path -(`pin`/`check`/`policy`) is a **read-only, definition-only** gate: it spawns the target -MCP server over stdio, captures the *declared* surface, and writes a baseline + machine -reports — no proxy, no runtime interception. The **v0.2** path added a transparent stdio +Where mcp-warden sits, what it talks to, and where its outputs go. The **definition-only +path introduced in v0.1** (`pin`/`check`/`policy`) is read-only: it captures the +*declared* surface and writes a baseline + machine reports — no proxy, no runtime +interception. **As of v1.1**, `pin` and `check` either spawn a target over stdio or +connect to an already-running Streamable HTTP endpoint. The **v0.2** path added a +transparent stdio **proxy** (`guard`) and an **offline analyzer** (`inspect`) that inspect tool *results* at runtime; see C3 below. **v0.3** promotes the deterministic tier to **block by default** (opt-OUT per category via `--no-block-`; `--audit-only` restores full shadow) and @@ -59,7 +61,7 @@ flowchart TB end subgraph target["Untrusted boundary"] - server["Target MCP server\n(spawned as argv array,\nNEVER via a shell)"] + server["Target MCP server\nstdio child (argv, no shell) OR\nalready-running Streamable HTTP endpoint"] end repo[("warden.lock\ncommitted baseline\n(root of trust)")] @@ -68,8 +70,8 @@ flowchart TB specs -. "implemented by" .-> warden - warden -- "1. spawn + initialize\n+ tools/list / resources/list\n/ prompts/list (stdio)" --> server - server -- "2. declared surface\n(definitions only)" --> warden + warden -- "1. stdio: spawn; HTTP: connect\n2. initialize + tools/list\nresources/list / prompts/list" --> server + server -- "3. declared surface\n(definitions only)" --> warden warden -- "pin: write baseline" --> repo repo -- "check: read baseline" --> warden @@ -90,12 +92,16 @@ sequenceDiagram autonumber participant CI as CI / operator participant W as mcp-warden - participant S as MCP server (stdio child) + participant S as MCP server (stdio child or HTTP endpoint) participant L as warden.lock Note over CI,L: pin (TOFU baseline) - CI->>W: pin --approve --approver - W->>S: spawn (argv array, no shell) + CI->>W: pin OR pin --url + alt stdio command + W->>S: spawn (argv array, no shell) + else --url + W->>S: connect (Streamable HTTP) + end W->>S: initialize S-->>W: protocolVersion W->>S: tools/list · resources/list · prompts/list @@ -104,9 +110,9 @@ sequenceDiagram W->>L: write warden.lock (hashes, redacted findings, approved_digest) Note over CI,L: check (later, in CI) - CI->>W: check + CI->>W: check OR check --url W->>L: read baseline - W->>S: spawn + initialize + list (same as pin) + W->>S: spawn or connect + initialize + list (same transport as pin) S-->>W: declared surface (possibly rug-pulled) W->>W: recompute digests, compute_drift(baseline, current) alt drift detected @@ -179,8 +185,9 @@ flowchart LR - **Trusted:** mcp-warden, the Python runtime it runs in, and `warden.lock` in the repo (delegated to host controls — PR review, branch protection). -- **Untrusted:** everything on the server side of the stdio pipe. -- The boundary is the **stdio channel** between mcp-warden and the spawned server. +- **Untrusted:** the target server and everything beyond its transport boundary. +- The boundary is the **stdio channel** to a spawned child or the network channel to the + configured Streamable HTTP endpoint. Runtime `guard` interception remains stdio-only. ## What is explicitly NOT in this picture @@ -196,7 +203,8 @@ flowchart LR - No network calls / no DNS resolution by checks, policy, or the proxy. Exfil + SSRF match on literal host strings **and (#54, D6) raw IP literals** in result text/args against the SSRF/exfil address ranges (`net_rules`) — deterministic, still no DNS-name resolution. -- stdio transport only (HTTP/SSE deferred for all of v0.1/v0.2/v0.3). +- No HTTP/SSE runtime proxy: `guard` remains stdio-only. Streamable HTTP support is limited + to definition capture by `pin` and `check` via `--url`. - The fuzzy `WRD-RES-INJECT-PHRASE` MONITOR tier is **never default-block**, even in v0.3 (opt-in only via `--block-inject-phrase`). - Windows lifecycle guarantees are **experimental** in v0.3 — job-object best-effort teardown,