feat(supervisor): add streaming HTTP request middleware evaluation #2431

Problem Statement

Parent: #1733

The v1 HTTP request hook, SupervisorMiddleware.EvaluateHttpRequest, is unary. The supervisor buffers the whole request body, builds one HttpRequestEvaluation, and sends it to middleware. Request and replacement bodies are capped at 4 MiB.

That works for small, full-body inspection. It does not work for:

• Middleware that decides from headers alone and should not force the body into a buffer.
• Large uploads that should pass through unchanged, such as headers-only SigV4 signing.
• Incremental inspection or transformation with backpressure.
• Long-lived or unknown-length request bodies.
• HTTP/2 and gRPC streams, which don't fit in one bounded unary payload.

HTTP/2 support (#2426) also needs middleware to run per logical HTTP stream, not on raw connection bytes. The streaming gRPC part of #2166 needs method, metadata, body, duration, rate, and cancellation controls without making middleware implement HTTP/2 framing.

Status

PR #4359 delivers the middleware contract half of this issue: the v2 HTTP hooks, for both requests and responses, over HTTP/1.x. The HTTP/2 half is still open. Today h2c and other traffic the supervisor can't parse reach v2 middleware only as an allow-or-refuse decision, described below.

Design (as implemented in #4359)

Contract

Two new bidirectional streaming RPCs share one set of event and result messages:

• SupervisorMiddleware.EvaluateHttpRequestV2 for HTTP_REQUEST_V2 at PRE_CREDENTIALS.
• SupervisorMiddleware.EvaluateHttpResponseV2 for HTTP_RESPONSE_V2 at PRE_RETURN.

The supervisor opens one stream per middleware stage and HTTP message. A request and its response use separate streams.

Each stream starts with a preflight carrying the head (target, headers, and status for responses), request context, the policy config, the permitted body modes, the modes OpenShell removed and why, the limits, and the declared body length when known. The stage answers with one of:

• continue_without_body, with optional header mutations. The stage is done and later stages still see the body.
• inspect with BUFFERED or STREAM.
• reject.

The supervisor runs every stage's preflight in chain order before any stage gets a body byte.

Body modes

• BUFFERED. One buffered_body with the full body and trailers, up to the stage's payload limit (max 4 MiB). The stage returns unchanged or a replacement, plus late header and trailer mutations. Nothing goes out until the whole chain approves.
• STREAM. Ordered input_chunk events (64 KiB max each) and one input_end with trailers. The stage writes its own output: output_start, any number of output_chunk, then finish. Input and output chunk counts are independent. No total size or time limit. A stage that stalls for 30 seconds fails.

For responses, the supervisor removes modes it can't honour and says why: BODYLESS, PARTIAL, NO_TRANSFORM, ENCODED, TRUNCATION_UNDETECTABLE, OPEN_ENDED, OVER_LIMIT. For requests it removes BUFFERED when the declared length is over the stage limit.

Request delivery

Request output streams to the upstream while the sandbox is still uploading. The supervisor withholds output (up to 4 MiB) and sends it with the head when the route needs the whole body: a BUFFERED stage, SigV4 signing, request body credential rewrite, the forward proxy, JSON-RPC/MCP/GraphQL endpoints, HTTP/1.0, or Upgrade. Body-aware endpoints re-check every replaced body against policy before the next stage or the upstream sees it. Credentials are injected after all middleware mutations.

If a STREAM stage fails after the head reached the upstream, the supervisor closes that upstream connection. It never replays a partly forwarded request.

Traffic the supervisor can't inspect

For each entry that selects a connection the supervisor can't show as HTTP, and whose service binds HTTP_REQUEST_V2, the supervisor opens a request exchange with an uninspectable preflight. Reasons are TLS_SKIP, H2C, UNSUPPORTED_TUNNEL, RAW_TCP, and SQL_PASSTHROUGH. No body mode is offered. continue_without_body allows the connection and anything else refuses it. That's why a policy can now attach v2 request middleware to tls: skip endpoints.

Failure behaviour

v2 hooks always fail closed. The gateway rejects on_error: fail_open on entries whose service runs v2 hooks. Requests get 403 with middleware_denied or middleware_failed. Responses get 403/502 before the head is committed, and an aborted delivery plus a high-severity finding after.

Version selection and compatibility

We dropped the capability-advertisement idea from the original proposal. The binding operation picks the hook version. Gateways and supervisors that predate v2 don't know HTTP_REQUEST_V2/HTTP_RESPONSE_V2 and refuse the service at Describe, so a v2 service never gets called through a v1 RPC.

We also dropped the unary-to-stream adapter and the single shared chain runner:

• v1 hooks keep their own engines and still work. They are deprecated for removal in 0.2.0, and the docs carry a migration guide.
• One service implements one HTTP hook version. The gateway refuses a manifest with both.
• One request or response chain can't mix versions. The gateway rejects policies whose overlapping selectors would do that, and the supervisor fails such a chain closed with middleware_hook_versions_mixed.

Remaining work: HTTP/2

#2426 tracks the transport gap: the L7 proxy only negotiates HTTP/1.1, so HTTP/2 needs tls: skip and skips L7 inspection. With v2 hooks, middleware can at least refuse that traffic, but it can't see it.

The v2 contract carries no HTTP/1-specific framing, so an HTTP/2 request stream should map onto the same exchange: one stream per stage and logical HTTP request. What's left:

• Negotiate HTTP/1.1 or HTTP/2 for inspected traffic and relay HTTP/2 through the L7 path without tls: skip.
• Run each HTTP/2 request stream through its own v2 hook sessions.
• Keep multiplexing, flow control, trailers, and cancellation working, and keep a denial or timeout on one stream from killing the others.
• Review the response mode rules for HTTP/2. TRUNCATION_UNDETECTABLE currently removes STREAM for anything that isn't HTTP/1.1.

The middleware API must not expose HTTP/2 frames, HPACK state, or stream IDs.

#2166

v2 hooks give the gRPC part of #2166 a base at the HTTP stream level: host, service, and method admission through authority and path, metadata limits, request byte, rate, duration, and concurrency limits, and cancellation and audit. gRPC message boundaries and protobuf parsing would need their own typed operation. Generic body chunks are not gRPC messages.

WebSocket middleware is tracked by #2428 and works the same under both hook versions.

Alternatives Considered

Advertise capabilities in the manifest. The original proposal. We picked binding operations instead. Old peers reject unknown operations, so there is no silent fallback, and there is nothing extra to negotiate.

Run v1 through an adapter on one shared runner. More moving parts in the supervisor to keep alive an API we plan to remove. Separate engines plus a no-mixing rule are simpler and easier to delete in 0.2.0.

Replace v1 right away. v1 was already public in 0.1.x. Deprecation with a migration guide gives services a release to move.

Add an HTTP/2-specific middleware API. Same request concepts as HTTP/1.x. Two APIs would duplicate chaining, transformation, and failure rules and leak transport details.

Keep buffering every request. Can't handle large uploads, long-lived streams, or headers-only processing.

Definition of Done

Covered by #4359. Check these off when it merges:

• Reviewed design for the streaming hooks, body modes, lifecycle, and compatibility. RFC 0009 and the V2 HTTP Hooks docs page describe it.
• Middleware can decide from headers without forcing the body into a buffer.
• BUFFERED mode holds the body under a 4 MiB bound and sends no bytes before the chain approves.
• STREAM mode has defined backpressure, transformation, rejection, and partial-disclosure behaviour.
• Mode selection can't silently weaken enforcement. Removed modes come with a reason, and routes that need the full body withhold output.
• v1 hooks keep working, with a deprecation date (0.2.0) and a migration guide.
• Middleware can allow or refuse traffic the supervisor can't inspect.
• Ordering, mutation validation, policy re-checks, findings, metadata, and timeouts stay consistent. v2 is always fail-closed.
• Unit tests and the e2e:middleware-http-v2 suite cover v2 over HTTP/1.x. The v1 e2e still passes.

Still open:

• The supervisor negotiates HTTP/1.1 or HTTP/2 for inspected traffic and relays HTTP/2 through the L7 path without tls: skip.
• HTTP/1.x requests and HTTP/2 request streams use the same v2 hook sessions.
• The HTTP/2 relay keeps multiplexing, flow control, trailers, cancellation, and isolation between streams.
• Tests cover v2 hooks over HTTP/2.
• Docs describe HTTP/2 behaviour.

Non-Goals

• Parsing raw HTTP/2 frames in middleware.
• Treating HTTP body chunks as gRPC messages.
• Protobuf or app-specific media inspection.
• WebSocket message middleware (feat(supervisor): add WebSocket message introspection middleware #2428).
• HTTP/3.