feat(httpproxy): fail-closed request handling, client auth, and distinct upstream errors

[dgenio]

Summary

Harden the streamable-HTTP proxy's request edge so every non-happy-path body is fail-closed instead of forwarded uninspected, add optional client authentication, and surface transport failures distinctly from policy blocks. All changes converge on internal/httpproxy (ServeHTTP/forward) plus supporting helpers in internal/mcp and flag wiring in the proxy-http CLI.

This is one coherent PR covering five tightly-coupled issues that all edit the same request path and would otherwise conflict as separate PRs.

What changed

• Gate JSON-RPC batch bodies (Gate or explicitly reject JSON-RPC batch bodies in the HTTP proxy #128, Investigate JSON-RPC batch handling across both transports end to end #145). A batch (array) body previously failed the single-request parser and fell through to a transparent, ungated forward — a documented policy bypass. New --on-batch reject|evaluate (default reject, fail-closed). evaluate gates every tools/call member and forwards the whole batch only if all are allowed (all-or-nothing); each member's decision is audited. Cross-transport design note added at docs/batch-handling.md.
• Oversize + unparseable bodies (Add a deny-on-oversize-or-unparseable-body policy switch to the HTTP proxy #152). Oversize bodies are refused (JSON-RPC error envelope when they look like JSON-RPC, else HTTP 413) rather than forwarded. New --on-unparsed forward|reject for bodies that don't parse as JSON-RPC. Valid non-tools/call methods (initialize/ping/notifications) are still forwarded so sessions keep working.
• Client auth + bind guardrail (Add client authentication and bind-address guardrails to the HTTP proxy #138). --auth-token (or $AGENTFENCE_PROXY_AUTH_TOKEN) requires Authorization: Bearer <token>; unauthenticated requests get HTTP 401 and reach neither the policy edge nor the upstream. The gate token is compared in constant time and never relayed upstream. A startup warning fires when --listen binds a non-loopback address without a token.
• Distinct failure errors (Surface upstream/subprocess failures to the agent and operator distinctly #133). Upstream-unreachable and internal proxy failures now return distinct JSON-RPC error envelopes (-32002 upstream unavailable, -32003 proxy error) addressed to the request id, replacing the generic HTTP 502, so an operator can tell a transport failure apart from a policy block.

Related issue

Fixes #128
Fixes #133
Fixes #138
Fixes #145
Fixes #152

Why

proxy-http is a security gate, so "documented hole" (ungated batches, transparently-forwarded unparseable/oversize bodies, an unauthenticated off-loopback listener) is still a hole. These were tracked as separate issues but share one code path; fixing them together keeps the request-handling contract consistent (one JSON-RPC error-envelope shape, one fail-closed boundary) and avoids three PRs racing on the same function.

How verified

• make ci (fmt-check, vet, go test -race + coverage) — all packages pass, 81.4% total coverage.
• New unit tests: batch reject/evaluate (all-allowed forwards, one-denied rejects all), oversize, unparsed forward vs reject, auth (missing/wrong/correct token, token not relayed upstream), upstream-failure error envelope; plus mcp helpers (IsBatch, LooksLikeJSONRPC, ParseBatch, error constructors) and CLI helpers (parseBatchPolicy, parseUnparsedPolicy, isLoopbackListen, bad-enum rejection).
• Manual smoke test of the built binary: invalid --on-batch exits non-zero with a clear message; off-loopback bind without a token prints the guardrail warning; the warning is silent when a token is set.

Tradeoffs / risks

• Behavior change (no back-compat constraint per request): batches are now rejected by default; clients that relied on transparent batch forwarding must send one request per body or use --on-batch evaluate. Documented in the CHANGELOG and design note (MCP is removing JSON-RPC batching, so reject-by-default is low-regret).
• Upstream/proxy failures now return HTTP 200 + JSON-RPC error envelope instead of HTTP 502 (the MCP-correct shape).
• --on-batch evaluate is intentionally all-or-nothing rather than per-member response reassembly, to stay fail-closed and avoid splitting streamed responses.

Scope notes

The stdio proxy also forwards batch lines ungated; an equivalent --on-batch switch there is noted as follow-up in docs/batch-handling.md (the HTTP path is where batches arrive as a single inspectable body). Related fuzzing of SSE/response parsing (#140) was deliberately left out to keep this PR focused on the request edge.

Checklist

• Tests added or updated
• Documentation updated if needed (threat-model, integration-guide, new batch-handling note, CHANGELOG)
• CI passes (make ci, race + coverage)
• Issue number included

https://claude.ai/code/session_01S6MZXehSbKmLFM9DGsee9y

---

Generated by Claude Code

[Copilot]

Pull request overview

This PR hardens the MCP-over-HTTP proxy request boundary (internal/httpproxy) to be fail-closed on previously ungated edge cases (notably JSON-RPC batch bodies), adds optional client authentication for the HTTP listener, and returns distinct JSON-RPC error envelopes for upstream vs internal proxy failures—backed by new unit tests and documentation updates.

Changes:

• Add batch detection + --on-batch reject|evaluate (default reject) and --on-unparsed forward|reject, and refuse oversize bodies instead of forwarding uninspected.
• Add optional bearer-token client authentication (--auth-token / $AGENTFENCE_PROXY_AUTH_TOKEN) plus a non-loopback bind warning.
• Introduce distinct JSON-RPC error codes/envelopes for upstream unavailable vs proxy error, and document the new behavior.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
internal/mcp/model.go	Adds new proxy-specific JSON-RPC error codes plus helpers for batch detection/parse and distinct upstream/proxy errors.
internal/mcp/model_test.go	Adds unit tests covering batch helpers and new error-code helpers.
internal/httpproxy/httpproxy.go	Implements fail-closed handling for batch/oversize/unparsed bodies, optional auth, and distinct upstream/proxy error envelopes.
internal/httpproxy/httpproxy_test.go	Adds tests for batch reject/evaluate behavior, unparsed forward/reject, auth enforcement, and upstream failure envelopes.
docs/threat-model.md	Updates HTTP-proxy surface documentation for auth, batch/unparsed/oversize behavior, and error shaping.
docs/integration-guide.md	Documents new proxy-http flags and links to updated operational guidance.
docs/batch-handling.md	Adds a cross-transport design note explaining the batch policy decision and migration implications.
cmd/agentfence/main.go	Wires new CLI flags, env fallback for auth token, bind guardrail, and enum parsing helpers.
cmd/agentfence/main_test.go	Adds tests for enum parsing, loopback detection, and failing fast on invalid flag values.
CHANGELOG.md	Documents the new fail-closed behaviors, auth option, and distinct upstream/proxy error envelopes.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.