diff --git a/README.md b/README.md index 24faef1..d0023ed 100644 --- a/README.md +++ b/README.md @@ -11,6 +11,7 @@ A lightweight, embeddable development dashboard for Go applications. Monitor log - **HTTP Server**: Track incoming HTTP requests to your application - **SQL Queries**: Monitor database queries with timing and arguments - **On-Demand Capture**: Start/stop capturing through the dashboard UI with session or global modes +- **Agent Access (MCP)**: Expose captured events to AI coding agents over the Model Context Protocol (opt-in) - **Multi-User Isolation**: Each user gets their own event storage with independent clearing - **Low Overhead**: Designed to be lightweight; no events captured until you start a session - **Easy to Integrate**: Embeds into your application with minimal configuration @@ -299,6 +300,133 @@ dlog := devlog.NewWithOptions(devlog.Options{ }) ``` +## Agent Access (MCP) + +devlog can expose its captured events to AI coding agents (Claude Code, Cursor, …) over the +[Model Context Protocol](https://modelcontextprotocol.io). An agent can then list requests, +inspect SQL queries and logs, fetch bodies, and control capture — giving it the same context a +developer reads in the dashboard. + +This consists of two parts: + +1. A read-only **Agent JSON API** mounted on the dashboard handler (under `/api/agent/v1/`). +2. The **`devlog` CLI** (`./cli` module), which exposes that API to an agent over MCP. + +> The browser-relayed variant for **deployed** (stage/production) instances is a separate, planned +> phase. What is described here is the local-development path. See +> [`docs/design/agent-access.md`](docs/design/agent-access.md) and +> [`docs/design/agent-access-relay.md`](docs/design/agent-access-relay.md) for the full design. + +### Enabling the API + +The Agent JSON API is **off by default** — unlike the dashboard UI (which shows data to a human on +screen), the agent API is consumed by tools that may forward data to third-party LLM providers, so +it must be enabled deliberately. Enable it with `WithAgentAPI()`: + +```go +mux.Handle("/_devlog/", http.StripPrefix("/_devlog", dlog.DashboardHandler("/_devlog", + dashboard.WithAgentAPI(), +))) +``` + +It inherits whatever authentication middleware you have placed in front of the dashboard. + +#### Redaction + +Sensitive header values are masked by default (`Authorization`, `Cookie`, `Set-Cookie`, +`WWW-Authenticate`, `Proxy-Authenticate`, `Proxy-Authorization`), with the key preserved so an agent +can tell a header is redacted rather than absent. Request/response bodies are never inlined in list +or detail responses — only metadata is — and are served (capped) by dedicated endpoints. + +```go +dlog.DashboardHandler("/_devlog", + dashboard.WithAgentAPI(), + dashboard.WithAgentRedactedHeaders("X-Api-Key"), // mask additional headers + dashboard.WithAgentMaxBodyBytes(64*1024), // cap body bytes served (default: 64 KiB) + dashboard.WithAgentRedactor(func(d *dashboard.EventDetail) *dashboard.EventDetail { + // Mutate the detail (e.g. clear d.RequestBody.Available to suppress a body), + // or return nil to drop the event from the agent API entirely. + return d + }), + // dashboard.WithAgentInsecureHeaders(), // disable header masking (local dev only) +) +``` + +#### Endpoints + +All responses are JSON; errors use `{"error": "..."}`. `{sid}` is the capture session id. + +| Method | Path | Purpose | +|--------|------|---------| +| GET | `/api/agent/v1/s/{sid}/events` | Event summaries, newest first. Filters: `type`, `since`/`until` (RFC 3339), `limit`, `status` (e.g. `5xx`), `path` | +| GET | `/api/agent/v1/s/{sid}/events/{id}` | Full event detail incl. child events and body metadata | +| GET | `/api/agent/v1/s/{sid}/events/{id}/request-body` | Raw request body (redaction- and cap-aware) | +| GET | `/api/agent/v1/s/{sid}/events/{id}/response-body` | Raw response body (redaction- and cap-aware) | +| GET | `/api/agent/v1/s/{sid}/capture/status` | `{active, mode, eventCount}` (read-only; capture is started/stopped by the user in the dashboard) | +| GET | `/api/agent/v1/sessions` | List active capture sessions (for the relay to attach to) | +| GET | `/api/agent/v1/stats` | Memory and event statistics | + +### The `devlog` CLI + +The CLI lives in the `./cli` module and bridges the JSON API to an agent over MCP. In `--direct` +mode it talks to a local devlog instance over plain HTTP — no browser tunnel involved. + +The relay does **not** create its own capture session — it **attaches to an existing one** so the +agent and your dashboard see the same events. Open the dashboard and start a capture first, then: + +```bash +# from the repository root (workspace) or the ./cli module +go run ./cli relay --direct http://localhost:8080/_devlog --mcp-port 4319 + +# or build a binary +go build -C cli -o devlog . +./cli/devlog relay --direct http://localhost:8080/_devlog --mcp-port 4319 +``` + +On startup the relay lists active sessions: if exactly one is active it attaches automatically; if +several are, it prompts you to choose; if none exist yet it waits until one appears. Use +`--session ` (the id from the dashboard URL) to attach non-interactively. Because the dashboard +tab owns the session, no agent-side lifetime management is needed — when you close the dashboard the +session ends normally. + +The relay binds to `127.0.0.1` only and serves an MCP endpoint at `http://127.0.0.1:/mcp`. The +endpoint validates the `Host` header is a loopback literal to block DNS-rebinding from a browser. + +MCP tools exposed (all read-only): `list_events`, `get_event`, `get_request_body`, +`get_response_body`, `capture_status`, `get_stats`, `list_environments`. The agent cannot start or +stop capture — you manage capture from the dashboard; the agent only piggy-backs on your session. + +### Integrating with an agent + +For an MCP client that supports streamable-HTTP servers (e.g. Claude Code), point it at the relay's +MCP URL. With Claude Code, add a project-scoped `.mcp.json`: + +```json +{ + "mcpServers": { + "devlog": { + "type": "http", + "url": "http://127.0.0.1:4319/mcp" + } + } +} +``` + +Keep the relay running while the agent is connected. A typical flow: you open the dashboard and start +capturing, the relay attaches to that session, you exercise the application, then the agent calls +`list_events` / `get_event` to inspect what happened. Capture is forward-looking — events are +captured from the moment capture starts in the dashboard. + +### Security notes + +- **Opt-in and auth-inherited.** The API is disabled unless `WithAgentAPI()` is set and is protected + by your existing dashboard authentication middleware. +- **Egress awareness.** Data returned to an agent may be sent to an LLM provider. Default header + redaction and the body size cap limit exposure; add a `WithAgentRedactor` for application-specific + scrubbing, and prefer leaving the API off where regulated data flows through. +- **Loopback only.** The CLI binds to `127.0.0.1` and rejects non-loopback `Host` headers on the MCP + endpoint. + ## Development ### Running Acceptance Tests diff --git a/cli/client.go b/cli/client.go new file mode 100644 index 0000000..7c4cf67 --- /dev/null +++ b/cli/client.go @@ -0,0 +1,199 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "net/url" + "strconv" + "strings" + "time" +) + +// directClient talks to a local devlog Agent JSON API over plain HTTP. It is +// attached to an existing capture session (owned by a dashboard browser tab); it +// never creates sessions of its own. +type directClient struct { + agentBase string // /api/agent/v1 + httpc *http.Client + sid string // attached session id +} + +func newDirectClient(baseURL, sid string) *directClient { + return &directClient{ + agentBase: agentBaseURL(baseURL), + httpc: defaultHTTPClient(), + sid: sid, + } +} + +func agentBaseURL(baseURL string) string { + return strings.TrimRight(baseURL, "/") + "/api/agent/v1" +} + +func defaultHTTPClient() *http.Client { + return &http.Client{Timeout: 30 * time.Second} +} + +func (c *directClient) sessionURL(suffix string) string { + return c.agentBase + "/s/" + c.sid + suffix +} + +func (c *directClient) listEvents(ctx context.Context, in listEventsInput) (json.RawMessage, error) { + q := url.Values{} + if in.Type != "" { + q.Set("type", in.Type) + } + if in.Since != "" { + q.Set("since", in.Since) + } + if in.Until != "" { + q.Set("until", in.Until) + } + if in.Status != "" { + q.Set("status", in.Status) + } + if in.Path != "" { + q.Set("path", in.Path) + } + if in.Limit > 0 { + q.Set("limit", strconv.Itoa(in.Limit)) + } + u := c.sessionURL("/events") + if len(q) > 0 { + u += "?" + q.Encode() + } + return c.expectOK(c.getRaw(ctx, u)) +} + +func (c *directClient) getEvent(ctx context.Context, eventID string) (json.RawMessage, error) { + u := c.sessionURL("/events/" + url.PathEscape(eventID)) + return c.expectOK(c.getRaw(ctx, u)) +} + +func (c *directClient) getStats(ctx context.Context) (json.RawMessage, error) { + return c.expectOK(c.getRaw(ctx, c.agentBase+"/stats")) +} + +func (c *directClient) captureStatus(ctx context.Context) (json.RawMessage, error) { + return c.expectOK(c.getRaw(ctx, c.sessionURL("/capture/status"))) +} + +// bodyResult is a fetched request/response body plus its content type and whether +// it was truncated by the server's size cap. +type bodyResult struct { + data []byte + contentType string + truncated bool +} + +// getBody fetches the raw request or response body (side is "request" or +// "response"). Non-200 responses (404 missing, 410 redacted, 400 wrong type) +// become errors carrying the server's message. +func (c *directClient) getBody(ctx context.Context, eventID, side string) (*bodyResult, error) { + u := c.sessionURL("/events/" + url.PathEscape(eventID) + "/" + side + "-body") + req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil) + if err != nil { + return nil, err + } + resp, err := c.httpc.Do(req) + if err != nil { + return nil, err + } + defer resp.Body.Close() + data, err := io.ReadAll(resp.Body) + if err != nil { + return nil, err + } + if resp.StatusCode != http.StatusOK { + return nil, apiError(resp.StatusCode, data) + } + return &bodyResult{ + data: data, + contentType: resp.Header.Get("Content-Type"), + truncated: resp.Header.Get("X-Devlog-Truncated") == "true", + }, nil +} + +func (c *directClient) getRaw(ctx context.Context, u string) (json.RawMessage, int, error) { + req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil) + if err != nil { + return nil, 0, err + } + resp, err := c.httpc.Do(req) + if err != nil { + return nil, 0, err + } + defer resp.Body.Close() + data, err := io.ReadAll(resp.Body) + if err != nil { + return nil, resp.StatusCode, err + } + return json.RawMessage(data), resp.StatusCode, nil +} + +func (c *directClient) expectOK(raw json.RawMessage, status int, err error) (json.RawMessage, error) { + if err != nil { + return nil, err + } + if status == http.StatusNotFound && isNoSessionError(raw) { + return nil, fmt.Errorf("the devlog session %s is no longer active (dashboard closed?); reconnect the relay", c.sid) + } + if status != http.StatusOK { + return nil, apiError(status, raw) + } + return raw, nil +} + +func isNoSessionError(raw json.RawMessage) bool { + var e struct { + Error string `json:"error"` + } + return json.Unmarshal(raw, &e) == nil && strings.Contains(e.Error, "no capture session") +} + +func apiError(status int, raw json.RawMessage) error { + var e struct { + Error string `json:"error"` + } + if json.Unmarshal(raw, &e) == nil && e.Error != "" { + return fmt.Errorf("agent API error (%d): %s", status, e.Error) + } + return fmt.Errorf("agent API error: status %d", status) +} + +// sessionInfo mirrors one entry of GET /api/agent/v1/sessions. +type sessionInfo struct { + SessionID string `json:"sessionId"` + Mode string `json:"mode"` + Capturing bool `json:"capturing"` + EventCount int `json:"eventCount"` + LastActiveMsAgo int64 `json:"lastActiveMsAgo"` +} + +// fetchSessions lists the active capture sessions of the devlog instance at baseURL. +func fetchSessions(ctx context.Context, baseURL string, httpc *http.Client) ([]sessionInfo, error) { + req, err := http.NewRequestWithContext(ctx, http.MethodGet, agentBaseURL(baseURL)+"/sessions", nil) + if err != nil { + return nil, err + } + resp, err := httpc.Do(req) + if err != nil { + return nil, err + } + defer resp.Body.Close() + data, err := io.ReadAll(resp.Body) + if err != nil { + return nil, err + } + if resp.StatusCode != http.StatusOK { + return nil, apiError(resp.StatusCode, data) + } + var sessions []sessionInfo + if err := json.Unmarshal(data, &sessions); err != nil { + return nil, fmt.Errorf("decoding sessions: %w", err) + } + return sessions, nil +} diff --git a/cli/direct_test.go b/cli/direct_test.go new file mode 100644 index 0000000..d579ed0 --- /dev/null +++ b/cli/direct_test.go @@ -0,0 +1,152 @@ +package main + +import ( + "context" + cryptorand "crypto/rand" + "encoding/json" + "fmt" + "io" + "net/http" + "net/http/httptest" + "net/url" + "strings" + "testing" + + "github.com/networkteam/devlog" + "github.com/networkteam/devlog/collector" + "github.com/networkteam/devlog/dashboard" +) + +// newTestDevlog starts an httptest server hosting a collected app plus the devlog +// dashboard with the agent API enabled. It returns the dashboard base URL (the +// --direct target) and the app URL whose traffic is captured. +func newTestDevlog(t *testing.T) (baseURL, appURL string) { + return newTestDevlogWith(t) +} + +func newTestDevlogWith(t *testing.T, agentOpts ...dashboard.HandlerOption) (baseURL, appURL string) { + t.Helper() + + dlog := devlog.NewWithOptions(devlog.Options{ + HTTPServerOptions: &collector.HTTPServerOptions{ + CaptureRequestBody: true, + CaptureResponseBody: true, + MaxBodySize: 1 << 20, + }, + }) + t.Cleanup(dlog.Close) + + app := dlog.CollectHTTPServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = io.Copy(io.Discard, r.Body) + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(`{"ok":true}`)) + })) + + opts := append([]dashboard.HandlerOption{dashboard.WithAgentAPI()}, agentOpts...) + outer := http.NewServeMux() + outer.Handle("/_devlog/", http.StripPrefix("/_devlog", dlog.DashboardHandler("/_devlog", opts...))) + outer.Handle("/", app) + + srv := httptest.NewServer(outer) + t.Cleanup(srv.Close) + + return srv.URL + "/_devlog", srv.URL + "/" +} + +// startSession simulates a dashboard browser tab creating a capture session via +// the dashboard's own capture/start endpoint, returning the session id the relay +// attaches to. (The agent API has no start endpoint — capture is user-managed.) +func startSession(t *testing.T, baseURL, mode string) string { + t.Helper() + sid := newTestSID() + resp, err := http.PostForm( + strings.TrimRight(baseURL, "/")+"/s/"+sid+"/capture/start", + url.Values{"mode": {mode}}, + ) + if err != nil { + t.Fatalf("start session: %v", err) + } + resp.Body.Close() + if resp.StatusCode != http.StatusOK { + t.Fatalf("start session: status %d", resp.StatusCode) + } + return sid +} + +func newTestSID() string { + var b [16]byte + _, _ = cryptorand.Read(b[:]) + b[6] = (b[6] & 0x0f) | 0x40 + b[8] = (b[8] & 0x3f) | 0x80 + return fmt.Sprintf("%x-%x-%x-%x-%x", b[0:4], b[4:6], b[6:8], b[8:10], b[10:16]) +} + +func TestDirectMode_ListEvents(t *testing.T) { + baseURL, appURL := newTestDevlog(t) + ctx := context.Background() + + // Simulate the dashboard creating a capture session. + sid := startSession(t, baseURL, "global") + + // Drive captured traffic through the collected app. + resp, err := http.Get(appURL) + if err != nil { + t.Fatalf("app request: %v", err) + } + resp.Body.Close() + + // The relay attaches to the existing session. + c := newDirectClient(baseURL, sid) + raw, err := c.listEvents(ctx, listEventsInput{}) + if err != nil { + t.Fatalf("listEvents: %v", err) + } + + var summaries []map[string]any + if err := json.Unmarshal(raw, &summaries); err != nil { + t.Fatalf("decode summaries: %v", err) + } + if len(summaries) == 0 { + t.Fatal("expected at least one captured event") + } + if summaries[0]["type"] != "http_server" { + t.Errorf("expected http_server event, got %v", summaries[0]["type"]) + } +} + +func TestSelectSession_SingleAutoSelect(t *testing.T) { + baseURL, _ := newTestDevlog(t) + sid := startSession(t, baseURL, "global") + + got, err := selectSession(context.Background(), baseURL, "") + if err != nil { + t.Fatalf("selectSession: %v", err) + } + if got != sid { + t.Errorf("auto-selected %s, want the only session %s", got, sid) + } +} + +func TestSelectSession_ExplicitOverride(t *testing.T) { + baseURL, _ := newTestDevlog(t) + got, err := selectSession(context.Background(), baseURL, "explicit-sid") + if err != nil { + t.Fatalf("selectSession: %v", err) + } + if got != "explicit-sid" { + t.Errorf("explicit sid not honored, got %s", got) + } +} + +func TestExpectOK_SessionGoneIsClearError(t *testing.T) { + baseURL, _ := newTestDevlog(t) + // Attach to a session id that was never created. + c := newDirectClient(baseURL, newTestSID()) + _, err := c.listEvents(context.Background(), listEventsInput{}) + if err == nil { + t.Fatal("expected an error for a missing session") + } + if !strings.Contains(err.Error(), "no longer active") { + t.Errorf("expected a clear session-ended error, got %v", err) + } +} diff --git a/cli/go.mod b/cli/go.mod new file mode 100644 index 0000000..4f3dc15 --- /dev/null +++ b/cli/go.mod @@ -0,0 +1,50 @@ +module github.com/networkteam/devlog/cli + +go 1.25.0 + +replace github.com/networkteam/devlog => ../ + +require ( + github.com/modelcontextprotocol/go-sdk v1.6.1 + github.com/networkteam/devlog v0.0.0 + github.com/spf13/cobra v1.9.1 + golang.org/x/term v0.44.0 +) + +require ( + github.com/a-h/parse v0.0.0-20250122154542-74294addb73e // indirect + github.com/a-h/templ v0.3.865 // indirect + github.com/alecthomas/chroma/v2 v2.17.2 // indirect + github.com/andybalholm/brotli v1.1.1 // indirect + github.com/apex/log v1.9.0 // indirect + github.com/cenkalti/backoff/v4 v4.3.0 // indirect + github.com/cli/browser v1.3.0 // indirect + github.com/dlclark/regexp2 v1.11.5 // indirect + github.com/fatih/color v1.18.0 // indirect + github.com/fsnotify/fsnotify v1.9.0 // indirect + github.com/gofrs/uuid v4.4.0+incompatible // indirect + github.com/google/jsonschema-go v0.4.3 // indirect + github.com/inconshreveable/mousetrap v1.1.0 // indirect + github.com/mattn/go-colorable v0.1.14 // indirect + github.com/mattn/go-isatty v0.0.20 // indirect + github.com/natefinch/atomic v1.0.1 // indirect + github.com/networkteam/refresh v1.15.0 // indirect + github.com/pkg/errors v0.9.1 // indirect + github.com/r3labs/sse/v2 v2.10.0 // indirect + github.com/rjeczalik/notify v0.9.3 // indirect + github.com/rs/cors v1.11.1 // indirect + github.com/samber/lo v1.50.0 // indirect + github.com/segmentio/asm v1.1.3 // indirect + github.com/segmentio/encoding v0.5.4 // indirect + github.com/spf13/pflag v1.0.6 // indirect + github.com/yosida95/uritemplate/v3 v3.0.2 // indirect + golang.org/x/mod v0.33.0 // indirect + golang.org/x/net v0.50.0 // indirect + golang.org/x/oauth2 v0.35.0 // indirect + golang.org/x/sync v0.19.0 // indirect + golang.org/x/sys v0.46.0 // indirect + golang.org/x/text v0.34.0 // indirect + golang.org/x/tools v0.42.0 // indirect + gopkg.in/cenkalti/backoff.v1 v1.1.0 // indirect + gopkg.in/yaml.v2 v2.4.0 // indirect +) diff --git a/cli/go.sum b/cli/go.sum new file mode 100644 index 0000000..adec1a0 --- /dev/null +++ b/cli/go.sum @@ -0,0 +1,173 @@ +github.com/a-h/parse v0.0.0-20250122154542-74294addb73e h1:HjVbSQHy+dnlS6C3XajZ69NYAb5jbGNfHanvm1+iYlo= +github.com/a-h/parse v0.0.0-20250122154542-74294addb73e/go.mod h1:3mnrkvGpurZ4ZrTDbYU84xhwXW2TjTKShSwjRi2ihfQ= +github.com/a-h/templ v0.3.865 h1:nYn5EWm9EiXaDgWcMQaKiKvrydqgxDUtT1+4zU2C43A= +github.com/a-h/templ v0.3.865/go.mod h1:oLBbZVQ6//Q6zpvSMPTuBK0F3qOtBdFBcGRspcT+VNQ= +github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0= +github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k= +github.com/alecthomas/chroma/v2 v2.17.2 h1:Rm81SCZ2mPoH+Q8ZCc/9YvzPUN/E7HgPiPJD8SLV6GI= +github.com/alecthomas/chroma/v2 v2.17.2/go.mod h1:RVX6AvYm4VfYe/zsk7mjHueLDZor3aWCNE14TFlepBk= +github.com/alecthomas/repr v0.4.0 h1:GhI2A8MACjfegCPVq9f1FLvIBS+DrQ2KQBFZP1iFzXc= +github.com/alecthomas/repr v0.4.0/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4= +github.com/andybalholm/brotli v1.1.1 h1:PR2pgnyFznKEugtsUo0xLdDop5SKXd5Qf5ysW+7XdTA= +github.com/andybalholm/brotli v1.1.1/go.mod h1:05ib4cKhjx3OQYUY22hTVd34Bc8upXjOLL2rKwwZBoA= +github.com/apex/log v1.9.0 h1:FHtw/xuaM8AgmvDDTI9fiwoAL25Sq2cxojnZICUU8l0= +github.com/apex/log v1.9.0/go.mod h1:m82fZlWIuiWzWP04XCTXmnX0xRkYYbCdYn8jbJeLBEA= +github.com/apex/logs v1.0.0/go.mod h1:XzxuLZ5myVHDy9SAmYpamKKRNApGj54PfYLcFrXqDwo= +github.com/aphistic/golf v0.0.0-20180712155816-02c07f170c5a/go.mod h1:3NqKYiepwy8kCu4PNA+aP7WUV72eXWJeP9/r3/K9aLE= +github.com/aphistic/sweet v0.2.0/go.mod h1:fWDlIh/isSE9n6EPsRmC0det+whmX6dJid3stzu0Xys= +github.com/aws/aws-sdk-go v1.20.6/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= +github.com/aybabtme/rgbterm v0.0.0-20170906152045-cc83f3b3ce59/go.mod h1:q/89r3U2H7sSsE2t6Kca0lfwTK8JdoNGS/yzM/4iH5I= +github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8= +github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE= +github.com/cli/browser v1.3.0 h1:LejqCrpWr+1pRqmEPDGnTZOjsMe7sehifLynZJuqJpo= +github.com/cli/browser v1.3.0/go.mod h1:HH8s+fOAxjhQoBUAsKuPCbqUuxZDhQ2/aD+SzsEfBTk= +github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/dlclark/regexp2 v1.11.5 h1:Q/sSnsKerHeCkc/jSTNq1oCm7KiVgUMZRDUoRu0JQZQ= +github.com/dlclark/regexp2 v1.11.5/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8= +github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= +github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM= +github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU= +github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= +github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k= +github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0= +github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= +github.com/gofrs/uuid v4.4.0+incompatible h1:3qXRTX8/NbyulANqlc0lchS1gqAVxRgsuW1YrTJupqA= +github.com/gofrs/uuid v4.4.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM= +github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY= +github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE= +github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/jsonschema-go v0.4.3 h1:/DBOLZTfDow7pe2GmaJNhltueGTtDKICi8V8p+DQPd0= +github.com/google/jsonschema-go v0.4.3/go.mod h1:r5quNTdLOYEz95Ru18zA0ydNbBuYoo9tgaYcxEYhJVE= +github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM= +github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg= +github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= +github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= +github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= +github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= +github.com/jpillora/backoff v0.0.0-20180909062703-3050d21c67d7/go.mod h1:2iMrUgbbvHEiQClaW2NsSzMyGHqN+rDFqY705q49KG0= +github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= +github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= +github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/mattn/go-colorable v0.1.1/go.mod h1:FuOcm+DKB9mbwrcAfNl7/TZVBZ6rcnceauSikq3lYCQ= +github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= +github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE= +github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8= +github.com/mattn/go-isatty v0.0.5/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= +github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE= +github.com/modelcontextprotocol/go-sdk v1.6.1 h1:0zOSupjKUxPKSocPT1Wtago+mUHU2/uZ4xSOY0FGReU= +github.com/modelcontextprotocol/go-sdk v1.6.1/go.mod h1:kzm3kzFL1/+AziGOE0nUs3gvPoNxMCvkxokMkuFapXQ= +github.com/natefinch/atomic v1.0.1 h1:ZPYKxkqQOx3KZ+RsbnP/YsgvxWQPGxjC0oBt2AhwV0A= +github.com/natefinch/atomic v1.0.1/go.mod h1:N/D/ELrljoqDyT3rZrsUmtsuzvHkeB/wWjHV22AZRbM= +github.com/networkteam/refresh v1.15.0 h1:8qobXuU29Ic08WkJC+lt4GHtoe9gDHTSEmgBqG5S0rg= +github.com/networkteam/refresh v1.15.0/go.mod h1:J6iKbX8RO9eERGJ5yYm0gXQLy5MrsgtwiQJF7K80uoE= +github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= +github.com/onsi/gomega v1.5.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY= +github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= +github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/r3labs/sse/v2 v2.10.0 h1:hFEkLLFY4LDifoHdiCN/LlGBAdVJYsANaLqNYa1l/v0= +github.com/r3labs/sse/v2 v2.10.0/go.mod h1:Igau6Whc+F17QUgML1fYe1VPZzTV6EMCnYktEmkNJ7I= +github.com/rjeczalik/notify v0.9.3 h1:6rJAzHTGKXGj76sbRgDiDcYj/HniypXmSJo1SWakZeY= +github.com/rjeczalik/notify v0.9.3/go.mod h1:gF3zSOrafR9DQEWSE8TjfI9NkooDxbyT4UgRGKZA0lc= +github.com/rogpeppe/fastuuid v1.1.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ= +github.com/rogpeppe/go-internal v1.11.0 h1:cWPaGQEPrBb5/AsnsZesgZZ9yb1OQ+GOISoDNXVBh4M= +github.com/rogpeppe/go-internal v1.11.0/go.mod h1:ddIwULY96R17DhadqLgMfk9H9tvdUzkipdSkR5nkCZA= +github.com/rs/cors v1.11.1 h1:eU3gRzXLRK57F5rKMGMZURNdIG4EoAmX8k94r9wXWHA= +github.com/rs/cors v1.11.1/go.mod h1:XyqrcTp5zjWr1wsJ8PIRZssZ8b/WMcMf71DJnit4EMU= +github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= +github.com/samber/lo v1.50.0 h1:XrG0xOeHs+4FQ8gJR97zDz5uOFMW7OwFWiFVzqopKgY= +github.com/samber/lo v1.50.0/go.mod h1:RjZyNk6WSnUFRKK6EyOhsRJMqft3G+pg7dCWHQCWvsc= +github.com/segmentio/asm v1.1.3 h1:WM03sfUOENvvKexOLp+pCqgb/WDjsi7EK8gIsICtzhc= +github.com/segmentio/asm v1.1.3/go.mod h1:Ld3L4ZXGNcSLRg4JBsZ3//1+f/TjYl0Mzen/DQy1EJg= +github.com/segmentio/encoding v0.5.4 h1:OW1VRern8Nw6ITAtwSZ7Idrl3MXCFwXHPgqESYfvNt0= +github.com/segmentio/encoding v0.5.4/go.mod h1:HS1ZKa3kSN32ZHVZ7ZLPLXWvOVIiZtyJnO1gPH1sKt0= +github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo= +github.com/smartystreets/assertions v1.0.0/go.mod h1:kHHU4qYBaI3q23Pp3VPrmWhuIUrLW/7eUrw0BU5VaoM= +github.com/smartystreets/go-aws-auth v0.0.0-20180515143844-0c1422d1fdb9/go.mod h1:SnhjPscd9TpLiy1LpzGSKh3bXCfxxXuqd9xmQJy3slM= +github.com/smartystreets/gunit v1.0.0/go.mod h1:qwPWnhz6pn0NnRBP++URONOVyNkPyr4SauJk4cUOwJs= +github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo= +github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0= +github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o= +github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= +github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= +github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +github.com/tj/assert v0.0.0-20171129193455-018094318fb0/go.mod h1:mZ9/Rh9oLWpLLDRpvE+3b7gP/C2YyLFYxNmcLnPTMe0= +github.com/tj/assert v0.0.3 h1:Df/BlaZ20mq6kuai7f5z2TvPFiwC3xaWJSDQNiIS3Rk= +github.com/tj/assert v0.0.3/go.mod h1:Ne6X72Q+TB1AteidzQncjw9PabbMp4PBMZ1k+vd1Pvk= +github.com/tj/go-buffer v1.1.0/go.mod h1:iyiJpfFcR2B9sXu7KvjbT9fpM4mOelRSDTbntVj52Uc= +github.com/tj/go-elastic v0.0.0-20171221160941-36157cbbebc2/go.mod h1:WjeM0Oo1eNAjXGDx2yma7uG2XoyRZTq1uv3M/o7imD0= +github.com/tj/go-kinesis v0.0.0-20171128231115-08b17f58cb1b/go.mod h1:/yhzCV0xPfx6jb1bBgRFjl5lytqVqZXEaeqWP8lTEao= +github.com/tj/go-spin v1.1.0/go.mod h1:Mg1mzmePZm4dva8Qz60H2lHwmJ2loum4VIrLgVnKwh4= +github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU= +github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E= +github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4= +github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/crypto v0.0.0-20190426145343-a29dc8fdc734/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/mod v0.33.0 h1:tHFzIWbBifEmbwtGz65eaWyGiGZatSrT9prnU8DbVL8= +golang.org/x/mod v0.33.0/go.mod h1:swjeQEj+6r7fODbD2cqrnje9PnziFuw4bmLbBZFrQ5w= +golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20191116160921-f9c825593386/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.50.0 h1:ucWh9eiCGyDR3vtzso0WMQinm2Dnt8cFMuQa9K33J60= +golang.org/x/net v0.50.0/go.mod h1:UgoSli3F/pBgdJBHCTc+tp3gmrU4XswgGRgtnwWTfyM= +golang.org/x/oauth2 v0.35.0 h1:Mv2mzuHuZuY2+bkyWXIHMfhNdJAdwW3FuWeCPYN5GVQ= +golang.org/x/oauth2 v0.35.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= +golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4= +golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20180926160741-c2ed4eda69e7/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw= +golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= +golang.org/x/term v0.44.0 h1:0rLvDRCtNj0gZkyIXhCyOb2OAzEhLVqc4B+hrsBhrmc= +golang.org/x/term v0.44.0/go.mod h1:7ze4MdzUzLXpSAoFP1H0bOI9aXDqveSvatT5vKcFh2Y= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= +golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk= +golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA= +golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.42.0 h1:uNgphsn75Tdz5Ji2q36v/nsFSfR/9BRFvqhGBaJGd5k= +golang.org/x/tools v0.42.0/go.mod h1:Ma6lCIwGZvHK6XtgbswSoWroEkhugApmsXyrUmBhfr0= +gopkg.in/cenkalti/backoff.v1 v1.1.0 h1:Arh75ttbsvlpVA7WtVpH4u9h6Zl46xuptxqLxPiSo4Y= +gopkg.in/cenkalti/backoff.v1 v1.1.0/go.mod h1:J6Vskwqd+OMVJl8C33mmtxTBs2gyzfv7UDAkHu8BrjI= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= +gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= +gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= +gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.0-20200605160147-a5ece683394c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/cli/main.go b/cli/main.go new file mode 100644 index 0000000..aacbcfe --- /dev/null +++ b/cli/main.go @@ -0,0 +1,57 @@ +// Command devlog is a CLI that exposes a local devlog instance's Agent JSON API +// to AI agents over the Model Context Protocol. +package main + +import ( + "errors" + "fmt" + "os" + "os/signal" + "syscall" + + "github.com/spf13/cobra" +) + +func main() { + if err := rootCmd().Execute(); err != nil { + fmt.Fprintln(os.Stderr, "error:", err) + os.Exit(1) + } +} + +func rootCmd() *cobra.Command { + root := &cobra.Command{ + Use: "devlog", + Short: "devlog agent relay", + SilenceUsage: true, + SilenceErrors: true, + } + root.AddCommand(relayCmd()) + return root +} + +func relayCmd() *cobra.Command { + var direct string + var session string + var mcpPort int + + cmd := &cobra.Command{ + Use: "relay", + Short: "Serve a devlog instance to an AI agent over MCP", + RunE: func(cmd *cobra.Command, _ []string) error { + if direct == "" { + return errors.New("--direct is required (browser relay mode is Phase 2)") + } + + ctx, stop := signal.NotifyContext(cmd.Context(), os.Interrupt, syscall.SIGTERM) + defer stop() + + return runRelayDirect(ctx, direct, session, mcpPort) + }, + } + + cmd.Flags().StringVar(&direct, "direct", "", "base URL of a local devlog dashboard, e.g. http://localhost:1095/_devlog") + cmd.Flags().StringVar(&session, "session", "", "capture session id to attach to (default: auto-select or prompt)") + cmd.Flags().IntVar(&mcpPort, "mcp-port", 0, "MCP server port on 127.0.0.1 (0 = random)") + return cmd +} diff --git a/cli/mcp_test.go b/cli/mcp_test.go new file mode 100644 index 0000000..36a8425 --- /dev/null +++ b/cli/mcp_test.go @@ -0,0 +1,286 @@ +package main + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "github.com/modelcontextprotocol/go-sdk/mcp" + + "github.com/networkteam/devlog/dashboard" +) + +// connectMCP wires an in-memory MCP client to a server backed by reg. +func connectMCP(t *testing.T, reg *registry) *mcp.ClientSession { + t.Helper() + ctx := context.Background() + + server := newMCPServer(reg) + serverTransport, clientTransport := mcp.NewInMemoryTransports() + if _, err := server.Connect(ctx, serverTransport, nil); err != nil { + t.Fatalf("server connect: %v", err) + } + client := mcp.NewClient(&mcp.Implementation{Name: "test", Version: "0"}, nil) + cs, err := client.Connect(ctx, clientTransport, nil) + if err != nil { + t.Fatalf("client connect: %v", err) + } + t.Cleanup(func() { _ = cs.Close() }) + return cs +} + +func toolText(t *testing.T, res *mcp.CallToolResult) string { + t.Helper() + for _, c := range res.Content { + if tc, ok := c.(*mcp.TextContent); ok { + if res.IsError { + t.Fatalf("tool returned error: %s", tc.Text) + } + return tc.Text + } + } + t.Fatal("no text content in tool result") + return "" +} + +// firstText returns the first text content without failing on IsError, for tests +// that assert on error results. +func firstText(res *mcp.CallToolResult) string { + for _, c := range res.Content { + if tc, ok := c.(*mcp.TextContent); ok { + return tc.Text + } + } + return "" +} + +func callTool(t *testing.T, cs *mcp.ClientSession, name string, args map[string]any) *mcp.CallToolResult { + t.Helper() + res, err := cs.CallTool(context.Background(), &mcp.CallToolParams{Name: name, Arguments: args}) + if err != nil { + t.Fatalf("CallTool %s: %v", name, err) + } + return res +} + +func TestMCP_ListTools(t *testing.T) { + baseURL, _ := newTestDevlog(t) + sid := startSession(t, baseURL, "global") + reg := newRegistry() + reg.add("local", newDirectClient(baseURL, sid)) + + cs := connectMCP(t, reg) + + res, err := cs.ListTools(context.Background(), nil) + if err != nil { + t.Fatalf("ListTools: %v", err) + } + got := make(map[string]bool) + for _, tl := range res.Tools { + got[tl.Name] = true + } + for _, want := range []string{"list_events", "get_event", "get_stats", "list_environments"} { + if !got[want] { + t.Errorf("missing tool %q", want) + } + } +} + +func TestMCP_ListAndGetEvent(t *testing.T) { + baseURL, appURL := newTestDevlog(t) + sid := startSession(t, baseURL, "global") + dc := newDirectClient(baseURL, sid) + reg := newRegistry() + reg.add("local", dc) + + cs := connectMCP(t, reg) + ctx := context.Background() + + // Capture is already on (started by the simulated dashboard); drive traffic. + resp, err := http.Get(appURL) + if err != nil { + t.Fatalf("app request: %v", err) + } + resp.Body.Close() + + // list_events via MCP. + listRes, err := cs.CallTool(ctx, &mcp.CallToolParams{Name: "list_events", Arguments: map[string]any{}}) + if err != nil { + t.Fatalf("CallTool list_events: %v", err) + } + var summaries []map[string]any + if err := json.Unmarshal([]byte(toolText(t, listRes)), &summaries); err != nil { + t.Fatalf("decode summaries: %v", err) + } + if len(summaries) == 0 { + t.Fatal("expected at least one event") + } + id, _ := summaries[0]["id"].(string) + if id == "" { + t.Fatal("expected an event id") + } + + // get_event via MCP. + getRes, err := cs.CallTool(ctx, &mcp.CallToolParams{Name: "get_event", Arguments: map[string]any{"eventId": id}}) + if err != nil { + t.Fatalf("CallTool get_event: %v", err) + } + detailText := toolText(t, getRes) + var detail map[string]any + if err := json.Unmarshal([]byte(detailText), &detail); err != nil { + t.Fatalf("decode detail: %v", err) + } + if detail["id"] != id { + t.Errorf("detail id mismatch: got %v want %s", detail["id"], id) + } + + // Structured content matches the direct API payload. + raw, err := dc.getEvent(ctx, id) + if err != nil { + t.Fatalf("direct getEvent: %v", err) + } + var apiDetail map[string]any + if err := json.Unmarshal(raw, &apiDetail); err != nil { + t.Fatalf("decode api detail: %v", err) + } + if apiDetail["id"] != detail["id"] || apiDetail["type"] != detail["type"] { + t.Errorf("tool payload does not match API payload") + } +} + +func TestMCP_CaptureStatus_ReadOnly(t *testing.T) { + baseURL, appURL := newTestDevlog(t) + sid := startSession(t, baseURL, "global") + reg := newRegistry() + reg.add("local", newDirectClient(baseURL, sid)) + cs := connectMCP(t, reg) + + resp, err := http.Get(appURL) + if err != nil { + t.Fatalf("app request: %v", err) + } + resp.Body.Close() + + statusRes := callTool(t, cs, "capture_status", map[string]any{}) + var status map[string]any + if err := json.Unmarshal([]byte(toolText(t, statusRes)), &status); err != nil { + t.Fatalf("decode status: %v", err) + } + if status["active"] != true || status["mode"] != "global" { + t.Errorf("unexpected status: %v", status) + } + if n, _ := status["eventCount"].(float64); n < 1 { + t.Errorf("expected eventCount >= 1, got %v", status["eventCount"]) + } + + // Capture control tools are not exposed to the agent. + tools, err := cs.ListTools(context.Background(), nil) + if err != nil { + t.Fatalf("ListTools: %v", err) + } + for _, tl := range tools.Tools { + if tl.Name == "capture_start" || tl.Name == "capture_stop" { + t.Errorf("agent must not expose %q", tl.Name) + } + } +} + +func TestMCP_GetRequestBody(t *testing.T) { + baseURL, appURL := newTestDevlog(t) + sid := startSession(t, baseURL, "global") + dc := newDirectClient(baseURL, sid) + reg := newRegistry() + reg.add("local", dc) + cs := connectMCP(t, reg) + + // POST with a body so a request body is captured. + payload := `{"hello":"body"}` + resp, err := http.Post(appURL, "application/json", strings.NewReader(payload)) + if err != nil { + t.Fatalf("app post: %v", err) + } + resp.Body.Close() + + listRes := callTool(t, cs, "list_events", map[string]any{}) + var summaries []map[string]any + if err := json.Unmarshal([]byte(toolText(t, listRes)), &summaries); err != nil { + t.Fatalf("decode summaries: %v", err) + } + if len(summaries) == 0 { + t.Fatal("expected a captured event") + } + id, _ := summaries[0]["id"].(string) + + bodyRes := callTool(t, cs, "get_request_body", map[string]any{"eventId": id}) + got := toolText(t, bodyRes) + if got != payload { + t.Errorf("request body mismatch: got %q want %q", got, payload) + } +} + +func TestMCP_GetBody_Redacted(t *testing.T) { + redactor := func(d *dashboard.EventDetail) *dashboard.EventDetail { + if d.RequestBody != nil { + d.RequestBody.Available = false // suppress request bodies + } + return d + } + baseURL, appURL := newTestDevlogWith(t, dashboard.WithAgentRedactor(redactor)) + sid := startSession(t, baseURL, "global") + dc := newDirectClient(baseURL, sid) + reg := newRegistry() + reg.add("local", dc) + cs := connectMCP(t, reg) + + resp, err := http.Post(appURL, "application/json", strings.NewReader(`{"secret":"x"}`)) + if err != nil { + t.Fatalf("app post: %v", err) + } + resp.Body.Close() + + listRes := callTool(t, cs, "list_events", map[string]any{}) + var summaries []map[string]any + if err := json.Unmarshal([]byte(toolText(t, listRes)), &summaries); err != nil { + t.Fatalf("decode summaries: %v", err) + } + if len(summaries) == 0 { + t.Fatal("expected a captured event") + } + id, _ := summaries[0]["id"].(string) + + bodyRes := callTool(t, cs, "get_request_body", map[string]any{"eventId": id}) + if !bodyRes.IsError { + t.Fatalf("expected an error result for a redacted body, got: %s", firstText(bodyRes)) + } + if msg := firstText(bodyRes); !strings.Contains(msg, "redacted") && !strings.Contains(msg, "410") { + t.Errorf("expected redaction message, got %q", msg) + } +} + +func TestMCP_HostValidation(t *testing.T) { + guarded := loopbackHostGuard(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusOK) + })) + + tests := []struct { + host string + want int + }{ + {"evil.example.com", http.StatusForbidden}, + {"127.0.0.1:4711", http.StatusOK}, + {"localhost:4711", http.StatusOK}, + {"[::1]:4711", http.StatusOK}, + } + for _, tc := range tests { + req := httptest.NewRequest(http.MethodPost, "http://example/mcp", nil) + req.Host = tc.host + rec := httptest.NewRecorder() + guarded.ServeHTTP(rec, req) + if rec.Code != tc.want { + t.Errorf("host %q: got %d, want %d", tc.host, rec.Code, tc.want) + } + } +} diff --git a/cli/registry.go b/cli/registry.go new file mode 100644 index 0000000..eab5a1d --- /dev/null +++ b/cli/registry.go @@ -0,0 +1,74 @@ +package main + +import ( + "fmt" + "sort" + "strings" + "sync" +) + +// registry maps environment names to their direct clients. In --direct mode there +// is a single environment ("local"); the structure anticipates the multi-origin +// relay of Phase 2. +type registry struct { + mu sync.RWMutex + envs map[string]*directClient +} + +func newRegistry() *registry { + return ®istry{envs: make(map[string]*directClient)} +} + +func (r *registry) add(name string, c *directClient) { + r.mu.Lock() + defer r.mu.Unlock() + r.envs[name] = c +} + +// resolve selects the client for the named environment. An empty name resolves to +// the sole environment when exactly one is connected; otherwise it is an error +// listing the available environments. +func (r *registry) resolve(name string) (*directClient, error) { + r.mu.RLock() + defer r.mu.RUnlock() + + if name == "" { + if len(r.envs) == 1 { + for _, c := range r.envs { + return c, nil + } + } + return nil, fmt.Errorf("environment is required; connected: %s", strings.Join(r.namesLocked(), ", ")) + } + + c, ok := r.envs[name] + if !ok { + return nil, fmt.Errorf("unknown environment %q; connected: %s", name, strings.Join(r.namesLocked(), ", ")) + } + return c, nil +} + +type environmentInfo struct { + Name string `json:"name"` + Session string `json:"session"` +} + +func (r *registry) list() []environmentInfo { + r.mu.RLock() + defer r.mu.RUnlock() + out := make([]environmentInfo, 0, len(r.envs)) + for name, c := range r.envs { + out = append(out, environmentInfo{Name: name, Session: c.sid}) + } + sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name }) + return out +} + +func (r *registry) namesLocked() []string { + names := make([]string, 0, len(r.envs)) + for name := range r.envs { + names = append(names, name) + } + sort.Strings(names) + return names +} diff --git a/cli/relay.go b/cli/relay.go new file mode 100644 index 0000000..1875cf6 --- /dev/null +++ b/cli/relay.go @@ -0,0 +1,83 @@ +package main + +import ( + "context" + "fmt" + "net" + "net/http" + "strconv" + + "github.com/modelcontextprotocol/go-sdk/mcp" +) + +const relayVersion = "0.1.0" + +// newMCPServer builds the MCP server and registers the read tools against reg. +func newMCPServer(reg *registry) *mcp.Server { + server := mcp.NewServer(&mcp.Implementation{Name: "devlog-agent", Version: relayVersion}, nil) + registerTools(server, reg) + return server +} + +// runRelayDirect attaches to an existing devlog capture session and serves the +// MCP endpoint on loopback, forwarding to the local devlog instance at baseURL. +// It blocks until ctx is canceled. sid may be empty to select interactively. +func runRelayDirect(ctx context.Context, baseURL, sid string, mcpPort int) error { + sid, err := selectSession(ctx, baseURL, sid) + if err != nil { + return err + } + + reg := newRegistry() + reg.add("local", newDirectClient(baseURL, sid)) + + server := newMCPServer(reg) + mcpHandler := mcp.NewStreamableHTTPHandler(func(*http.Request) *mcp.Server { return server }, nil) + guarded := loopbackHostGuard(mcpHandler) + + mux := http.NewServeMux() + mux.Handle("/mcp", guarded) + mux.Handle("/mcp/", guarded) + + ln, err := net.Listen("tcp", net.JoinHostPort("127.0.0.1", strconv.Itoa(mcpPort))) + if err != nil { + return err + } + + fmt.Printf("devlog relay (direct) → %s (session %s)\n", baseURL, sid) + fmt.Printf("MCP endpoint: http://%s/mcp\n", ln.Addr()) + + srv := &http.Server{Handler: mux} + go func() { + <-ctx.Done() + _ = srv.Close() + }() + if err := srv.Serve(ln); err != nil && err != http.ErrServerClosed { + return err + } + return nil +} + +// loopbackHostGuard rejects requests whose Host header is not a loopback literal. +// This blocks DNS-rebinding attacks from a browser against the local MCP endpoint. +func loopbackHostGuard(next http.Handler) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if !isLoopbackHost(r.Host) { + http.Error(w, "forbidden: non-loopback host", http.StatusForbidden) + return + } + next.ServeHTTP(w, r) + }) +} + +func isLoopbackHost(host string) bool { + h, _, err := net.SplitHostPort(host) + if err != nil { + h = host // no port present + } + if h == "localhost" { + return true + } + ip := net.ParseIP(h) + return ip != nil && ip.IsLoopback() +} diff --git a/cli/session.go b/cli/session.go new file mode 100644 index 0000000..f87c275 --- /dev/null +++ b/cli/session.go @@ -0,0 +1,81 @@ +package main + +import ( + "bufio" + "context" + "fmt" + "os" + "strconv" + "strings" + "time" + + "golang.org/x/term" +) + +const sessionPollInterval = 2 * time.Second + +// selectSession resolves the capture session to attach to. An explicit sid is +// used as-is. Otherwise: a single active session is auto-selected, several prompt +// interactively, and none triggers a wait until one appears. +func selectSession(ctx context.Context, baseURL, sid string) (string, error) { + if sid != "" { + return sid, nil + } + + httpc := defaultHTTPClient() + waited := false + for { + sessions, err := fetchSessions(ctx, baseURL, httpc) + if err != nil { + return "", fmt.Errorf("listing devlog sessions (is the dashboard running with the agent API enabled?): %w", err) + } + + switch { + case len(sessions) == 1: + s := sessions[0] + fmt.Printf("Attaching to devlog session %s (%s, %d events)\n", s.SessionID, s.Mode, s.EventCount) + return s.SessionID, nil + case len(sessions) > 1: + return promptSession(sessions) + default: + if !waited { + fmt.Println("No active devlog session. Open the dashboard and start capturing — waiting…") + waited = true + } + select { + case <-ctx.Done(): + return "", ctx.Err() + case <-time.After(sessionPollInterval): + } + } + } +} + +// promptSession asks the user to choose among several active sessions. Without a +// terminal it errors and asks for an explicit --session. +func promptSession(sessions []sessionInfo) (string, error) { + if !term.IsTerminal(int(os.Stdin.Fd())) { + return "", fmt.Errorf("%d active sessions; re-run with --session to choose", len(sessions)) + } + + fmt.Println("Multiple active devlog sessions:") + for i, s := range sessions { + fmt.Printf(" [%d] %s (%s, %d events, active %ds ago)\n", + i+1, s.SessionID, s.Mode, s.EventCount, s.LastActiveMsAgo/1000) + } + + reader := bufio.NewReader(os.Stdin) + for { + fmt.Printf("Attach to which session? [1-%d]: ", len(sessions)) + line, err := reader.ReadString('\n') + if err != nil { + return "", err + } + n, err := strconv.Atoi(strings.TrimSpace(line)) + if err != nil || n < 1 || n > len(sessions) { + fmt.Println("Please enter a valid number.") + continue + } + return sessions[n-1].SessionID, nil + } +} diff --git a/cli/tools.go b/cli/tools.go new file mode 100644 index 0000000..878eac9 --- /dev/null +++ b/cli/tools.go @@ -0,0 +1,208 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "strings" + + "github.com/modelcontextprotocol/go-sdk/mcp" +) + +// Tool input types. Each read tool accepts an optional environment selector. + +type listEventsInput struct { + Environment string `json:"environment,omitempty" jsonschema:"target environment; optional when only one is connected"` + Type string `json:"type,omitempty" jsonschema:"filter by event type: http_server, http_client, db_query or log"` + Since string `json:"since,omitempty" jsonschema:"only events at or after this RFC3339 timestamp"` + Until string `json:"until,omitempty" jsonschema:"only events at or before this RFC3339 timestamp"` + Status string `json:"status,omitempty" jsonschema:"filter by HTTP status class (e.g. 5xx) or exact code"` + Path string `json:"path,omitempty" jsonschema:"substring match on request path or URL"` + Limit int `json:"limit,omitempty" jsonschema:"maximum number of events to return"` +} + +type getEventInput struct { + Environment string `json:"environment,omitempty" jsonschema:"target environment; optional when only one is connected"` + EventID string `json:"eventId" jsonschema:"the id of the event to fetch"` +} + +type statsInput struct { + Environment string `json:"environment,omitempty" jsonschema:"target environment; optional when only one is connected"` +} + +type bodyInput struct { + Environment string `json:"environment,omitempty" jsonschema:"target environment; optional when only one is connected"` + EventID string `json:"eventId" jsonschema:"the id of the event whose body to fetch"` +} + +type captureControlInput struct { + Environment string `json:"environment,omitempty" jsonschema:"target environment; optional when only one is connected"` +} + +type listEnvironmentsInput struct{} + +// registerTools attaches the read-only MCP tools to the server. +func registerTools(server *mcp.Server, reg *registry) { + mcp.AddTool(server, &mcp.Tool{ + Name: "list_events", + Description: "List captured devlog events (newest first) with optional filters.", + }, reg.toolListEvents) + + mcp.AddTool(server, &mcp.Tool{ + Name: "get_event", + Description: "Get the full detail of a single devlog event, including child events.", + }, reg.toolGetEvent) + + mcp.AddTool(server, &mcp.Tool{ + Name: "get_stats", + Description: "Get devlog memory and event statistics.", + }, reg.toolGetStats) + + mcp.AddTool(server, &mcp.Tool{ + Name: "get_request_body", + Description: "Fetch the captured request body of an event (subject to redaction and a size cap).", + }, reg.toolGetRequestBody) + + mcp.AddTool(server, &mcp.Tool{ + Name: "get_response_body", + Description: "Fetch the captured response body of an event (subject to redaction and a size cap).", + }, reg.toolGetResponseBody) + + mcp.AddTool(server, &mcp.Tool{ + Name: "capture_status", + Description: "Report whether capture is active, the mode, and the captured event count. Capture is started and stopped by the user in the devlog dashboard, not by the agent.", + }, reg.toolCaptureStatus) + + mcp.AddTool(server, &mcp.Tool{ + Name: "list_environments", + Description: "List the devlog environments this relay is connected to.", + }, reg.toolListEnvironments) +} + +func (r *registry) toolListEvents(ctx context.Context, _ *mcp.CallToolRequest, in listEventsInput) (*mcp.CallToolResult, any, error) { + c, err := r.resolve(in.Environment) + if err != nil { + return errorResult(err), nil, nil + } + raw, err := c.listEvents(ctx, in) + if err != nil { + return errorResult(err), nil, nil + } + return jsonResult(raw), nil, nil +} + +func (r *registry) toolGetEvent(ctx context.Context, _ *mcp.CallToolRequest, in getEventInput) (*mcp.CallToolResult, any, error) { + c, err := r.resolve(in.Environment) + if err != nil { + return errorResult(err), nil, nil + } + raw, err := c.getEvent(ctx, in.EventID) + if err != nil { + return errorResult(err), nil, nil + } + return jsonResult(raw), nil, nil +} + +func (r *registry) toolGetStats(ctx context.Context, _ *mcp.CallToolRequest, in statsInput) (*mcp.CallToolResult, any, error) { + c, err := r.resolve(in.Environment) + if err != nil { + return errorResult(err), nil, nil + } + raw, err := c.getStats(ctx) + if err != nil { + return errorResult(err), nil, nil + } + return jsonResult(raw), nil, nil +} + +func (r *registry) toolGetRequestBody(ctx context.Context, _ *mcp.CallToolRequest, in bodyInput) (*mcp.CallToolResult, any, error) { + return r.getBody(ctx, in.Environment, in.EventID, "request") +} + +func (r *registry) toolGetResponseBody(ctx context.Context, _ *mcp.CallToolRequest, in bodyInput) (*mcp.CallToolResult, any, error) { + return r.getBody(ctx, in.Environment, in.EventID, "response") +} + +func (r *registry) getBody(ctx context.Context, environment, eventID, side string) (*mcp.CallToolResult, any, error) { + c, err := r.resolve(environment) + if err != nil { + return errorResult(err), nil, nil + } + br, err := c.getBody(ctx, eventID, side) + if err != nil { + return errorResult(err), nil, nil + } + return bodyToolResult(br), nil, nil +} + +func (r *registry) toolCaptureStatus(ctx context.Context, _ *mcp.CallToolRequest, in captureControlInput) (*mcp.CallToolResult, any, error) { + c, err := r.resolve(in.Environment) + if err != nil { + return errorResult(err), nil, nil + } + raw, err := c.captureStatus(ctx) + if err != nil { + return errorResult(err), nil, nil + } + return jsonResult(raw), nil, nil +} + +func (r *registry) toolListEnvironments(_ context.Context, _ *mcp.CallToolRequest, _ listEnvironmentsInput) (*mcp.CallToolResult, any, error) { + data, err := json.Marshal(r.list()) + if err != nil { + return errorResult(err), nil, nil + } + return jsonResult(data), nil, nil +} + +// bodyToolResult renders a fetched body as MCP content: textual bodies inline as +// text; binary bodies return a descriptive note instead of raw bytes so we don't +// dump binary into the model. +func bodyToolResult(br *bodyResult) *mcp.CallToolResult { + if isTextualContentType(br.contentType) { + text := string(br.data) + if br.truncated { + text += "\n\n[devlog: body truncated to the configured size cap]" + } + return &mcp.CallToolResult{Content: []mcp.Content{&mcp.TextContent{Text: text}}} + } + note := fmt.Sprintf("binary body not inlined: content-type %q, %d bytes", br.contentType, len(br.data)) + if br.truncated { + note += " (truncated to the configured size cap)" + } + return &mcp.CallToolResult{Content: []mcp.Content{&mcp.TextContent{Text: note}}} +} + +func isTextualContentType(ct string) bool { + ct = strings.ToLower(strings.TrimSpace(ct)) + if ct == "" { + return true // unknown/short bodies: treat as text + } + if i := strings.IndexByte(ct, ';'); i >= 0 { + ct = strings.TrimSpace(ct[:i]) + } + switch { + case strings.HasPrefix(ct, "text/"): + return true + case strings.Contains(ct, "json"), + strings.Contains(ct, "xml"), + strings.Contains(ct, "x-www-form-urlencoded"), + strings.Contains(ct, "graphql"): + return true + default: + return false + } +} + +func jsonResult(raw json.RawMessage) *mcp.CallToolResult { + return &mcp.CallToolResult{ + Content: []mcp.Content{&mcp.TextContent{Text: string(raw)}}, + } +} + +func errorResult(err error) *mcp.CallToolResult { + return &mcp.CallToolResult{ + IsError: true, + Content: []mcp.Content{&mcp.TextContent{Text: err.Error()}}, + } +} diff --git a/dashboard/agent_api.go b/dashboard/agent_api.go new file mode 100644 index 0000000..02d235d --- /dev/null +++ b/dashboard/agent_api.go @@ -0,0 +1,476 @@ +package dashboard + +import ( + "encoding/json" + "net/http" + "slices" + "strconv" + "strings" + "time" + + "github.com/gofrs/uuid" + + "github.com/networkteam/devlog/collector" + "github.com/networkteam/devlog/dashboard/views" +) + +// defaultAgentListLimit is the fallback number of events returned by the list +// endpoint when no explicit limit is requested (capped at truncateAfter). +const defaultAgentListLimit = 100 + +// agentListEvents handles GET /api/agent/v1/s/{sid}/events +func (h *Handler) agentListEvents(w http.ResponseWriter, r *http.Request) { + sessionID, ok := h.getSessionID(r) + if !ok { + writeJSONError(w, http.StatusBadRequest, "invalid session id") + return + } + + storage := h.sessions.Get(sessionID) + if storage == nil { + writeJSONError(w, http.StatusNotFound, "no capture session for this id") + return + } + + // Keep the session alive while an agent polls it. + h.sessions.UpdateActivity(sessionID) + + filters, err := parseEventFilters(r.URL.Query(), h.truncateAfter) + if err != nil { + writeJSONError(w, http.StatusBadRequest, err.Error()) + return + } + + events := storage.GetEvents(h.truncateAfter) + slices.Reverse(events) // newest first + + summaries := make([]EventSummary, 0, len(events)) + for _, e := range events { + detail := h.redactDetail(mapEventDetail(e)) + if detail == nil { + continue // suppressed by redactor + } + if !filters.matches(detail) { + continue + } + summaries = append(summaries, summaryFromDetail(detail)) + if len(summaries) >= filters.limit { + break + } + } + + writeJSON(w, http.StatusOK, summaries) +} + +// agentEventDetail handles GET /api/agent/v1/s/{sid}/events/{eventId} +func (h *Handler) agentEventDetail(w http.ResponseWriter, r *http.Request) { + sessionID, ok := h.getSessionID(r) + if !ok { + writeJSONError(w, http.StatusBadRequest, "invalid session id") + return + } + + storage := h.sessions.Get(sessionID) + if storage == nil { + writeJSONError(w, http.StatusNotFound, "no capture session for this id") + return + } + + h.sessions.UpdateActivity(sessionID) + + eventID, err := uuid.FromString(r.PathValue("eventId")) + if err != nil { + writeJSONError(w, http.StatusBadRequest, "invalid event id") + return + } + + event, exists := storage.GetEvent(eventID) + if !exists { + writeJSONError(w, http.StatusNotFound, "event not found") + return + } + + detail := h.redactDetail(mapEventDetail(event)) + if detail == nil { + // Suppressed by redactor. + writeJSONError(w, http.StatusNotFound, "event not found") + return + } + + writeJSON(w, http.StatusOK, detail) +} + +// redactDetail marks body truncation, applies default header masking (unless +// disabled), runs the embedder's redactor, then normalizes body suppression. +// A nil return suppresses the event. +func (h *Handler) redactDetail(d *EventDetail) *EventDetail { + if d == nil { + return nil + } + markBodyTruncationRecursive(d, h.agentMaxBodyBytes) + if !h.agentInsecureHeaders { + maskHeadersRecursive(d, h.agentRedactedHeaders) + } + if h.agentRedactor != nil { + d = h.agentRedactor(d) + if d == nil { + return nil + } + } + normalizeBodyRedactionRecursive(d) + return d +} + +// markBodyTruncationRecursive flags each captured body whose size exceeds the cap, +// so the detail view reflects what a body fetch would return. +func markBodyTruncationRecursive(d *EventDetail, maxBytes uint64) { + if d == nil { + return + } + for _, m := range []*BodyMeta{d.RequestBody, d.ResponseBody} { + if m != nil && maxBytes > 0 && m.Size > maxBytes { + m.Truncated = true + } + } + for _, child := range d.Children { + markBodyTruncationRecursive(child, maxBytes) + } +} + +// normalizeBodyRedactionRecursive reconciles the two ways a redactor can suppress +// a body: a non-nil BodyMeta with Available=false is treated as redacted, so the +// detail consistently shows redacted=true. (An absent body is represented by a +// nil BodyMeta, so this never mislabels "no body" as "redacted".) +func normalizeBodyRedactionRecursive(d *EventDetail) { + if d == nil { + return + } + for _, m := range []*BodyMeta{d.RequestBody, d.ResponseBody} { + if m != nil && !m.Available { + m.Redacted = true + } + } + for _, child := range d.Children { + normalizeBodyRedactionRecursive(child) + } +} + +// bodySide identifies which body a body endpoint serves. +type bodySide int + +const ( + requestBodySide bodySide = iota + responseBodySide +) + +func (h *Handler) agentRequestBody(w http.ResponseWriter, r *http.Request) { + h.agentBody(w, r, requestBodySide) +} + +func (h *Handler) agentResponseBody(w http.ResponseWriter, r *http.Request) { + h.agentBody(w, r, responseBodySide) +} + +// agentBody serves the raw request or response body for an event, honoring +// redaction and the configured size cap. +func (h *Handler) agentBody(w http.ResponseWriter, r *http.Request, side bodySide) { + sessionID, ok := h.getSessionID(r) + if !ok { + writeJSONError(w, http.StatusBadRequest, "invalid session id") + return + } + + storage := h.sessions.Get(sessionID) + if storage == nil { + writeJSONError(w, http.StatusNotFound, "no capture session for this id") + return + } + + h.sessions.UpdateActivity(sessionID) + + eventID, err := uuid.FromString(r.PathValue("eventId")) + if err != nil { + writeJSONError(w, http.StatusBadRequest, "invalid event id") + return + } + + event, exists := storage.GetEvent(eventID) + if !exists { + writeJSONError(w, http.StatusNotFound, "event not found") + return + } + + // Run the full redaction pipeline; a suppressed event is invisible. + detail := h.redactDetail(mapEventDetail(event)) + if detail == nil { + writeJSONError(w, http.StatusNotFound, "event not found") + return + } + + body, contentType, isBodyType, hasBody := extractBody(event.Data, side) + if !isBodyType { + writeJSONError(w, http.StatusBadRequest, "event type does not have a body") + return + } + if !hasBody { + writeJSONError(w, http.StatusNotFound, "no body available") + return + } + + meta := detail.RequestBody + if side == responseBodySide { + meta = detail.ResponseBody + } + if meta != nil && (meta.Redacted || !meta.Available) { + writeJSONError(w, http.StatusGone, "body redacted") + return + } + + if h.agentMaxBodyBytes > 0 && uint64(len(body)) > h.agentMaxBodyBytes { + body = body[:h.agentMaxBodyBytes] + w.Header().Set("X-Devlog-Truncated", "true") + } + + if contentType == "" { + contentType = "application/octet-stream" + } + w.Header().Set("Content-Type", contentType) + w.Header().Set("Content-Length", strconv.Itoa(len(body))) + w.WriteHeader(http.StatusOK) + _, _ = w.Write(body) +} + +// maskHeadersRecursive replaces the values of redacted headers with the sentinel +// across an EventDetail and all of its children, preserving the keys. +func maskHeadersRecursive(d *EventDetail, redacted map[string]bool) { + if d == nil { + return + } + maskHeader(d.RequestHeaders, redacted) + maskHeader(d.ResponseHeaders, redacted) + for _, child := range d.Children { + maskHeadersRecursive(child, redacted) + } +} + +func maskHeader(h http.Header, redacted map[string]bool) { + for key := range h { + if redacted[strings.ToLower(key)] { + h[key] = []string{redactedHeaderValue} + } + } +} + +// eventFilters holds the parsed list query parameters. +type eventFilters struct { + eventType string + since time.Time + until time.Time + statusFrom int // inclusive, 0 = unset + statusTo int // exclusive, 0 = unset + path string + limit int +} + +func parseEventFilters(q map[string][]string, truncateAfter uint64) (eventFilters, error) { + get := func(key string) string { + if vs := q[key]; len(vs) > 0 { + return vs[0] + } + return "" + } + + f := eventFilters{ + eventType: get("type"), + path: get("path"), + } + + maxLimit := int(truncateAfter) + if maxLimit <= 0 { + maxLimit = defaultAgentListLimit + } + f.limit = defaultAgentListLimit + if f.limit > maxLimit { + f.limit = maxLimit + } + if s := get("limit"); s != "" { + n, err := strconv.Atoi(s) + if err != nil || n < 0 { + return f, errInvalidParam("limit") + } + if n > 0 && n < maxLimit { + f.limit = n + } else { + f.limit = maxLimit + } + } + + if s := get("since"); s != "" { + t, err := time.Parse(time.RFC3339, s) + if err != nil { + return f, errInvalidParam("since") + } + f.since = t + } + if s := get("until"); s != "" { + t, err := time.Parse(time.RFC3339, s) + if err != nil { + return f, errInvalidParam("until") + } + f.until = t + } + + if s := get("status"); s != "" { + from, to, err := parseStatusClass(s) + if err != nil { + return f, err + } + f.statusFrom, f.statusTo = from, to + } + + return f, nil +} + +// parseStatusClass parses a status filter that is either a class like "5xx" or an +// exact code like "404", returning an inclusive-from/exclusive-to range. +func parseStatusClass(s string) (int, int, error) { + s = strings.ToLower(strings.TrimSpace(s)) + if len(s) == 3 && strings.HasSuffix(s, "xx") { + switch s[0] { + case '1', '2', '3', '4', '5': + base := int(s[0]-'0') * 100 + return base, base + 100, nil + } + return 0, 0, errInvalidParam("status") + } + n, err := strconv.Atoi(s) + if err != nil || n < 100 || n > 599 { + return 0, 0, errInvalidParam("status") + } + return n, n + 1, nil +} + +func (f eventFilters) matches(d *EventDetail) bool { + if f.eventType != "" && string(d.Type) != f.eventType { + return false + } + if !f.since.IsZero() && d.Start.Before(f.since) { + return false + } + if !f.until.IsZero() && d.Start.After(f.until) { + return false + } + if f.statusTo != 0 { + if d.StatusCode < f.statusFrom || d.StatusCode >= f.statusTo { + return false + } + } + if f.path != "" { + hay := d.Path + if hay == "" { + hay = d.URL + } + if !strings.Contains(hay, f.path) { + return false + } + } + return true +} + +// agentCaptureStatusResponse is the JSON body for the capture control endpoints. +type agentCaptureStatusResponse struct { + Active bool `json:"active"` + Mode string `json:"mode"` + EventCount int `json:"eventCount"` +} + +// agentCaptureStatus handles GET /api/agent/v1/s/{sid}/capture/status. +// +// Capture is started and stopped by the user in the dashboard UI; the agent only +// observes a user-managed session, so there is intentionally no agent start/stop. +func (h *Handler) agentCaptureStatus(w http.ResponseWriter, r *http.Request) { + sessionID, ok := h.getSessionID(r) + if !ok { + writeJSON(w, http.StatusOK, inactiveCaptureStatus()) + return + } + storage := h.sessions.Get(sessionID) + if storage == nil { + writeJSON(w, http.StatusOK, inactiveCaptureStatus()) + return + } + + writeJSON(w, http.StatusOK, agentCaptureStatusResponse{ + Active: storage.IsCapturing(), + Mode: storage.CaptureMode().String(), + EventCount: countEvents(storage), + }) +} + +// agentSessionInfo is one entry in the sessions listing. +type agentSessionInfo struct { + SessionID string `json:"sessionId"` + Mode string `json:"mode"` + Capturing bool `json:"capturing"` + EventCount int `json:"eventCount"` + LastActiveMsAgo int64 `json:"lastActiveMsAgo"` +} + +// agentListSessions handles GET /api/agent/v1/sessions - lists active capture +// sessions so an agent relay can attach to one rather than creating its own. +func (h *Handler) agentListSessions(w http.ResponseWriter, r *http.Request) { + now := time.Now() + infos := h.sessions.List() + out := make([]agentSessionInfo, 0, len(infos)) + for _, s := range infos { + out = append(out, agentSessionInfo{ + SessionID: s.SessionID.String(), + Mode: s.Mode.String(), + Capturing: s.Capturing, + EventCount: s.EventCount, + LastActiveMsAgo: now.Sub(s.LastActive).Milliseconds(), + }) + } + writeJSON(w, http.StatusOK, out) +} + +// agentStats handles GET /api/agent/v1/stats +func (h *Handler) agentStats(w http.ResponseWriter, r *http.Request) { + stats := h.eventAggregator.CalculateStats() + writeJSON(w, http.StatusOK, StatsResponse{ + MemoryBytes: stats.TotalMemory, + MemoryFormatted: views.FormatBytes(stats.TotalMemory), + SessionCount: h.sessions.SessionCount(), + MaxSessions: h.sessions.MaxSessions(), + EventCount: stats.EventCount, + }) +} + +func inactiveCaptureStatus() agentCaptureStatusResponse { + return agentCaptureStatusResponse{Active: false, Mode: collector.CaptureModeSession.String()} +} + +func countEvents(s *collector.CaptureStorage) int { + return len(s.GetEvents(^uint64(0))) +} + +// jsonError is the consistent error body for all agent endpoints. +type jsonError struct { + Error string `json:"error"` +} + +func writeJSON(w http.ResponseWriter, status int, v any) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(status) + _ = json.NewEncoder(w).Encode(v) +} + +func writeJSONError(w http.ResponseWriter, status int, msg string) { + writeJSON(w, status, jsonError{Error: msg}) +} + +type paramError struct{ name string } + +func (e paramError) Error() string { return "invalid parameter: " + e.name } + +func errInvalidParam(name string) error { return paramError{name: name} } diff --git a/dashboard/agent_api_test.go b/dashboard/agent_api_test.go new file mode 100644 index 0000000..cd87a20 --- /dev/null +++ b/dashboard/agent_api_test.go @@ -0,0 +1,579 @@ +package dashboard + +import ( + "bytes" + "encoding/json" + "io" + "log/slog" + "net/http" + "net/http/httptest" + "net/url" + "testing" + "time" + + "github.com/gofrs/uuid" + + "github.com/networkteam/devlog/collector" +) + +// newAgentTestHandler builds a Handler with the agent API enabled (plus any extra +// options) and a single capture session, returning the handler and the session id. +func newAgentTestHandler(t *testing.T, opts ...HandlerOption) (*Handler, uuid.UUID) { + t.Helper() + + aggregator := collector.NewEventAggregator() + all := append([]HandlerOption{WithAgentAPI(), WithStorageCapacity(100)}, opts...) + h := NewHandler(aggregator, all...) + t.Cleanup(h.Close) + + sid := uuid.Must(uuid.NewV4()) + if _, _, err := h.sessions.GetOrCreate(sid, collector.CaptureModeGlobal); err != nil { + t.Fatalf("GetOrCreate: %v", err) + } + return h, sid +} + +func addEvent(t *testing.T, h *Handler, sid uuid.UUID, e *collector.Event) { + t.Helper() + storage := h.sessions.Get(sid) + if storage == nil { + t.Fatal("no storage for session") + } + storage.Add(e) +} + +// --- fixture builders --- + +func httpServerEvent(start time.Time, method, path string, status int, reqHeaders http.Header) *collector.Event { + return &collector.Event{ + ID: uuid.Must(uuid.NewV4()), + Start: start, + End: start.Add(10 * time.Millisecond), + Data: collector.HTTPServerRequest{ + ID: uuid.Must(uuid.NewV4()), + Method: method, + Path: path, + URL: "http://example.test" + path, + StatusCode: status, + RequestHeaders: reqHeaders, + }, + } +} + +func dbQueryEvent(start time.Time, query string) *collector.Event { + return &collector.Event{ + ID: uuid.Must(uuid.NewV4()), + Start: start, + End: start.Add(2 * time.Millisecond), + Data: collector.DBQuery{ + Query: query, + Language: "postgresql", + }, + } +} + +// bodyWith builds a fully-captured collector.Body containing data, with a capture +// limit large enough to avoid capture-time truncation. +func bodyWith(data []byte) *collector.Body { + b := collector.NewBody(io.NopCloser(bytes.NewReader(data)), len(data)+16) + b.Close() // reads the reader into the buffer + return b +} + +// httpServerEventWithReqBody builds an HTTP server event carrying a request body. +func httpServerEventWithReqBody(start time.Time, path, contentType string, body []byte) *collector.Event { + headers := http.Header{} + if contentType != "" { + headers.Set("Content-Type", contentType) + } + return &collector.Event{ + ID: uuid.Must(uuid.NewV4()), + Start: start, + End: start.Add(5 * time.Millisecond), + Data: collector.HTTPServerRequest{ + ID: uuid.Must(uuid.NewV4()), + Method: "POST", + Path: path, + URL: "http://example.test" + path, + StatusCode: 200, + RequestHeaders: headers, + RequestBody: bodyWith(body), + }, + } +} + +func logEvent(start time.Time, level slog.Level, msg string) *collector.Event { + rec := slog.NewRecord(start, level, msg, 0) + rec.AddAttrs(slog.String("key", "value"), slog.Int("count", 3)) + return &collector.Event{ + ID: uuid.Must(uuid.NewV4()), + Start: start, + End: start, + Data: rec, + } +} + +// --- request helpers --- + +func doGET(t *testing.T, h *Handler, path string) *http.Response { + t.Helper() + req := httptest.NewRequest(http.MethodGet, path, nil) + rec := httptest.NewRecorder() + h.ServeHTTP(rec, req) + return rec.Result() +} + +func decodeSummaries(t *testing.T, resp *http.Response) []EventSummary { + t.Helper() + var out []EventSummary + if err := json.NewDecoder(resp.Body).Decode(&out); err != nil { + t.Fatalf("decode summaries: %v", err) + } + return out +} + +func decodeDetail(t *testing.T, resp *http.Response) EventDetail { + t.Helper() + var out EventDetail + if err := json.NewDecoder(resp.Body).Decode(&out); err != nil { + t.Fatalf("decode detail: %v", err) + } + return out +} + +func eventsURL(sid uuid.UUID) string { + return "/api/agent/v1/s/" + sid.String() + "/events" +} + +func capturePath(sid uuid.UUID, action string) string { + return "/api/agent/v1/s/" + sid.String() + "/capture/" + action +} + +func decodeCaptureStatus(t *testing.T, resp *http.Response) agentCaptureStatusResponse { + t.Helper() + var out agentCaptureStatusResponse + if err := json.NewDecoder(resp.Body).Decode(&out); err != nil { + t.Fatalf("decode capture status: %v", err) + } + return out +} + +// --- tests --- + +func TestAgentAPI_Disabled_ByDefault(t *testing.T) { + aggregator := collector.NewEventAggregator() + h := NewHandler(aggregator) // no WithAgentAPI + defer h.Close() + + sid := uuid.Must(uuid.NewV4()) + resp := doGET(t, h, eventsURL(sid)) + if resp.StatusCode != http.StatusNotFound { + t.Fatalf("expected 404 when agent API disabled, got %d", resp.StatusCode) + } +} + +func TestAgentAPI_Events_EmptySession(t *testing.T) { + aggregator := collector.NewEventAggregator() + h := NewHandler(aggregator, WithAgentAPI()) + defer h.Close() + + // Valid but unknown session id. + sid := uuid.Must(uuid.NewV4()) + resp := doGET(t, h, eventsURL(sid)) + if resp.StatusCode != http.StatusNotFound { + t.Fatalf("expected 404 for unknown session, got %d", resp.StatusCode) + } + var body jsonError + if err := json.NewDecoder(resp.Body).Decode(&body); err != nil { + t.Fatalf("decode error body: %v", err) + } + if body.Error == "" { + t.Fatal("expected non-empty error message") + } +} + +func TestAgentAPI_Events_ListsSummaries(t *testing.T) { + h, sid := newAgentTestHandler(t) + base := time.Now().Add(-time.Minute) + + addEvent(t, h, sid, httpServerEvent(base, "GET", "/a", 200, nil)) + addEvent(t, h, sid, dbQueryEvent(base.Add(time.Second), "SELECT 1")) + addEvent(t, h, sid, logEvent(base.Add(2*time.Second), slog.LevelInfo, "hello")) + + resp := doGET(t, h, eventsURL(sid)) + if resp.StatusCode != http.StatusOK { + t.Fatalf("expected 200, got %d", resp.StatusCode) + } + summaries := decodeSummaries(t, resp) + if len(summaries) != 3 { + t.Fatalf("expected 3 summaries, got %d", len(summaries)) + } + // Newest first: log, db, http. + wantTypes := []EventType{EventTypeLog, EventTypeDBQuery, EventTypeHTTPServer} + for i, want := range wantTypes { + if summaries[i].Type != want { + t.Errorf("summary[%d]: want type %s, got %s", i, want, summaries[i].Type) + } + } +} + +func TestAgentAPI_Events_FilterByType(t *testing.T) { + h, sid := newAgentTestHandler(t) + base := time.Now().Add(-time.Minute) + addEvent(t, h, sid, httpServerEvent(base, "GET", "/a", 200, nil)) + addEvent(t, h, sid, dbQueryEvent(base.Add(time.Second), "SELECT 1")) + addEvent(t, h, sid, logEvent(base.Add(2*time.Second), slog.LevelInfo, "hello")) + + resp := doGET(t, h, eventsURL(sid)+"?type=db_query") + summaries := decodeSummaries(t, resp) + if len(summaries) != 1 { + t.Fatalf("expected 1 db_query summary, got %d", len(summaries)) + } + if summaries[0].Type != EventTypeDBQuery { + t.Errorf("expected db_query, got %s", summaries[0].Type) + } +} + +func TestAgentAPI_Events_FilterByTimeAndLimit(t *testing.T) { + h, sid := newAgentTestHandler(t) + base := time.Now().Add(-time.Hour).Truncate(time.Second) + addEvent(t, h, sid, httpServerEvent(base, "GET", "/old", 200, nil)) + addEvent(t, h, sid, httpServerEvent(base.Add(10*time.Minute), "GET", "/mid", 200, nil)) + addEvent(t, h, sid, httpServerEvent(base.Add(20*time.Minute), "GET", "/new", 200, nil)) + + // since excludes the oldest; limit caps to a single (newest) result. + since := base.Add(5 * time.Minute).Format(time.RFC3339) + resp := doGET(t, h, eventsURL(sid)+"?since="+url.QueryEscape(since)+"&limit=1") + summaries := decodeSummaries(t, resp) + if len(summaries) != 1 { + t.Fatalf("expected 1 summary, got %d", len(summaries)) + } + if summaries[0].Path != "/new" { + t.Errorf("expected newest /new, got %s", summaries[0].Path) + } +} + +func TestAgentAPI_EventDetail_HTTPServerWithChildren(t *testing.T) { + h, sid := newAgentTestHandler(t) + base := time.Now().Add(-time.Minute) + + parent := httpServerEvent(base, "POST", "/checkout", 200, nil) + child := dbQueryEvent(base.Add(time.Millisecond), "SELECT * FROM orders") + parent.Children = []*collector.Event{child} + // Give the server event a request body so metadata appears. + srv := parent.Data.(collector.HTTPServerRequest) + srv.RequestHeaders = http.Header{"Content-Type": []string{"application/json"}} + srv.RequestBody = collector.NewBody(nil, 1024) + parent.Data = srv + addEvent(t, h, sid, parent) + + resp := doGET(t, h, eventsURL(sid)+"/"+parent.ID.String()) + if resp.StatusCode != http.StatusOK { + t.Fatalf("expected 200, got %d", resp.StatusCode) + } + detail := decodeDetail(t, resp) + if detail.Type != EventTypeHTTPServer { + t.Fatalf("expected http_server, got %s", detail.Type) + } + if len(detail.Children) != 1 || detail.Children[0].Type != EventTypeDBQuery { + t.Fatalf("expected one db_query child, got %+v", detail.Children) + } + if detail.Children[0].Query == "" { + t.Error("expected child query present") + } + if detail.DurationMs <= 0 { + t.Errorf("expected positive durationMs, got %d", detail.DurationMs) + } + if detail.RequestBody == nil || !detail.RequestBody.Available { + t.Error("expected request body metadata to be available") + } +} + +func TestAgentAPI_EventDetail_LogRecord(t *testing.T) { + h, sid := newAgentTestHandler(t) + e := logEvent(time.Now().Add(-time.Second), slog.LevelWarn, "something happened") + addEvent(t, h, sid, e) + + resp := doGET(t, h, eventsURL(sid)+"/"+e.ID.String()) + detail := decodeDetail(t, resp) + if detail.Type != EventTypeLog { + t.Fatalf("expected log, got %s", detail.Type) + } + if detail.Level != slog.LevelWarn.String() { + t.Errorf("expected level %s, got %s", slog.LevelWarn, detail.Level) + } + if detail.Message != "something happened" { + t.Errorf("unexpected message %q", detail.Message) + } + if detail.Attrs["key"] != "value" { + t.Errorf("expected attr key=value, got %v", detail.Attrs["key"]) + } +} + +func TestAgentAPI_DefaultHeaderRedaction(t *testing.T) { + h, sid := newAgentTestHandler(t) + headers := http.Header{} + headers.Set("Authorization", "Bearer secret-token") + headers.Set("Cookie", "session=abc") + headers.Set("X-Trace-Id", "trace-123") + e := httpServerEvent(time.Now().Add(-time.Second), "GET", "/secure", 200, headers) + addEvent(t, h, sid, e) + + resp := doGET(t, h, eventsURL(sid)+"/"+e.ID.String()) + detail := decodeDetail(t, resp) + + if got := detail.RequestHeaders.Get("Authorization"); got != redactedHeaderValue { + t.Errorf("Authorization not redacted: %q", got) + } + if got := detail.RequestHeaders.Get("Cookie"); got != redactedHeaderValue { + t.Errorf("Cookie not redacted: %q", got) + } + if got := detail.RequestHeaders.Get("X-Trace-Id"); got != "trace-123" { + t.Errorf("non-sensitive header should be preserved, got %q", got) + } + // Key preserved. + if _, ok := detail.RequestHeaders["Authorization"]; !ok { + t.Error("Authorization key should be preserved") + } +} + +func TestAgentAPI_RedactedHeaders_Extended(t *testing.T) { + h, sid := newAgentTestHandler(t, WithAgentRedactedHeaders("X-Api-Key")) + headers := http.Header{} + headers.Set("X-Api-Key", "key-123") + headers.Set("Authorization", "Bearer t") + e := httpServerEvent(time.Now().Add(-time.Second), "GET", "/x", 200, headers) + addEvent(t, h, sid, e) + + resp := doGET(t, h, eventsURL(sid)+"/"+e.ID.String()) + detail := decodeDetail(t, resp) + if got := detail.RequestHeaders.Get("X-Api-Key"); got != redactedHeaderValue { + t.Errorf("X-Api-Key not redacted: %q", got) + } + if got := detail.RequestHeaders.Get("Authorization"); got != redactedHeaderValue { + t.Errorf("default Authorization redaction lost: %q", got) + } +} + +func TestAgentAPI_InsecureHeaders_OptOut(t *testing.T) { + h, sid := newAgentTestHandler(t, WithAgentInsecureHeaders()) + headers := http.Header{} + headers.Set("Authorization", "Bearer visible") + e := httpServerEvent(time.Now().Add(-time.Second), "GET", "/x", 200, headers) + addEvent(t, h, sid, e) + + resp := doGET(t, h, eventsURL(sid)+"/"+e.ID.String()) + detail := decodeDetail(t, resp) + if got := detail.RequestHeaders.Get("Authorization"); got != "Bearer visible" { + t.Errorf("expected Authorization visible with insecure headers, got %q", got) + } +} + +func TestAgentAPI_Redactor_DropsEvent(t *testing.T) { + redactor := func(d *EventDetail) *EventDetail { + if d.Type == EventTypeHTTPServer && d.Path == "/auth/login" { + return nil + } + return d + } + h, sid := newAgentTestHandler(t, WithAgentRedactor(redactor)) + base := time.Now().Add(-time.Minute) + authEvent := httpServerEvent(base, "POST", "/auth/login", 200, nil) + okEvent := httpServerEvent(base.Add(time.Second), "GET", "/dashboard", 200, nil) + addEvent(t, h, sid, authEvent) + addEvent(t, h, sid, okEvent) + + // List omits the dropped event. + resp := doGET(t, h, eventsURL(sid)) + summaries := decodeSummaries(t, resp) + if len(summaries) != 1 { + t.Fatalf("expected 1 summary (auth dropped), got %d", len(summaries)) + } + if summaries[0].Path != "/dashboard" { + t.Errorf("expected /dashboard, got %s", summaries[0].Path) + } + + // Detail of the dropped event is 404. + detailResp := doGET(t, h, eventsURL(sid)+"/"+authEvent.ID.String()) + if detailResp.StatusCode != http.StatusNotFound { + t.Errorf("expected 404 for dropped event, got %d", detailResp.StatusCode) + } +} + +func TestAgentAPI_RequestBody_Raw(t *testing.T) { + h, sid := newAgentTestHandler(t) + payload := []byte(`{"hello":"world"}`) + e := httpServerEventWithReqBody(time.Now().Add(-time.Second), "/submit", "application/json", payload) + addEvent(t, h, sid, e) + + resp := doGET(t, h, eventsURL(sid)+"/"+e.ID.String()+"/request-body") + if resp.StatusCode != http.StatusOK { + t.Fatalf("expected 200, got %d", resp.StatusCode) + } + if ct := resp.Header.Get("Content-Type"); ct != "application/json" { + t.Errorf("expected application/json, got %q", ct) + } + got, _ := io.ReadAll(resp.Body) + if !bytes.Equal(got, payload) { + t.Errorf("body mismatch: got %q want %q", got, payload) + } + if resp.Header.Get("X-Devlog-Truncated") == "true" { + t.Error("did not expect truncation for a small body") + } +} + +func TestAgentAPI_RequestBody_Truncated(t *testing.T) { + const maxBytes = 8 + h, sid := newAgentTestHandler(t, WithAgentMaxBodyBytes(maxBytes)) + payload := []byte("0123456789ABCDEF") // 16 bytes > maxBytes + e := httpServerEventWithReqBody(time.Now().Add(-time.Second), "/big", "text/plain", payload) + addEvent(t, h, sid, e) + + // Body endpoint truncates and signals via header. + resp := doGET(t, h, eventsURL(sid)+"/"+e.ID.String()+"/request-body") + if resp.StatusCode != http.StatusOK { + t.Fatalf("expected 200, got %d", resp.StatusCode) + } + if resp.Header.Get("X-Devlog-Truncated") != "true" { + t.Error("expected X-Devlog-Truncated: true") + } + got, _ := io.ReadAll(resp.Body) + if len(got) != maxBytes || !bytes.Equal(got, payload[:maxBytes]) { + t.Errorf("expected first %d bytes, got %q", maxBytes, got) + } + + // Detail metadata reflects truncation. + detailResp := doGET(t, h, eventsURL(sid)+"/"+e.ID.String()) + detail := decodeDetail(t, detailResp) + if detail.RequestBody == nil || !detail.RequestBody.Truncated { + t.Errorf("expected requestBody.truncated=true, got %+v", detail.RequestBody) + } +} + +func TestAgentAPI_Redactor_RedactsBody(t *testing.T) { + redactor := func(d *EventDetail) *EventDetail { + if d.RequestBody != nil { + d.RequestBody.Available = false // suppress the body + } + return d + } + h, sid := newAgentTestHandler(t, WithAgentRedactor(redactor)) + e := httpServerEventWithReqBody(time.Now().Add(-time.Second), "/login", "application/json", []byte(`{"pw":"secret"}`)) + addEvent(t, h, sid, e) + + // Detail shows the body as redacted. + detailResp := doGET(t, h, eventsURL(sid)+"/"+e.ID.String()) + detail := decodeDetail(t, detailResp) + if detail.RequestBody == nil || !detail.RequestBody.Redacted { + t.Errorf("expected requestBody.redacted=true, got %+v", detail.RequestBody) + } + + // Body endpoint refuses with 410 Gone. + bodyResp := doGET(t, h, eventsURL(sid)+"/"+e.ID.String()+"/request-body") + if bodyResp.StatusCode != http.StatusGone { + t.Fatalf("expected 410 for redacted body, got %d", bodyResp.StatusCode) + } +} + +func TestAgentAPI_RequestBody_NotCaptured(t *testing.T) { + h, sid := newAgentTestHandler(t) + // HTTP event without any request body. + e := httpServerEvent(time.Now().Add(-time.Second), "GET", "/nobody", 200, nil) + addEvent(t, h, sid, e) + + resp := doGET(t, h, eventsURL(sid)+"/"+e.ID.String()+"/request-body") + if resp.StatusCode != http.StatusNotFound { + t.Fatalf("expected 404 for missing body, got %d", resp.StatusCode) + } +} + +func TestAgentAPI_RequestBody_NonBodyEventType(t *testing.T) { + h, sid := newAgentTestHandler(t) + e := dbQueryEvent(time.Now().Add(-time.Second), "SELECT 1") + addEvent(t, h, sid, e) + + resp := doGET(t, h, eventsURL(sid)+"/"+e.ID.String()+"/request-body") + if resp.StatusCode != http.StatusBadRequest { + t.Fatalf("expected 400 for non-body event type, got %d", resp.StatusCode) + } +} + +func TestAgentAPI_CaptureStatus_ReadOnly(t *testing.T) { + // The session is created by the dashboard (here via SessionManager directly); + // the agent only reads status — there is no agent start/stop. + h, sid := newAgentTestHandler(t) // creates a global-mode session + addEvent(t, h, sid, httpServerEvent(time.Now(), "GET", "/x", 200, nil)) + addEvent(t, h, sid, dbQueryEvent(time.Now(), "SELECT 1")) + + resp := doGET(t, h, capturePath(sid, "status")) + st := decodeCaptureStatus(t, resp) + if !st.Active || st.Mode != "global" || st.EventCount != 2 { + t.Fatalf("unexpected status: %+v", st) + } +} + +func TestAgentAPI_CaptureStartStop_NotExposed(t *testing.T) { + h, sid := newAgentTestHandler(t) + for _, action := range []string{"start", "stop"} { + req := httptest.NewRequest(http.MethodPost, capturePath(sid, action), nil) + rec := httptest.NewRecorder() + h.ServeHTTP(rec, req) + if rec.Result().StatusCode != http.StatusNotFound { + t.Errorf("capture/%s should not be exposed to agents, got %d", action, rec.Result().StatusCode) + } + } +} + +func TestAgentAPI_Stats(t *testing.T) { + h, sid := newAgentTestHandler(t) + addEvent(t, h, sid, httpServerEvent(time.Now(), "GET", "/a", 200, nil)) + addEvent(t, h, sid, dbQueryEvent(time.Now(), "SELECT 1")) + + resp := doGET(t, h, "/api/agent/v1/stats") + if resp.StatusCode != http.StatusOK { + t.Fatalf("expected 200, got %d", resp.StatusCode) + } + var sr StatsResponse + if err := json.NewDecoder(resp.Body).Decode(&sr); err != nil { + t.Fatalf("decode stats: %v", err) + } + + want := h.eventAggregator.CalculateStats() + if sr.EventCount != want.EventCount { + t.Errorf("eventCount: got %d want %d", sr.EventCount, want.EventCount) + } + if sr.EventCount != 2 { + t.Errorf("expected 2 events, got %d", sr.EventCount) + } + if sr.SessionCount != 1 { + t.Errorf("expected 1 session, got %d", sr.SessionCount) + } + if sr.MemoryFormatted == "" { + t.Error("expected a formatted memory string") + } +} + +func TestAgentAPI_KeepsSessionAlive(t *testing.T) { + aggregator := collector.NewEventAggregator() + h := NewHandler(aggregator, WithAgentAPI(), WithSessionIdleTimeout(60*time.Millisecond)) + defer h.Close() + + sid := uuid.Must(uuid.NewV4()) + if _, _, err := h.sessions.GetOrCreate(sid, collector.CaptureModeGlobal); err != nil { + t.Fatalf("GetOrCreate: %v", err) + } + + // Poll repeatedly across more than one idle-timeout window. + deadline := time.Now().Add(150 * time.Millisecond) + for time.Now().Before(deadline) { + resp := doGET(t, h, eventsURL(sid)) + if resp.StatusCode != http.StatusOK { + t.Fatalf("expected session to stay alive, got %d", resp.StatusCode) + } + time.Sleep(20 * time.Millisecond) + } + + if storage := h.sessions.Get(sid); storage == nil { + t.Fatal("session was cleaned up despite active polling") + } +} diff --git a/dashboard/agent_api_types.go b/dashboard/agent_api_types.go new file mode 100644 index 0000000..eddc702 --- /dev/null +++ b/dashboard/agent_api_types.go @@ -0,0 +1,287 @@ +package dashboard + +import ( + "database/sql/driver" + "log/slog" + "net/http" + "time" + + "github.com/networkteam/devlog/collector" +) + +// EventType is the agent-API discriminator for an event's payload. +type EventType string + +const ( + EventTypeHTTPServer EventType = "http_server" + EventTypeHTTPClient EventType = "http_client" + EventTypeDBQuery EventType = "db_query" + EventTypeLog EventType = "log" + EventTypeUnknown EventType = "unknown" +) + +// AgentRedactor is an embedder-supplied hook applied to each EventDetail before +// it is encoded for the agent API. It may mutate the detail in place and return +// it, or return nil to suppress the event entirely (omitted from lists, 404 on +// the detail endpoint). +type AgentRedactor func(*EventDetail) *EventDetail + +// BodyMeta describes a captured request or response body without inlining its +// bytes. Raw bytes are served only by the dedicated body endpoints (Task 2). +type BodyMeta struct { + Size uint64 `json:"size"` + ContentType string `json:"contentType,omitempty"` + // Available is true when a body was captured and can be fetched. + Available bool `json:"available"` + // Redacted is true when redaction has suppressed the body (distinct from + // Available=false because no body was captured). + Redacted bool `json:"redacted"` + // Truncated is true when the captured body is larger than the agent body + // size cap, so a fetch via the body endpoint returns only the first cap bytes. + Truncated bool `json:"truncated,omitempty"` +} + +// DBArg is a single database query argument. +type DBArg struct { + Name string `json:"name,omitempty"` + Ordinal int `json:"ordinal,omitempty"` + Value any `json:"value,omitempty"` +} + +// EventSummary is the compact representation returned by the list endpoint. +type EventSummary struct { + ID string `json:"id"` + Type EventType `json:"type"` + Start time.Time `json:"start"` + DurationMs int64 `json:"durationMs"` + + // Per-type summary fields (only the relevant ones are populated). + Method string `json:"method,omitempty"` + Path string `json:"path,omitempty"` + StatusCode int `json:"statusCode,omitempty"` + Query string `json:"query,omitempty"` + Level string `json:"level,omitempty"` + Message string `json:"message,omitempty"` + Error string `json:"error,omitempty"` +} + +// EventDetail is the full representation returned by the detail endpoint and used +// as the unit of redaction. Body bytes are never inlined; only metadata is shown. +type EventDetail struct { + ID string `json:"id"` + Type EventType `json:"type"` + Start time.Time `json:"start"` + End time.Time `json:"end"` + DurationMs int64 `json:"durationMs"` + + // HTTP (server and client) + Method string `json:"method,omitempty"` + Path string `json:"path,omitempty"` + URL string `json:"url,omitempty"` + StatusCode int `json:"statusCode,omitempty"` + RemoteAddr string `json:"remoteAddr,omitempty"` + RequestHeaders http.Header `json:"requestHeaders,omitempty"` + ResponseHeaders http.Header `json:"responseHeaders,omitempty"` + RequestBody *BodyMeta `json:"requestBody,omitempty"` + ResponseBody *BodyMeta `json:"responseBody,omitempty"` + Tags map[string]string `json:"tags,omitempty"` + + // DB query + Query string `json:"query,omitempty"` + Args []DBArg `json:"args,omitempty"` + Language string `json:"language,omitempty"` + + // Log record + Level string `json:"level,omitempty"` + Message string `json:"message,omitempty"` + Attrs map[string]any `json:"attrs,omitempty"` + + Error string `json:"error,omitempty"` + Children []*EventDetail `json:"children,omitempty"` +} + +// defaultRedactedHeaders are the header names whose values are masked by default, +// regardless of any configured redactor. Stored lowercased for case-insensitive +// matching. Set mirrors the otelhttptrace default set. +var defaultRedactedHeaders = []string{ + "authorization", + "cookie", + "set-cookie", + "www-authenticate", + "proxy-authenticate", + "proxy-authorization", +} + +// redactedHeaderValue is the sentinel used in place of a masked header value. +const redactedHeaderValue = "[REDACTED]" + +// mapEventDetail maps a collector.Event (and its children, recursively) to an +// EventDetail. It does not perform redaction; see Handler.redactDetail. +func mapEventDetail(e *collector.Event) *EventDetail { + if e == nil { + return nil + } + + d := &EventDetail{ + ID: e.ID.String(), + Start: e.Start, + End: e.End, + DurationMs: durationMs(e.Start, e.End), + Type: EventTypeUnknown, + } + + switch data := e.Data.(type) { + case collector.HTTPServerRequest: + d.Type = EventTypeHTTPServer + d.Method = data.Method + d.Path = data.Path + d.URL = data.URL + d.StatusCode = data.StatusCode + d.RemoteAddr = data.RemoteAddr + d.RequestHeaders = cloneHeader(data.RequestHeaders) + d.ResponseHeaders = cloneHeader(data.ResponseHeaders) + d.RequestBody = bodyMeta(data.RequestBody, data.RequestHeaders.Get("Content-Type")) + d.ResponseBody = bodyMeta(data.ResponseBody, data.ResponseHeaders.Get("Content-Type")) + d.Tags = data.Tags + if data.Error != nil { + d.Error = data.Error.Error() + } + case collector.HTTPClientRequest: + d.Type = EventTypeHTTPClient + d.Method = data.Method + d.URL = data.URL + d.StatusCode = data.StatusCode + d.RequestHeaders = cloneHeader(data.RequestHeaders) + d.ResponseHeaders = cloneHeader(data.ResponseHeaders) + d.RequestBody = bodyMeta(data.RequestBody, data.RequestHeaders.Get("Content-Type")) + d.ResponseBody = bodyMeta(data.ResponseBody, data.ResponseHeaders.Get("Content-Type")) + d.Tags = data.Tags + if data.Error != nil { + d.Error = data.Error.Error() + } + case collector.DBQuery: + d.Type = EventTypeDBQuery + d.Query = data.Query + d.Language = data.Language + d.Args = mapDBArgs(data.Args) + if data.Error != nil { + d.Error = data.Error.Error() + } + case slog.Record: + d.Type = EventTypeLog + d.Level = data.Level.String() + d.Message = data.Message + d.Attrs = mapLogAttrs(data) + } + + for _, child := range e.Children { + if cd := mapEventDetail(child); cd != nil { + d.Children = append(d.Children, cd) + } + } + + return d +} + +// summaryFromDetail derives the compact list representation from a (redacted) +// detail, so the list cannot bypass redaction. +func summaryFromDetail(d *EventDetail) EventSummary { + return EventSummary{ + ID: d.ID, + Type: d.Type, + Start: d.Start, + DurationMs: d.DurationMs, + Method: d.Method, + Path: summaryPath(d), + StatusCode: d.StatusCode, + Query: d.Query, + Level: d.Level, + Message: d.Message, + Error: d.Error, + } +} + +// summaryPath returns the path-ish field for a summary: Path for server events, +// URL for client events, empty otherwise. +func summaryPath(d *EventDetail) string { + if d.Path != "" { + return d.Path + } + return d.URL +} + +func mapDBArgs(args []driver.NamedValue) []DBArg { + if len(args) == 0 { + return nil + } + out := make([]DBArg, 0, len(args)) + for _, a := range args { + out = append(out, DBArg{Name: a.Name, Ordinal: a.Ordinal, Value: a.Value}) + } + return out +} + +func mapLogAttrs(record slog.Record) map[string]any { + if record.NumAttrs() == 0 { + return nil + } + attrs := make(map[string]any, record.NumAttrs()) + record.Attrs(func(a slog.Attr) bool { + attrs[a.Key] = a.Value.Any() + return true + }) + return attrs +} + +// extractBody pulls the raw bytes and content type for the requested side of an +// event. isBodyType is false for event types that have no bodies (DB, log); +// hasBody is false when the event type supports bodies but none was captured. +func extractBody(data any, side bodySide) (body []byte, contentType string, isBodyType bool, hasBody bool) { + switch d := data.(type) { + case collector.HTTPServerRequest: + if side == requestBodySide { + return bodyBytes(d.RequestBody), d.RequestHeaders.Get("Content-Type"), true, d.RequestBody != nil + } + return bodyBytes(d.ResponseBody), d.ResponseHeaders.Get("Content-Type"), true, d.ResponseBody != nil + case collector.HTTPClientRequest: + if side == requestBodySide { + return bodyBytes(d.RequestBody), d.RequestHeaders.Get("Content-Type"), true, d.RequestBody != nil + } + return bodyBytes(d.ResponseBody), d.ResponseHeaders.Get("Content-Type"), true, d.ResponseBody != nil + default: + return nil, "", false, false + } +} + +func bodyBytes(b *collector.Body) []byte { + if b == nil { + return nil + } + return b.Bytes() +} + +func bodyMeta(b *collector.Body, contentType string) *BodyMeta { + if b == nil { + return nil + } + return &BodyMeta{ + Size: b.Size(), + ContentType: contentType, + Available: true, + Redacted: false, + } +} + +func cloneHeader(h http.Header) http.Header { + if h == nil { + return nil + } + return h.Clone() +} + +func durationMs(start, end time.Time) int64 { + if end.IsZero() || end.Before(start) { + return 0 + } + return end.Sub(start).Milliseconds() +} diff --git a/dashboard/handler.go b/dashboard/handler.go index 4069d3d..5666412 100644 --- a/dashboard/handler.go +++ b/dashboard/handler.go @@ -6,6 +6,7 @@ import ( "fmt" "net/http" "slices" + "strings" "time" "github.com/a-h/templ" @@ -22,6 +23,10 @@ const DefaultStorageCapacity uint64 = 1000 // DefaultSessionIdleTimeout is the default time before an inactive session is cleaned up const DefaultSessionIdleTimeout = 30 * time.Second +// DefaultAgentMaxBodyBytes is the default cap on body bytes served by the agent +// body endpoints. +const DefaultAgentMaxBodyBytes uint64 = 64 * 1024 + type Handler struct { sessions *SessionManager eventAggregator *collector.EventAggregator @@ -29,6 +34,12 @@ type Handler struct { pathPrefix string truncateAfter uint64 + // Agent API configuration (see WithAgentAPI and friends). + agentRedactor AgentRedactor + agentRedactedHeaders map[string]bool // lowercased header names to mask + agentInsecureHeaders bool + agentMaxBodyBytes uint64 + mux http.Handler } @@ -59,6 +70,11 @@ func NewHandler(eventAggregator *collector.EventAggregator, opts ...HandlerOptio sessionIdleTimeout = DefaultSessionIdleTimeout } + agentMaxBodyBytes := options.AgentMaxBodyBytes + if agentMaxBodyBytes == 0 { + agentMaxBodyBytes = DefaultAgentMaxBodyBytes + } + sessions := NewSessionManager(SessionManagerOptions{ EventAggregator: eventAggregator, StorageCapacity: storageCapacity, @@ -67,11 +83,15 @@ func NewHandler(eventAggregator *collector.EventAggregator, opts ...HandlerOptio }) handler := &Handler{ - sessions: sessions, - eventAggregator: eventAggregator, - truncateAfter: truncateAfter, - pathPrefix: options.PathPrefix, - mux: mux, + sessions: sessions, + eventAggregator: eventAggregator, + truncateAfter: truncateAfter, + pathPrefix: options.PathPrefix, + agentRedactor: options.AgentRedactor, + agentRedactedHeaders: buildRedactedHeaderSet(options.AgentExtraRedactedHeaders), + agentInsecureHeaders: options.AgentInsecureHeaders, + agentMaxBodyBytes: agentMaxBodyBytes, + mux: mux, } // Static assets (no session required) @@ -99,9 +119,34 @@ func NewHandler(eventAggregator *collector.EventAggregator, opts ...HandlerOptio mux.HandleFunc("GET /s/{sid}/capture/status", handler.captureStatus) mux.HandleFunc("POST /s/{sid}/capture/cleanup", handler.captureCleanup) + // Agent JSON API (opt-in). Routes only exist when enabled, so they 404 + // naturally otherwise. + if options.AgentAPI { + mux.HandleFunc("GET /api/agent/v1/s/{sid}/events", handler.agentListEvents) + mux.HandleFunc("GET /api/agent/v1/s/{sid}/events/{eventId}", handler.agentEventDetail) + mux.HandleFunc("GET /api/agent/v1/s/{sid}/events/{eventId}/request-body", handler.agentRequestBody) + mux.HandleFunc("GET /api/agent/v1/s/{sid}/events/{eventId}/response-body", handler.agentResponseBody) + mux.HandleFunc("GET /api/agent/v1/s/{sid}/capture/status", handler.agentCaptureStatus) + mux.HandleFunc("GET /api/agent/v1/sessions", handler.agentListSessions) + mux.HandleFunc("GET /api/agent/v1/stats", handler.agentStats) + } + return handler } +// buildRedactedHeaderSet returns the lowercased set of header names to mask, +// combining the built-in defaults with any embedder-provided extras. +func buildRedactedHeaderSet(extra []string) map[string]bool { + set := make(map[string]bool, len(defaultRedactedHeaders)+len(extra)) + for _, name := range defaultRedactedHeaders { + set[strings.ToLower(name)] = true + } + for _, name := range extra { + set[strings.ToLower(name)] = true + } + return set +} + // withHandlerOptions is a helper to set HandlerOptions in context before rendering func (h *Handler) withHandlerOptions(r *http.Request, sessionID string, captureActive bool, captureMode string) *http.Request { ctx := views.WithHandlerOptions(r.Context(), views.HandlerOptions{ diff --git a/dashboard/options.go b/dashboard/options.go index fcfc02e..939dd6e 100644 --- a/dashboard/options.go +++ b/dashboard/options.go @@ -15,6 +15,22 @@ type handlerOptions struct { SessionIdleTimeout time.Duration // MaxSessions is the maximum number of concurrent sessions (0 = unlimited). MaxSessions int + + // AgentAPI enables the read-only agent JSON API under /api/agent/v1/. + // Off by default. + AgentAPI bool + // AgentRedactor is an optional hook applied to each EventDetail before + // encoding for the agent API. + AgentRedactor AgentRedactor + // AgentExtraRedactedHeaders are additional header names whose values are + // masked in agent API responses, on top of the built-in defaults. + AgentExtraRedactedHeaders []string + // AgentInsecureHeaders disables all built-in header masking in the agent + // API. Intended only for trusted local development. + AgentInsecureHeaders bool + // AgentMaxBodyBytes caps the number of body bytes served by the agent body + // endpoints (0 = use DefaultAgentMaxBodyBytes). + AgentMaxBodyBytes uint64 } // HandlerOption configures a dashboard Handler. @@ -60,3 +76,53 @@ func WithMaxSessions(limit int) HandlerOption { o.MaxSessions = limit } } + +// WithAgentAPI enables the read-only agent JSON API under /api/agent/v1/. +// +// The agent API is off by default: unlike the dashboard UI (which shows data to +// a human on screen), the agent API is consumed by tools that may forward data +// to third-party LLM providers, so it must be enabled deliberately. It inherits +// whatever auth middleware the embedder has placed in front of the dashboard. +func WithAgentAPI() HandlerOption { + return func(o *handlerOptions) { + o.AgentAPI = true + } +} + +// WithAgentRedactor sets a hook applied to each EventDetail before it is encoded +// for the agent API. The hook may mutate the detail and return it, or return nil +// to suppress the event entirely (omitted from lists, 404 on the detail endpoint). +// It runs after the built-in header masking. +func WithAgentRedactor(redactor AgentRedactor) HandlerOption { + return func(o *handlerOptions) { + o.AgentRedactor = redactor + } +} + +// WithAgentRedactedHeaders adds header names whose values are masked in agent API +// responses, on top of the built-in defaults (Authorization, Cookie, Set-Cookie, +// WWW-Authenticate, Proxy-Authenticate, Proxy-Authorization). Matching is +// case-insensitive. +func WithAgentRedactedHeaders(names ...string) HandlerOption { + return func(o *handlerOptions) { + o.AgentExtraRedactedHeaders = append(o.AgentExtraRedactedHeaders, names...) + } +} + +// WithAgentInsecureHeaders disables all built-in header masking in the agent API. +// This exposes Authorization, Cookie, and similar headers verbatim and should not +// be used outside trusted local development. +func WithAgentInsecureHeaders() HandlerOption { + return func(o *handlerOptions) { + o.AgentInsecureHeaders = true + } +} + +// WithAgentMaxBodyBytes caps the number of body bytes served by the agent body +// endpoints. Bodies larger than the cap are truncated to the cap and the response +// is marked truncated. Default is DefaultAgentMaxBodyBytes (64 KiB). +func WithAgentMaxBodyBytes(n uint64) HandlerOption { + return func(o *handlerOptions) { + o.AgentMaxBodyBytes = n + } +} diff --git a/dashboard/session_manager.go b/dashboard/session_manager.go index ec7cf87..2031664 100644 --- a/dashboard/session_manager.go +++ b/dashboard/session_manager.go @@ -140,6 +140,41 @@ func (sm *SessionManager) Delete(sessionID uuid.UUID) { delete(sm.sessions, sessionID) } +// SessionInfo describes an active capture session. +type SessionInfo struct { + SessionID uuid.UUID + Mode collector.CaptureMode + Capturing bool + EventCount int + LastActive time.Time +} + +// List returns info about all active sessions. +func (sm *SessionManager) List() []SessionInfo { + sm.sessionsMu.RLock() + defer sm.sessionsMu.RUnlock() + + out := make([]SessionInfo, 0, len(sm.sessions)) + for sessionID, state := range sm.sessions { + storage := sm.eventAggregator.GetStorage(state.storageID) + if storage == nil { + continue + } + cs, ok := storage.(*collector.CaptureStorage) + if !ok { + continue + } + out = append(out, SessionInfo{ + SessionID: sessionID, + Mode: cs.CaptureMode(), + Capturing: cs.IsCapturing(), + EventCount: len(cs.GetEvents(^uint64(0))), + LastActive: state.lastActive, + }) + } + return out +} + // UpdateActivity updates the last active time for a session func (sm *SessionManager) UpdateActivity(sessionID uuid.UUID) { sm.sessionsMu.Lock() diff --git a/docs/design/agent-access-relay.md b/docs/design/agent-access-relay.md new file mode 100644 index 0000000..d97bc8a --- /dev/null +++ b/docs/design/agent-access-relay.md @@ -0,0 +1,160 @@ +# Technical Design: Agent Access to devlog — Browser Relay (Phase 2) + +Based on [discussion #10](https://github.com/networkteam/devlog/discussions/10), including the author's follow-up comments (CLI-first pairing, multiplexing). Builds on the Agent JSON API defined in **[Phase 1: agent-access.md](./agent-access.md)** — read that first. + +## Goals & Requirements + +Let an AI agent inspect devlog data on a **deployed (stage/production)** instance, so a developer debugging a production bug can give the agent the same request/SQL/log context they'd read in the dashboard — complementing the Sentry MCP. The agent reaches the deployed instance **through the developer's own authenticated browser tab**; devlog never gains its own agent auth and never validates IdP tokens. + +Because this exposes captured production traffic to an LLM, Phase 2 is security-sensitive. The design centers on five guarantees: + +1. **The developer, not the agent, controls capture.** The agent has **no** capture-control tools at all — start/stop and the session vs global mode are managed entirely by the developer in the dashboard UI. The agent only attaches to (piggy-backs on) the developer's existing session. +2. **Bodies are never sent without explicit per-request human approval** in the dashboard. +3. **Opt-in.** The relay-facing surface is off unless the embedder enables it. +4. **The browser tab owns the connection's lifetime.** Closing the tab tears everything down immediately. +5. **The local MCP/relay endpoints are hardened** against other local processes and DNS-rebinding. + +**Out of scope:** push/streaming channels to the agent, scoped writes beyond capture control, broker transport. + +## High-Level Architecture + +``` + deployed instance developer machine +┌─────────────────────────┐ ┌──────────────────────────────────────┐ +│ Host app │ │ Browser tab (authenticated) │ +│ └ auth middleware │ HTTPS │ devlog dashboard │ +│ └ devlog dashboard │◄────────────│ + agent-relay.js │ +│ └ /api/agent/v1/ │ fetch() │ · forwards ONLY /api/agent/v1/* │ +│ (JSON+redact, │ w/ creds │ · shows body-approval dialogs │ +│ Phase 1) │ │ · stop beacon on pagehide │ +└─────────────────────────┘ └───────────────┬──────────────────────┘ + ▲ │ WebSocket (loopback) + body-approval │ ▼ + request frame │ ┌──────────────────────────────────────┐ + └──│ devlog relay (CLI, 127.0.0.1 only)│ + │ · HMAC pairing (binds sid+origin) │ + │ · connection registry by origin │ + │ · MCP server (streamable HTTP) │ + └───────────────┬──────────────────────┘ + │ MCP + ▼ + AI agent (Claude Code, …) +``` + +Every forwarded request passes through the embedder's auth middleware exactly like a dashboard click. The tab is the credential holder *and* the lifetime owner. + +## Architecture & Design Decisions + +| Decision | Choice | Rationale / Grounding | +|---|---|---| +| Relay surface availability | Opt-in: dashboard relay UI (button/dialog/badge + `agent-relay.js`) rendered only when the embedder sets `dashboard.WithAgentRelay()`. Independent of Phase 1's `WithAgentAPI()`, which it requires | Egress-sensitive; conservative default (Guarantee 3) | +| **No agent capture control** | The agent has no `capture_start`/`capture_stop` tools (same as Phase 1). The developer starts/stops capture and chooses session vs global **in the dashboard**, then connects the agent, which attaches to that session via its `sid`. The agent only reads (`capture_status` plus the events/bodies/stats reads). | Closes the sharpest risk — in global mode a session captures *all* users' traffic (`capture-session-system.md`) — by keeping capture entirely a human, dashboard-side decision. No mode-locking, mode-in-handshake, or escalation checks are needed (Guarantee 1). | +| **Body retrieval requires dashboard approval** | `get_request_body` / `get_response_body` do not fetch immediately. The CLI sends a body-approval request frame to the paired tab; the dashboard shows an "Agent requests the {request\|response} body for `{method} {path}` — Allow once / Allow for this session / Deny" dialog. Only on Allow does the CLI forward the body fetch. Deny or 30 s timeout → the MCP tool returns an error to the agent | Bodies are the highest-value, highest-risk payload; a human gate at the authenticated surface prevents silent bulk exfiltration (Guarantee 2). Independent of any agent-client-side tool approval, which can be allowlisted away | +| **Tab = lifetime owner** | Three teardown layers (see below) ensure no agent activity outlives the tab | Guarantee 4 | +| MCP endpoint hardening | `/mcp` rejects requests whose `Host` is not a loopback literal (DNS-rebinding defense); the WS `/relay` endpoint validates `Origin` against paired/`--allow-origin` origins via `websocket.AcceptOptions.OriginPatterns` | Guarantee 5. The MCP endpoint is otherwise unauthenticated on loopback (standard); the SDK's `StreamableHTTPHandler` also tracks session IDs to prevent hijacking | +| Pairing protocol | CLI-first HMAC mutual challenge–response; secret never transmitted; precise wire format below | RFC author's follow-up: avoids fixed-port squatting and secret exposure via argv/shell history | +| Multiplexing | CLI keeps a connection registry keyed by page origin; MCP tools take an optional `environment` parameter (origin hostname), defaulting to the single connection when only one exists | RFC author's follow-up; **kept** — we need to compare stage vs production in one agent session | +| WebSocket library (CLI side) | `github.com/coder/websocket` (ISC) | Research: actively maintained (v1.8.14 Sep 2025, commits through Mar 2026), native `context.Context`, `wsjson` helpers, zero deps, safe concurrent writes, `OriginPatterns` origin check built in ([pkg.go.dev](https://pkg.go.dev/github.com/coder/websocket), [comparison](https://websocket.org/guides/languages/go/)) | +| Browser relay JS | Vanilla JS `dashboard/static/agent-relay.js`, dialog + button + approval modal in `header.templ` | Convention: dashboard uses templ + static assets (`dashboard/static/assets.go`), no JS framework | +| Loopback / mixed-content model | No PNA/CORS preflight handling; rely on loopback being "potentially trustworthy" | Research: Chrome's PNA preflights were replaced by the Local Network Access *permission prompt* in Chrome 142 (Oct 2025); WebSockets are not yet gated by LNA (crbug.com/421156866), so the relay works prompt-free today; when WS is gated, the IP-literal target yields a one-time prompt, no code change ([Chrome blog](https://developer.chrome.com/blog/local-network-access)) | +| Browser support | Chrome/Edge/Firefox supported; Safari unsupported — the dialog detects Safari and shows a hint | Research: `ws://127.0.0.1` from HTTPS is not mixed content in Chrome/Firefox (Firefox since 2020, [bug 1376309](https://bugzilla.mozilla.org/show_bug.cgi?id=1376309)); WebKit still blocks loopback as mixed content ([bug 171934](https://bugs.webkit.org/show_bug.cgi?id=171934), open mid-2026). No `wss://` self-signed workaround is designed | + +## Pairing & Handshake (precise wire format) + +Transport is `ws://127.0.0.1:` — unencrypted loopback. The protocol therefore never transmits the secret and binds every security-relevant field into the MAC. + +**Connect string** (printed by the CLI): `dlr1::` where `S` is 32 bytes from a CSPRNG (`crypto/rand`). `S` lives for the CLI process lifetime and is reused across origin pairings. The developer pastes this into the dashboard dialog. (Capture mode is not chosen here — it is whatever the developer already set when starting capture in the dashboard.) + +**Handshake** — all frames JSON; `||` denotes length-prefixed concatenation; MAC is HMAC-SHA256 over the byte transcript: + +1. **CLI → Page** `{"t":"challenge","ns":""}` — `Ns` = 32 random bytes (`crypto/rand`). +2. **Page → CLI** `{"t":"auth","nc":"","origin":"","sid":"","mac":""}` — `Nc` = 32 random bytes (`crypto.getRandomValues`); `Mc = HMAC(S, "devlog-relay-v1:client" || Ns || Nc || origin || sid)`. +3. **CLI** verifies `Mc` in constant time (`hmac.Equal`) and that `origin` matches the allowlist. On success **CLI → Page** `{"t":"auth-ok","mac":""}` — `Ms = HMAC(S, "devlog-relay-v1:server" || Ns || Nc)`. +4. **Page** verifies `Ms`. Only now is the channel trusted by both ends. + +Properties: + +- **Secret never on the wire** — only HMACs of fresh nonces; HMAC preimage resistance protects `S` from a loopback eavesdropper (who is already a privileged local attacker). +- **Transcript binding** — `origin` and `sid` are inside `Mc`, so a tampered origin or sid fails verification (the agent reads exactly the session the page authenticated for). +- **Mutual auth & no reflection** — distinct domain-separation labels (`:client` / `:server`) prevent reflecting the server's challenge back as a client response. +- **Replay-resistant** — fresh `Ns`+`Nc` per handshake; a replayed `auth` fails against a new `Ns`. +- **Bounded** — 5 s handshake timeout; any field mismatch, bad MAC, or disallowed origin closes the socket. Constant-time comparison on both ends. + +The CLI records `{origin → {ws conn, dashboard sid}}` in the registry. Re-pairing the same origin replaces the entry. + +## Tab Lifetime & Teardown + +**Invariant: the agent has no path to devlog except through a live browser tab.** Every agent request is a `fetch()` issued by that tab; there is no other route. Two layers enforce prompt teardown when the tab closes: + +1. **WS close → environment dropped.** When the page's WebSocket closes (tab close, reload, navigation), the CLI removes that origin from the registry immediately. MCP tools targeting it return `environment disconnected`. This is the relay-specific teardown — the agent loses all access the instant the tab goes away. +2. **Dashboard session lifecycle → capture ends.** The session belongs to the dashboard tab, not the relay. Its existing lifecycle applies unchanged: the SSE keep-alive stops when the tab closes, and the `SessionManager` idle timeout then reclaims the session (stopping capture). The relay never starts or stops capture, so it adds no capture-control beacon of its own. + +The "Agent connected" badge is driven by the live WS, so it disappears exactly when access ends. + +## Implementation Changes + +### CLI (`./cli` module) — browser-relay mode + +`devlog relay [--mcp-port ] [--relay-port ] [--allow-origin ...]` (no `--direct`) + +- Binds `127.0.0.1` only; one HTTP server with `/relay` (WebSocket for browser tabs) and `/mcp` (MCP `StreamableHTTPHandler`). Random ephemeral ports by default; prints connect string + MCP URL on startup. +- Runs the pairing handshake on each new `/relay` socket; maintains the origin registry. +- Forwarding frames (JSON over WS): `{id, method, path, body?}` → `{id, status, contentType, body}` (`body` base64 for binary). **Path must start with `/api/agent/v1/`** and must not contain `..` after normalization — enforced on **both** sides (page allowlist is the real security boundary, since the page holds the credentials; CLI never emits other paths). +- Body-approval frames: `{id, t:"approve-body", which, method, path}` → `{id, t:"approve-body-result", decision}` before any body fetch is forwarded. +- MCP tools as in Phase 1 (all read-only; no `capture_start`/`capture_stop`), with the `environment` parameter now meaningful. `list_environments` reports each origin and its attached `sid`. + +### Dashboard (server side) + +- `dashboard/static/agent-relay.js` — **NEW**: parses connect string, opens `ws://127.0.0.1:/relay`, runs the handshake (sending the tab's `sid`), forwards allowlisted frames via `fetch(pathPrefix + path, {credentials:'same-origin'})`, renders body-approval modals, auto-reconnects with backoff while connected, exposes a disconnect button. (It does not touch capture — that stays in the dashboard's own controls.) +- `dashboard/views/header.templ` — **NEW** UI: "Connect agent" button, pairing dialog (connect-string input + Safari hint), body-approval modal, persistent "Agent connected" badge. +- `dashboard/options.go` — `WithAgentRelay()` (gates rendering of the relay UI; requires `WithAgentAPI()`). +- `dashboard/handler.go` — serve `agent-relay.js`; render relay UI only under `WithAgentRelay()`. + +### Files to Modify + +| File | Changes | +|---|---| +| `cli/` | Relay (browser) mode: pairing, registry, forwarding, body-approval round-trip, Origin/Host hardening | +| `dashboard/static/agent-relay.js` | **NEW** — pairing + forwarding + approval + teardown client | +| `dashboard/views/header.templ` | Connect-agent button, pairing dialog, approval modal, connected badge | +| `dashboard/options.go` | `WithAgentRelay` | +| `dashboard/handler.go` | Serve relay JS; conditional relay UI | + +## Test Cases + +### 1. CLI pairing & forwarding (`cli/relay_test.go` — NEW) + +| Test | Fixture | Action | Expectation | +|---|---|---|---| +| `TestPairing_ValidSecret` | Relay + test WS client with secret | Full handshake | Paired; registry records origin + attached `sid` | +| `TestPairing_WrongSecret` | Relay | Handshake with wrong `Mc` | Socket closed, not registered | +| `TestPairing_TamperedSid` | Relay | Valid-looking `auth` but `sid` altered after MAC | MAC verification fails, socket closed | +| `TestPairing_DisallowedOrigin` | Relay with `--allow-origin` | Handshake from other origin | Rejected | +| `TestForwarding_RoundTrip` | Paired fake browser echoing frames | Forward GET events | Frame correlation by `id`, body decoded | +| `TestForwarding_RejectsNonAllowlistedPath` | Paired client | Internal request to `/s/{sid}/event-list` and `/api/agent/v1/../x` | Refused before emitting a frame | +| `TestNoCaptureControlTools` | Paired client | MCP `tools/list` | `capture_start`/`capture_stop` absent | +| `TestRegistry_MultipleOrigins` | Two paired clients (origins A, B) | Forward with `environment=B` | Routed to B; missing param with 2 origins → error listing environments | +| `TestTeardown_WSCloseDropsEnvironment` | Paired client | Close WS | Environment removed immediately; subsequent tool call errors `disconnected` | +| `TestMCP_HostValidation` | Relay MCP server | Request with non-loopback `Host` | Rejected | +| `TestMCP_WSOriginValidation` | Relay `/relay` | WS upgrade with bad `Origin` | Rejected | + +### 2. Body approval (`cli/approval_test.go` — NEW) + +| Test | Fixture | Action | Expectation | +|---|---|---|---| +| `TestBodyApproval_Allowed` | Paired client auto-approving | `get_request_body` | Approval frame sent; body fetched and returned | +| `TestBodyApproval_Denied` | Paired client denying | `get_request_body` | No body fetch forwarded; tool returns error | +| `TestBodyApproval_Timeout` | Paired client that never answers | `get_request_body` | Tool errors after 30 s; no fetch | +| `TestBodyApproval_AllowForSession` | Paired client approving "for session" | Two `get_request_body` calls | One approval prompt; both bodies served | + +### 3. Acceptance / E2E (`acceptance/agent_relay_test.go` — NEW; reuse `testapp.go` + playwright harness) + +| Test | Fixture | Action | Expectation | +|---|---|---|---| +| `TestAgentRelay_EndToEnd` | Test app + dashboard in browser + `devlog relay` in-process | Connect agent (mode=session), generate traffic, MCP `list_events`, then `get_request_body` with dashboard approval | Events delivered through tunnel; badge visible; body served only after approval | +| `TestAgentRelay_TabCloseTearsDown` | As above, paired and capturing | Close the dashboard tab | Capture stops promptly (stop beacon), badge gone, agent tool calls error | + +## Notes on Residual Risk (accepted) + +- A privileged local attacker who can read loopback traffic sees nonces/MACs but cannot derive the ephemeral secret or forge a handshake; `Host`/`Origin` checks block the realistic browser/DNS-rebinding vectors. This is the standard local-MCP threat model. +- `global` mode still captures all users' traffic — but only when the developer chooses it in the dashboard; the agent cannot. Default header redaction, metadata-only event listings + the body size cap, and per-body human approval bound what reaches the LLM. Embedders handling regulated data should add a `WithAgentRedactor` and consider leaving the relay off for production. diff --git a/docs/design/agent-access.md b/docs/design/agent-access.md new file mode 100644 index 0000000..42e7bf9 --- /dev/null +++ b/docs/design/agent-access.md @@ -0,0 +1,139 @@ +# Technical Design: Agent Access to devlog — JSON API & Local MCP (Phase 1) + +Based on the concept in [discussion #10 — RFC: Agent Access to devlog via Browser-Relayed Tunnel](https://github.com/networkteam/devlog/discussions/10). + +This is **Phase 1 of two specs**: + +- **Phase 1 (this doc)** — the Agent JSON API on the devlog handler, plus a `devlog` CLI that exposes it to agents over MCP for **local development** (`--direct` mode). No browser, no pairing, no production data egress. +- **Phase 2 ([agent-access-relay.md](./agent-access-relay.md))** — the browser-relayed tunnel that lets an agent reach **deployed (stage/production)** devlog instances through the developer's authenticated browser tab. Builds on the API defined here. + +The split exists because the two phases have very different risk profiles: Phase 1 touches only the developer's own machine; Phase 2 exposes captured production traffic to an LLM and is security-sensitive. Phase 1 is independently useful and shippable. + +## Goals & Requirements + +Enable AI coding agents (Claude Code, Cursor, …) to inspect devlog data — captured requests, SQL queries, logs, timings — on a **local dev instance**, without coupling to the embedder's auth. + +Constraints from the RFC that apply already in Phase 1: + +- devlog is an embedded handler. The agent API must inherit the embedder's middleware, never validate tokens itself. +- Agent responses end up at LLM providers — sensitive data must be redactable before it leaves the process. +- Read-only by default; the only mutation is capture control (start/stop), scoped to the agent's own session. + +**Out of scope** (Phase 2 or the RFC's optional phase 4): browser relay, deployed-instance access, push/streaming channels, scoped writes beyond capture control, broker transport. + +## High-Level Architecture (Phase 1) + +``` + developer machine (local dev) +┌──────────────────────────────────────────────────────────────┐ +│ Host app (dev) │ +│ └ devlog dashboard handler │ +│ └ /api/agent/v1/ (JSON + redaction) ◄──── HTTP ───┐ │ +│ │ │ +│ devlog relay --direct http://localhost:PORT/_devlog │ │ +│ └ MCP server (streamable HTTP, loopback) ───────────────┘ │ +│ ▲ │ +│ │ MCP │ +│ AI agent (Claude Code, …) │ +└──────────────────────────────────────────────────────────────┘ +``` + +In `--direct` mode the CLI calls the JSON API itself over plain HTTP. There is no browser hop and no credential relaying — it is the developer's own local instance. + +## Architecture & Design Decisions + +| Decision | Choice | Rationale / Grounding | +|---|---|---| +| Agent API placement | New routes on the existing dashboard `http.ServeMux` under `/api/agent/v1/` | Inherits the embedder's middleware for free (RFC requirement). Convention: route registration in `dashboard/handler.go` (`NewHandler`) | +| API session scoping | Session ID in path: `/api/agent/v1/s/{sid}/…` | Convention: existing UI routes `/s/{sid}/…` in `dashboard/handler.go:87-100`; storage is per-session (`SessionManager`). | +| **Attach, don't create** | The CLI does not mint a session. It lists active sessions (`GET /api/agent/v1/sessions`) and **attaches** to an existing dashboard session, so the agent and the developer's dashboard tab see the same events. The browser tab owns the session's lifetime (its SSE keep-alive); no agent-side heartbeat or cleanup. | Matches the Phase 2 relay, where the paired tab's `sid` is shared by construction. Avoids the 30s idle-timeout reclaiming an agent-owned session and the ownership/cleanup questions that come with one. | +| Session selection (`--direct`) | One active session → auto-attach; several → interactive prompt; none → wait until one appears. `--session ` overrides non-interactively. | Keeps the common case zero-config while staying scriptable. | +| API package layout | New files in the `dashboard` package (`agent_api.go`, `agent_api_types.go`) | Convention: `dashboard` is a flat package; handlers need unexported access to `SessionManager`. A sub-package would force exporting session internals | +| DTOs instead of rendering `collector.Event` directly | Explicit `EventSummary` / `EventDetail` JSON structs mapped via type switch on `Event.Data` | `Event.Data` is `any` (`collector/event.go:20`) holding `HTTPServerRequest`, `HTTPClientRequest`, `DBQuery`, `slog.Record`; same type-switch pattern as `downloadRequestBody` (`dashboard/handler.go:453`). DTOs give a stable wire format and a single place for redaction | +| **Agent API availability** | **Opt-in: `dashboard.WithAgentAPI()`. Off by default.** | The dashboard UI shows data to a human on screen; the agent API ships it to a third-party LLM — a different risk profile, so default-on is not safe even behind the same auth. Opt-in is the conservative default; embedders enable it deliberately | +| **Bodies never inlined** | List and detail responses carry only body **metadata** (`{size, contentType, available, redacted}`). Raw bytes are served only by the dedicated body endpoints / MCP tools | Smallest default egress: the agent receives bodies only when it explicitly asks. In Phase 2 those body tools are additionally gated behind dashboard confirmation | +| Body size cap | Body endpoints truncate to `WithAgentMaxBodyBytes` (default 64 KiB) with an explicit `truncated: true` marker | Avoids shipping multi-MB or binary blobs to the LLM; assumption — tune default during implementation | +| Redaction hook | `dashboard.WithAgentRedactor(func(*EventDetail) *EventDetail)` — mutate and return, or return `nil` to suppress the event entirely. Applied before encoding; summaries derive from redacted details | Research: mirrors Sentry's `BeforeSend` (scrub-or-drop in one hook, [Sentry data scrubbing](https://develop.sentry.dev/sdk/foundations/data-scrubbing/)). Operating on DTOs (not `collector` types) means redaction can't corrupt stored events | +| Default header redaction | Independent of the hook, always mask values of `Authorization`, `Cookie`, `Set-Cookie`, `WWW-Authenticate`, `Proxy-Authenticate`, `Proxy-Authorization` with `[REDACTED]` (key preserved). `WithAgentRedactedHeaders(names...)` extends; `WithAgentInsecureHeaders()` is the explicit opt-out | Research: default set and additive-extend/loud-opt-out pattern from [otelhttptrace](https://pkg.go.dev/go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace); value-masked-key-preserved per [OTel HTTP semconv](https://opentelemetry.io/docs/specs/semconv/http/http-spans/) | +| Redaction observability | Masked values use sentinels; suppressed bodies get `redacted: true` — never silent omission | Research: an LLM consumer must distinguish "absent" from "redacted" to avoid false inferences | +| CLI location | New Go module `./cli` in `go.work`, binary `devlog`, subcommand `relay` (with `--direct` here; browser mode added in Phase 2) | Convention: separate modules for non-library code (`./acceptance`, `./example`, `./dbadapter/sqllogger` in `go.work`). Keeps the library's `go.mod` free of CLI deps | +| CLI framework | `spf13/cobra` | Already in the dependency graph as indirect dep; stdlib `flag` would also do — keep cobra only if more commands are expected, otherwise simplify during implementation | +| MCP server | Official SDK `github.com/modelcontextprotocol/go-sdk` (v1.x), `StreamableHTTPHandler` on loopback | Research: verified the SDK ships `StreamableHTTPHandler` — a persistent `http.Handler` for networked deployments with per-session tracking ([streamable.go](https://github.com/modelcontextprotocol/go-sdk/blob/main/mcp/streamable.go), [examples/http](https://pkg.go.dev/github.com/modelcontextprotocol/go-sdk/examples/http)). HTTP (not stdio) is required because the relay is a long-running server reachable by both the agent and, in Phase 2, the browser | +| No agent capture control | The agent **cannot start or stop capture**, in any mode. Capture (and its mode) is managed by the user in the dashboard UI; the agent only piggy-backs on a user-managed session. Only `capture_status` (read-only) is exposed, plus the events/bodies/stats reads. | Keeps the human in control of what is captured (matters most for Phase 2's production traffic), removes the need for mode-locking/pairing-mode machinery, and matches the attach model — the session is the dashboard's, the agent observes it. | +| Dashboard "agent connected" badge | **Deferred** — a follow-up that shows in the dashboard when an agent is attached to a session. Needs server-side attach tracking + a templ change. | Nice-to-have for visibility; not required for the attach flow to work. | + +## Implementation Changes + +### Agent JSON API (`dashboard` package) + +New endpoints registered in `NewHandler`, only when `WithAgentAPI()` is set (all JSON, no HTMX branch): + +| Method | Path | Purpose | +|---|---|---| +| GET | `/api/agent/v1/s/{sid}/events` | Event summaries, newest first. Query params: `type` (`http_server`, `http_client`, `db_query`, `log`), `since`/`until` (RFC 3339), `limit`, `status` (HTTP status class, e.g. `5xx`), `path` (substring match) | +| GET | `/api/agent/v1/s/{sid}/events/{eventId}` | Full detail incl. children tree: SQL with `InterpolatedQuery`, log records, timings, headers, **body metadata only** | +| GET | `/api/agent/v1/s/{sid}/events/{eventId}/request-body` | Raw body (≤ cap), original content type (logic of `downloadRequestBody`) | +| GET | `/api/agent/v1/s/{sid}/events/{eventId}/response-body` | Raw body, ditto | +| GET | `/api/agent/v1/s/{sid}/capture/status` | `{active, mode, eventCount}` (read-only; no agent start/stop) | +| GET | `/api/agent/v1/sessions` | List active capture sessions (`sid`, mode, capturing, eventCount, lastActiveMsAgo) so the relay can attach | +| GET | `/api/agent/v1/stats` | Same data as existing `getStats` JSON branch | + +Notes: + +- `{sid}` not found → 404 with JSON error body `{"error": "..."}`; consistent error shape on all endpoints. +- Event list endpoints call `sessions.UpdateActivity(sid)` so agent polling keeps the session alive (as the SSE handler does). +- DTO mapping (`agent_api_types.go`): `EventSummary{id, type, start, durationMs, summary fields per type}`; `EventDetail` embeds the full per-type payload (`method/path/status/headers` for HTTP, `query/interpolatedQuery/args/language/error` for DB, `level/message/attrs` for logs via `slog.Record.Attrs` iteration) plus recursive `children`, and `requestBody`/`responseBody` metadata objects `{size, contentType, available, redacted}`. +- Redaction order per event: built-in header masking first, then the embedder's redactor (`nil` return → event omitted from lists, 404 on detail). A redacted body returns 410 Gone with a JSON error so agents see "redacted", not "missing". + +### CLI (`./cli` module, binary `devlog`) — `--direct` mode + +`devlog relay --direct [--mcp-port ]` + +- Binds `127.0.0.1` only; serves MCP at `/mcp` via the SDK's `StreamableHTTPHandler`. Random ephemeral MCP port by default; prints the MCP URL on startup. +- Issues plain HTTP requests to `/api/agent/v1/…`. On startup, selects an existing session via `GET /api/agent/v1/sessions` (auto / prompt / wait, or `--session`) and attaches to it; registers a synthetic environment `local`. It never creates a session. +- MCP tools (all read-only): `list_events`, `get_event`, `get_request_body`, `get_response_body`, `capture_status`, `get_stats`, `list_environments`. No `capture_start`/`capture_stop` — capture is user-managed in the dashboard. (The `environment` parameter and multiplexing matter in Phase 2; in `--direct` there is one environment `local`.) +- MCP endpoint is unauthenticated on loopback (standard for local MCP servers) **but validates the `Host` header is a loopback literal** to block DNS-rebinding from a browser; data exposure is further bounded by the redaction hook server-side. + +### Files to Modify + +| File | Changes | +|---|---| +| `dashboard/agent_api.go` | **NEW** — agent endpoint handlers | +| `dashboard/agent_api_types.go` | **NEW** — DTOs, type-switch mapping, redactor type | +| `dashboard/handler.go` | Register `/api/agent/v1/` routes when `WithAgentAPI()` is set | +| `dashboard/options.go` | `WithAgentAPI`, `WithAgentRedactor`, `WithAgentRedactedHeaders`, `WithAgentInsecureHeaders`, `WithAgentMaxBodyBytes` | +| `cli/` | **NEW module** — `main.go`, `relay` command (`--direct`), MCP server + tool definitions, direct HTTP client | +| `go.work` | Add `./cli` | + +## Test Cases + +### 1. Agent API handler tests (`dashboard/agent_api_test.go` — NEW; follow `TestE2E` setup in `devlog_e2e_test.go`: build an `Instance` with `WithAgentAPI()`, collect fixture events through the aggregator, serve via `httptest`) + +| Test | Fixture | Action | Expectation | +|---|---|---|---| +| `TestAgentAPI_Disabled_ByDefault` | Handler without `WithAgentAPI` | GET any agent route | 404 | +| `TestAgentAPI_Events_EmptySession` | Handler, no session | GET `/api/agent/v1/s/{sid}/events` | 404, JSON error body | +| `TestAgentAPI_Events_ListsSummaries` | Session with HTTP+DB+log events | GET events | 200, newest first, correct `type` per event | +| `TestAgentAPI_Events_FilterByType` | Mixed events | GET `?type=db_query` | Only DB query summaries | +| `TestAgentAPI_Events_FilterByTimeAndLimit` | Events with known timestamps | GET `?since=…&limit=1` | Window + limit respected | +| `TestAgentAPI_EventDetail_HTTPServerWithChildren` | Request event with DB child | GET detail | Children tree, `interpolatedQuery`, durations, body metadata only (no inline bytes) | +| `TestAgentAPI_EventDetail_LogRecord` | slog event with attrs | GET detail | level/message/attrs mapped | +| `TestAgentAPI_RequestBody_Raw` | Event with JSON request body | GET request-body | 200, original content type, exact bytes | +| `TestAgentAPI_RequestBody_Truncated` | Event with body > cap | GET request-body | Truncated to cap, `truncated: true` signaled | +| `TestAgentAPI_DefaultHeaderRedaction` | Event with `Authorization` + `Cookie`, no redactor | GET detail | Values `[REDACTED]`, keys preserved | +| `TestAgentAPI_RedactedHeaders_Extended` | `WithAgentRedactedHeaders("X-Api-Key")` | GET detail | Custom header masked in addition to defaults | +| `TestAgentAPI_InsecureHeaders_OptOut` | `WithAgentInsecureHeaders()` | GET detail | `Authorization` value visible | +| `TestAgentAPI_Redactor_DropsEvent` | Redactor returning `nil` for `/auth/*` | GET list + detail + body | Omitted from list, 404 on detail and body | +| `TestAgentAPI_Redactor_RedactsBody` | Redactor clearing request-body availability | GET detail + request-body | Detail shows `redacted: true`; body endpoint returns 410 | +| `TestAgentAPI_CaptureLifecycle` | Fresh sid (client-generated UUID) | POST start (global) → traffic → GET status → POST stop | Session created, events counted, capture stops | +| `TestAgentAPI_KeepsSessionAlive` | Session near idle timeout | GET events repeatedly past timeout | Session not cleaned up | + +### 2. CLI direct-mode + MCP tests (`cli/direct_test.go`, `cli/mcp_test.go` — NEW; `httptest` devlog instance + in-process MCP client/server transport from the SDK) + +| Test | Fixture | Action | Expectation | +|---|---|---|---| +| `TestDirectMode_ListEvents` | `httptest` devlog instance; a session is created (simulating the dashboard) | `--direct` client attaches to the session → list | Events returned without any browser | +| `TestMCP_ListTools` | Relay in `--direct` mode | MCP `tools/list` | All tools present with schemas | +| `TestMCP_ListAndGetEvent` | devlog instance with fixture events | `list_events` → `get_event` | Structured content matches API payloads | +| `TestMCP_CaptureStatus_ReadOnly` | Session created via dashboard + traffic | `capture_status` + `tools/list` | Status reported; `capture_start`/`capture_stop` not exposed | +| `TestMCP_HostValidation` | Relay MCP server | Request with non-loopback `Host` header | Rejected (DNS-rebinding defense) | diff --git a/docs/tasks/agent-access/README.md b/docs/tasks/agent-access/README.md new file mode 100644 index 0000000..cb90891 --- /dev/null +++ b/docs/tasks/agent-access/README.md @@ -0,0 +1,32 @@ +# Task Breakdown: Agent Access to devlog — Phase 1 (JSON API & Local MCP) + +Implements **[../../design/agent-access.md](../../design/agent-access.md)** — the Agent JSON API on the devlog dashboard handler plus a `devlog` CLI exposing it to agents over MCP for local development (`--direct` mode). Phase 2 (browser relay) is a separate spec ([agent-access-relay.md](../../design/agent-access-relay.md)) and is **not** covered here. + +## Approach + +Five vertical slices. The Agent JSON API is the foundation (verified directly via `httptest`, the way `devlog_e2e_test.go` works); the `devlog` CLI is a consumer layer over that API (verified via an in-process MCP client/server transport). Each task is independently mergeable and verified by the spec's own test cases — none is infrastructure-only. + +Phase 1 touches **no `.templ` files**, so there is no `templ generate` step — it is pure Go plus the new `./cli` module. + +A cross-cutting security note baked into the slicing: **default header redaction ships in Task 1**, not later — a read endpoint exposing headers without masking `Authorization`/`Cookie` by default would be insecure on first merge. + +## Tasks (dependency order) + +| # | Title | Depends on | +|---|---|---| +| 1 | [Agent API read endpoints (opt-in, list + detail, DTOs, redaction)](./task-1-agent-api-read.md) | — | +| 2 | [Body endpoints, size cap, body redaction](./task-2-body-endpoints.md) | 1 | +| 3 | [Capture control + stats endpoints](./task-3-capture-and-stats.md) | 1 | +| 4 | [CLI module + `--direct` client + MCP read tools](./task-4-cli-direct-mcp.md) | 1, 3 | +| 5 | [CLI MCP body + capture tools](./task-5-cli-body-capture-tools.md) | 2, 3, 4 | + +Order: **1 → (2, 3) → 4 → 5**. Tasks 2 and 3 can proceed in parallel once 1 lands. + +## Conventions (apply to every task) + +- **No build step.** Verify compilation with `go vet ./...`; run tests with `go test ./... -failfast` (and `go test ./dashboard/...` / `go test ./cli/...` for the focused packages). +- **Functional options.** All new dashboard configuration is a `dashboard.HandlerOption` in `dashboard/options.go`, following the existing `WithStorageCapacity`/`WithPathPrefix` pattern. Embedders pass them through `dlog.DashboardHandler(prefix, opts...)` (`devlog.go:128`). +- **Reads go through per-session storage.** `Handler.sessions.Get(sid)` returns a `*collector.CaptureStorage` exposing `GetEvents(limit)`, `GetEvent(id)`, `CaptureMode()`, `IsCapturing()`, `SetCapturing(bool)`. The agent endpoints mirror the existing UI handlers in `dashboard/handler.go`. +- **Event data is a type switch.** `Event.Data` (`collector/event.go:20`) is one of `collector.HTTPServerRequest`, `collector.HTTPClientRequest`, `collector.DBQuery`, `slog.Record`. Follow the switch in `downloadRequestBody` (`dashboard/handler.go:453`). +- **Test fixtures.** Build an instance and drive events through the collectors as in `devlog_e2e_test.go`; for handler-level tests, construct sessions/storages as in `dashboard/session_manager_test.go`. Prefer table-driven tests. +- **Authorization** stays with the embedder's middleware; devlog adds none per-endpoint. The opt-in gate (`WithAgentAPI`) is the only access decision devlog makes. diff --git a/docs/tasks/agent-access/task-1-agent-api-read.md b/docs/tasks/agent-access/task-1-agent-api-read.md new file mode 100644 index 0000000..a3f1dbb --- /dev/null +++ b/docs/tasks/agent-access/task-1-agent-api-read.md @@ -0,0 +1,77 @@ +# Task 1 — Agent API read endpoints (opt-in, list + detail, DTOs, redaction) + +## Context + +You are implementing Phase 1 of the spec at **`docs/design/agent-access.md`** (read the "Agent JSON API" section and the "Architecture & Design Decisions" table). This is the **foundational slice**: the read side of the Agent JSON API on the devlog dashboard handler, gated behind an opt-in option, with redaction built in from the start. + +This task has **no dependencies**. It establishes files and patterns that Tasks 2–5 extend. + +Repository: `github.com/networkteam/devlog`. No build step — verify with `go vet ./...` and `go test ./dashboard/...`. + +## Goal + +Behind a new opt-in `dashboard.WithAgentAPI()` option (the agent API is **off by default**), expose: + +- `GET /api/agent/v1/s/{sid}/events` — event summaries, newest first, with query filters: `type` (`http_server`|`http_client`|`db_query`|`log`), `since`/`until` (RFC 3339), `limit`, `status` (HTTP status class such as `5xx`), `path` (substring match). +- `GET /api/agent/v1/s/{sid}/events/{eventId}` — full detail including the recursive children tree, with **body metadata only** (no raw bytes — those come in Task 2). + +All responses are JSON (no HTMX branch). Errors use a consistent `{"error": "..."}` body. Unknown `{sid}` → 404. + +## What to implement + +### Options (`dashboard/options.go`) +Follow the existing `HandlerOption` pattern (`WithStorageCapacity`, etc.). Add: +- `WithAgentAPI()` — enables registration of the `/api/agent/v1/` routes. Store a bool on `handlerOptions`. +- `WithAgentRedactor(func(*EventDetail) *EventDetail)` — embedder hook applied to each detail DTO before encoding; returning `nil` suppresses the event entirely (omitted from lists, 404 on detail). +- `WithAgentRedactedHeaders(names ...string)` — extends the default redacted-header set (additive). +- `WithAgentInsecureHeaders()` — disables all built-in header redaction (explicit opt-out). + +### Route registration (`dashboard/handler.go`) +In `NewHandler`, register the agent routes on the existing `mux` **only when `WithAgentAPI()` was set**. Mirror the existing route style (`dashboard/handler.go:86-100`). When not enabled, the routes must not exist (so requests 404 naturally). + +### Handlers (`dashboard/agent_api.go` — NEW) +- Resolve the session with `h.getSessionID(r)` + `h.sessions.Get(sid)` (see `dashboard/handler.go:127`, `:174`). Nil storage → 404 JSON error. +- Call `h.sessions.UpdateActivity(sid)` on reads so agent polling keeps the session alive (the SSE handler does this — `dashboard/handler.go:365`). +- List: read `storage.GetEvents(limit)` (default a sane limit if absent; cap at `h.truncateAfter`), reverse to newest-first as `loadRecentEvents` does (`dashboard/handler.go:421`), apply filters, map to `EventSummary`, encode. +- Detail: `storage.GetEvent(eventID)` → map to `EventDetail` (recursing children) → run redaction → encode; not found → 404. +- A small JSON error helper for the consistent `{"error": ...}` shape. + +### DTOs + mapping (`dashboard/agent_api_types.go` — NEW) +- `EventSummary{ id, type, start, durationMs, ... per-type summary fields }`. +- `EventDetail{ ... full per-type payload, children []EventDetail, requestBody/responseBody metadata }`. Body metadata is an object `{size, contentType, available, redacted}` (the `available`/`redacted`/byte-serving semantics are finalized in Task 2 — here, populate `size`/`contentType`/`available` from the stored `*Body` presence). +- Map via a type switch over `Event.Data` (follow `downloadRequestBody`, `dashboard/handler.go:453`): + - `collector.HTTPServerRequest` / `collector.HTTPClientRequest`: method, path/URL, status, headers, timings, body metadata. + - `collector.DBQuery`: `query`, `interpolatedQuery` (nullable `*string`), `args`, `language`, `error`, duration. + - `slog.Record`: `level`, `message`, and attrs via `record.Attrs(func(slog.Attr) bool)` iteration. +- `type AgentRedactor func(*EventDetail) *EventDetail`. + +### Redaction +- **Default header masking** (independent of the hook): replace the *values* of `Authorization`, `Cookie`, `Set-Cookie`, `WWW-Authenticate`, `Proxy-Authenticate`, `Proxy-Authorization` (case-insensitive) with the sentinel string `[REDACTED]`, **preserving the keys**, in every HTTP `EventDetail`. `WithAgentRedactedHeaders` adds names; `WithAgentInsecureHeaders` skips masking entirely. +- Apply order per event: header masking first, then the embedder's `WithAgentRedactor`. A `nil` return drops the event from list responses and yields 404 on the detail endpoint. +- Summaries shown in the list must be derived from the **redacted** detail (map detail → redact → derive summary), so redaction can't be bypassed via the list. + +## Acceptance criteria & verification + +Implement these table-driven tests in `dashboard/agent_api_test.go` (NEW). Follow the `httptest` setup of `devlog_e2e_test.go` for building an instance and driving fixture events through the collectors; use `dashboard/session_manager_test.go` for session/storage construction patterns. Enable the API with `WithAgentAPI()` except where noted. + +| Test | Expectation | +|---|---| +| `TestAgentAPI_Disabled_ByDefault` | Handler built **without** `WithAgentAPI` → any `/api/agent/v1/...` route returns 404 | +| `TestAgentAPI_Events_EmptySession` | Unknown `{sid}` → 404 with JSON error body | +| `TestAgentAPI_Events_ListsSummaries` | Session with HTTP + DB + log events → 200, newest-first, correct `type` per summary | +| `TestAgentAPI_Events_FilterByType` | `?type=db_query` → only DB query summaries | +| `TestAgentAPI_Events_FilterByTimeAndLimit` | `?since=…&limit=1` → time window and limit both respected | +| `TestAgentAPI_EventDetail_HTTPServerWithChildren` | HTTP server event with a DB child → children tree present, `interpolatedQuery` present, durations set, body metadata only (no inline bytes) | +| `TestAgentAPI_EventDetail_LogRecord` | slog event → `level`/`message`/`attrs` mapped | +| `TestAgentAPI_DefaultHeaderRedaction` | Event with `Authorization` + `Cookie`, no redactor configured → values are `[REDACTED]`, keys preserved | +| `TestAgentAPI_RedactedHeaders_Extended` | `WithAgentRedactedHeaders("X-Api-Key")` → custom header masked in addition to defaults | +| `TestAgentAPI_InsecureHeaders_OptOut` | `WithAgentInsecureHeaders()` → `Authorization` value visible | +| `TestAgentAPI_Redactor_DropsEvent` | Redactor returning `nil` for `/auth/*` paths → event omitted from list and 404 on its detail | +| `TestAgentAPI_KeepsSessionAlive` | Repeated list reads past the idle timeout keep the session alive | + +Also: `go vet ./...` clean; `go test ./dashboard/...` green. + +## Out of scope (later tasks) +- Raw body bytes, size cap, body `available/redacted` 410 behavior → Task 2. +- Capture control and stats endpoints → Task 3. +- CLI / MCP → Tasks 4–5. diff --git a/docs/tasks/agent-access/task-2-body-endpoints.md b/docs/tasks/agent-access/task-2-body-endpoints.md new file mode 100644 index 0000000..5d0f142 --- /dev/null +++ b/docs/tasks/agent-access/task-2-body-endpoints.md @@ -0,0 +1,45 @@ +# Task 2 — Body endpoints, size cap, body redaction + +## Context + +Phase 1 of **`docs/design/agent-access.md`**. This task adds the raw-body endpoints to the Agent JSON API and finalizes body redaction semantics. + +**Depends on Task 1** (`docs/tasks/agent-access/task-1-agent-api-read.md`) — the agent API skeleton, `WithAgentAPI()` gating, the `EventDetail` DTO with body metadata fields, the JSON error helper, and the `WithAgentRedactor` hook already exist. Read that task file to know what to expect. + +Repository: `github.com/networkteam/devlog`. Verify with `go vet ./...` and `go test ./dashboard/...`. + +## Goal + +Add: +- `GET /api/agent/v1/s/{sid}/events/{eventId}/request-body` +- `GET /api/agent/v1/s/{sid}/events/{eventId}/response-body` + +serving raw body bytes with the original content type, bounded by a configurable size cap, and honoring redaction (a redacted body is not served). + +## What to implement + +### Size cap option (`dashboard/options.go`) +- `WithAgentMaxBodyBytes(n uint64)` — default **64 KiB** when unset. When a body exceeds the cap, serve the first `n` bytes and signal truncation (a response header such as `X-Devlog-Truncated: true`, and set `truncated: true` in the body metadata of the detail DTO). + +### Body endpoints (`dashboard/agent_api.go`) +- Reuse the type-switch logic of the existing `downloadRequestBody`/`downloadResponseBody` handlers (`dashboard/handler.go:429`, `:484`) to extract `*collector.Body` and content type from `HTTPServerRequest`/`HTTPClientRequest`. Other event types → 400, JSON error. +- Apply the size cap with truncation. +- **Redaction:** build the event's `EventDetail` and run the same redaction pipeline as Task 1 (header masking → `WithAgentRedactor`). If the redactor cleared the body's `available` flag (or returned `nil` for the whole event), the body endpoint returns **410 Gone** with `{"error":"body redacted"}`. Missing body (never captured) → 404. + +### Finalize body metadata (`dashboard/agent_api_types.go`) +- The `requestBody`/`responseBody` metadata object is `{size, contentType, available, redacted}`. Define the redaction contract: a redactor that sets `available=false` (or `redacted=true`) on a body marks it as suppressed; the detail DTO reflects `redacted: true`, and the corresponding body endpoint returns 410. Keep `redacted` distinct from `available=false`-because-absent so an agent can tell "redacted" from "no body". + +## Acceptance criteria & verification + +Extend `dashboard/agent_api_test.go`: + +| Test | Expectation | +|---|---| +| `TestAgentAPI_RequestBody_Raw` | Event with a JSON request body → 200, original content type, exact bytes | +| `TestAgentAPI_RequestBody_Truncated` | Body larger than the cap → bytes truncated to the cap, truncation signaled (header + metadata `truncated: true`) | +| `TestAgentAPI_Redactor_RedactsBody` | Redactor clearing request-body availability → detail shows `redacted: true`; the request-body endpoint returns 410 | + +Also: a body never captured → 404; non-body event type → 400. `go vet ./...` clean; `go test ./dashboard/...` green. + +## Out of scope +- Capture control / stats → Task 3. CLI/MCP → Tasks 4–5. diff --git a/docs/tasks/agent-access/task-3-capture-and-stats.md b/docs/tasks/agent-access/task-3-capture-and-stats.md new file mode 100644 index 0000000..ea70c22 --- /dev/null +++ b/docs/tasks/agent-access/task-3-capture-and-stats.md @@ -0,0 +1,40 @@ +# Task 3 — Capture control + stats endpoints + +## Context + +Phase 1 of **`docs/design/agent-access.md`**. Adds capture lifecycle control and a stats endpoint to the Agent JSON API. + +**Depends on Task 1** (`docs/tasks/agent-access/task-1-agent-api-read.md`) — agent API skeleton, `WithAgentAPI()` gating, JSON error helper. Can be done in parallel with Task 2. + +Repository: `github.com/networkteam/devlog`. Verify with `go vet ./...` and `go test ./dashboard/...`. + +## Goal + +Add: +- `POST /api/agent/v1/s/{sid}/capture/start` — JSON body `{"mode":"session"|"global"}`. Create/resume the session's capture. +- `POST /api/agent/v1/s/{sid}/capture/stop` — pause capture, keep the session and events. +- `GET /api/agent/v1/s/{sid}/capture/status` — `{active, mode, eventCount}`. +- `GET /api/agent/v1/stats` — the same data the existing `getStats` JSON branch returns. + +## What to implement (`dashboard/agent_api.go`) + +- **start:** parse `mode` from the JSON body (default `session`) using `collector.ParseCaptureModeOrDefault`. Create/resume via `h.sessions.GetOrCreate(sid, mode)` and `storage.SetCapturing(true)` / `storage.SetCaptureMode(mode)` — mirror the existing `captureStart` logic (`dashboard/handler.go:547`) **but without any cookie handling** (agents have no cookies). On capacity errors from `GetOrCreate`, return 503 JSON error (as the UI handler does). +- **stop:** `storage.SetCapturing(false)`, keep storage intact (mirror `captureStop`, `dashboard/handler.go:587`). No session → return inactive status, not an error. +- **status:** `{active: storage.IsCapturing(), mode: storage.CaptureMode().String(), eventCount: len(storage.GetEvents())}`. No session → `{active:false, mode:"session", eventCount:0}`. +- **stats:** reuse `h.eventAggregator.CalculateStats()` and the `StatsResponse` shape from `getStats` (`dashboard/handler.go:721`); JSON only. + +Note: in Phase 1 `mode` is agent-selectable (local dev). Phase 2 locks it at pairing — do not add that restriction here. + +## Acceptance criteria & verification + +Extend `dashboard/agent_api_test.go`: + +| Test | Expectation | +|---|---| +| `TestAgentAPI_CaptureLifecycle` | Fresh client-generated `{sid}` (UUIDv4): POST start (`global`) → session created & capturing; drive traffic → events captured; GET status → `active:true, mode:"global", eventCount>0`; POST stop → `active:false`, events retained | +| `TestAgentAPI_Stats` | GET `/api/agent/v1/stats` → JSON with memory/session/event counts matching `CalculateStats()` | + +Also: start with an invalid mode → 400 JSON error; `go vet ./...` clean; `go test ./dashboard/...` green. + +## Out of scope +- CLI / MCP → Tasks 4–5. diff --git a/docs/tasks/agent-access/task-4-cli-direct-mcp.md b/docs/tasks/agent-access/task-4-cli-direct-mcp.md new file mode 100644 index 0000000..03447eb --- /dev/null +++ b/docs/tasks/agent-access/task-4-cli-direct-mcp.md @@ -0,0 +1,57 @@ +# Task 4 — CLI module + `--direct` client + MCP read tools + +## Context + +Phase 1 of **`docs/design/agent-access.md`** (read the "CLI (`./cli` module) — `--direct` mode" section and the MCP/CLI decision rows). This task creates the `devlog` CLI that exposes the Agent JSON API to AI agents over MCP, for **local development** — no browser, no pairing. + +**Depends on Tasks 1 and 3** — the read endpoints (`/events`, `/events/{id}`) and capture/stats endpoints must exist, since the CLI drives a session and reads events through them. + +Repository: `github.com/networkteam/devlog`. Verify with `go vet ./...` and `go test ./cli/...`. + +## Goal + +A new `./cli` Go module producing the `devlog` binary with a `relay` subcommand in `--direct` mode: + +``` +devlog relay --direct [--mcp-port ] +``` + +It runs an MCP server on loopback that an agent (e.g. Claude Code) connects to, translating MCP tool calls into HTTP calls against `/api/agent/v1/…` on the developer's local devlog instance. + +## What to implement + +### Module setup +- New module at `./cli` with its own `go.mod` (module path `github.com/networkteam/devlog/cli`). Add `./cli` to `go.work`. +- Add the MCP Go SDK dependency: `github.com/modelcontextprotocol/go-sdk`. **Before wiring, verify the exact server API** — read the SDK's `mcp` package docs / `examples/http` to confirm the constructor and registration calls (the spec references `StreamableHTTPHandler`, an `http.Handler` serving streamable MCP sessions). Use the real, current signatures; do not guess. +- CLI framework: `spf13/cobra` (already in the dependency graph). A single `relay` command is fine. + +### `relay --direct` command +- Bind the MCP server to `127.0.0.1` only. Default the MCP port to a random ephemeral port; print the MCP URL on startup so the user can configure their agent. +- **Host-header hardening:** wrap the MCP `http.Handler` so requests whose `Host` is not a loopback literal (`127.0.0.1[:port]`, `localhost[:port]`, `[::1][:port]`) are rejected (e.g. 403). This blocks DNS-rebinding from a browser. (In Phase 2 the same wrapper protects the relay's MCP endpoint.) +- A "direct client": an HTTP client targeting `/api/agent/v1/…`. On first use, generate a session UUIDv4 and start capture (`POST .../capture/start`); register a single synthetic environment named `local`. + +### MCP read tools +Register these tools (each takes an optional `environment` parameter that defaults to the sole `local` environment in direct mode): +- `list_events` — params mirror the API filters (`type`, `since`, `until`, `limit`, `status`, `path`); returns summaries. +- `get_event` — `eventId`; returns the detail DTO. +- `get_stats` — returns the stats payload. +- `list_environments` — returns `[{name:"local", mode:...}]`. + +Map tool results to MCP structured content. Body and capture-control tools are added in Task 5. + +## Acceptance criteria & verification + +Tests in `cli/direct_test.go` and `cli/mcp_test.go`. Spin up a real devlog instance with `httptest` (build it with `WithAgentAPI()`, drive fixture events through the collectors as in `devlog_e2e_test.go`), point the direct client at the test server's URL, and exercise MCP through the SDK's in-process client/server transport. + +| Test | Expectation | +|---|---| +| `TestDirectMode_ListEvents` | Direct client against an httptest devlog instance: capture start → list → fixture events returned, no browser involved | +| `TestMCP_ListTools` | MCP `tools/list` → `list_events`, `get_event`, `get_stats`, `list_environments` present with schemas | +| `TestMCP_ListAndGetEvent` | `list_events` then `get_event` on a returned id → structured content matches the API payloads | +| `TestMCP_HostValidation` | A request to the MCP endpoint with a non-loopback `Host` header is rejected | + +Also: `go vet ./...` clean; `go test ./cli/...` green; `go work` builds with the new module. + +## Out of scope +- `get_request_body` / `get_response_body` / `capture_start` / `capture_stop` / `capture_status` MCP tools → Task 5. +- Browser relay, pairing, multiplexing across multiple environments → Phase 2. diff --git a/docs/tasks/agent-access/task-5-cli-body-capture-tools.md b/docs/tasks/agent-access/task-5-cli-body-capture-tools.md new file mode 100644 index 0000000..779daaf --- /dev/null +++ b/docs/tasks/agent-access/task-5-cli-body-capture-tools.md @@ -0,0 +1,40 @@ +# Task 5 — CLI MCP body + capture tools + +## Context + +Phase 1 of **`docs/design/agent-access.md`**. Completes the `--direct` MCP surface by adding the body-retrieval and capture-control tools. + +**Depends on Tasks 2, 3, and 4** — the body endpoints (Task 2), capture/status endpoints (Task 3), and the CLI/MCP server + direct client + read tools (Task 4) must all exist. Read `docs/tasks/agent-access/task-4-cli-direct-mcp.md` for the established CLI patterns to extend. + +Repository: `github.com/networkteam/devlog`. Verify with `go vet ./...` and `go test ./cli/...`. + +## Goal + +Add these MCP tools to the `relay --direct` server, each wrapping an already-tested API endpoint and accepting the optional `environment` parameter: + +- `get_request_body` — `eventId`; fetches `…/events/{eventId}/request-body`. Returns the body (respecting the server's size cap / truncation marker); surfaces a redacted body (410) as a clear tool error/message, not a crash. +- `get_response_body` — `eventId`; fetches `…/events/{eventId}/response-body`, same handling. +- `capture_start` — `mode` (`session`|`global`); calls `…/capture/start`. +- `capture_stop` — calls `…/capture/stop`. +- `capture_status` — calls `…/capture/status`; returns `{active, mode, eventCount}`. + +Note: in direct mode `capture_start` accepts `mode` (local dev). The Phase 2 relay locks the mode at pairing and removes the agent-supplied `mode` — do **not** implement that restriction here. + +## What to implement + +Extend the CLI's tool registration and direct client (from Task 4) with the five tools above. Map raw body bytes to MCP content sensibly (text content for textual content types; for binary, return metadata + a note rather than dumping raw bytes into the model — the size cap from Task 2 still applies server-side). Translate non-2xx API responses (404 missing, 410 redacted, 400 wrong type) into informative tool errors. + +## Acceptance criteria & verification + +Extend `cli/mcp_test.go` (same httptest + in-process MCP transport setup as Task 4): + +| Test | Expectation | +|---|---| +| `TestMCP_CaptureControl` | `capture_start` → drive traffic → `list_events` shows the events; `capture_status` reflects `active`/`eventCount`; `capture_stop` stops capture | +| `TestMCP_GetRequestBody` | `get_request_body` for an event with a body → bytes returned via the tool | +| `TestMCP_GetBody_Redacted` | Server configured with a redactor suppressing a body → the body tool surfaces a redacted/410 result as a clear tool error, not a panic | + +Also: `go vet ./...` clean; `go test ./cli/...` and `go test ./... -failfast` green. + +## Out of scope +- Browser relay, pairing, body-approval dialogs, multiplexing → Phase 2 (`docs/design/agent-access-relay.md`). diff --git a/go.work b/go.work index d38b1cd..0c9d55b 100644 --- a/go.work +++ b/go.work @@ -1,8 +1,9 @@ -go 1.23.8 +go 1.25.0 use ( . ./acceptance + ./cli ./dbadapter/sqllogger ./example ) diff --git a/go.work.sum b/go.work.sum index 16a7b35..74a4c31 100644 --- a/go.work.sum +++ b/go.work.sum @@ -1,3 +1,5 @@ +cloud.google.com/go/compute/metadata v0.3.0 h1:Tz+eQXMEqDIKRsmY3cHTL6FVaynIjX2QxYC4trgAKZc= +cloud.google.com/go/compute/metadata v0.3.0/go.mod h1:zFmK7XCadkQkj6TtorcaGlCW1hT1fIilQDwofLpJ20k= github.com/a-h/htmlformat v0.0.0-20250209131833-673be874c677 h1:H1EoxMpNo/TnCZEgSaKsomi+dQ05dXiVSKDN86Lap9A= github.com/a-h/htmlformat v0.0.0-20250209131833-673be874c677/go.mod h1:FMIm5afKmEfarNbIXOaPHFY8X7fo+fRQB6I9MPG2nB0= github.com/apex/logs v1.0.0 h1:adOwhOTeXzZTnVuEK13wuJNBFutP0sOfutRS8NY+G6A= @@ -53,23 +55,32 @@ golang.org/x/crypto v0.37.0 h1:kJNSjF/Xp7kU0iB2Z+9viTPMW4EqqsrywMXLJOOsXSE= golang.org/x/crypto v0.37.0/go.mod h1:vg+k43peMZ0pUMhYmVAWysMK35e6ioLh3wB8ZCAfbVc= golang.org/x/crypto v0.38.0 h1:jt+WWG8IZlBnVbomuhg2Mdq0+BBQaHbtqHEFEigjUV8= golang.org/x/crypto v0.38.0/go.mod h1:MvrbAqul58NNYPKnOra203SB9vpuZW0e+RRZV+Ggqjw= +golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts= +golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos= golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA= golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= +golang.org/x/mod v0.32.0/go.mod h1:SgipZ/3h2Ci89DlEtEXWUk/HteuRin+HHhN+WbNhguU= golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ= golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sys v0.14.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.25.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= golang.org/x/telemetry v0.0.0-20240521205824-bda55230c457 h1:zf5N6UOrA487eEFacMePxjXAJctxKmyjKUsjA11Uzuk= golang.org/x/telemetry v0.0.0-20240521205824-bda55230c457/go.mod h1:pRgIJT+bRLFKnoM1ldnzKoxTIn14Yxz928LQRYYgIN0= +golang.org/x/telemetry v0.0.0-20260209163413-e7419c687ee4 h1:bTLqdHv7xrGlFbvf5/TXNxy/iUwwdkjhqQTJDjW7aj0= +golang.org/x/telemetry v0.0.0-20260209163413-e7419c687ee4/go.mod h1:g5NllXBEermZrmR51cJDQxmJUHUOfRAaNyWBM+R+548= golang.org/x/term v0.31.0 h1:erwDkOK1Msy6offm1mOgvspSkslFnIGsFnxOKoufg3o= golang.org/x/term v0.31.0/go.mod h1:R4BeIy7D95HzImkxGkTW1UQTtP54tio2RyHz7PwK0aw= golang.org/x/term v0.32.0 h1:DR4lr0TjUs3epypdhTOkMmuF5CDFJ/8pOnbzMZPQ7bg= golang.org/x/term v0.32.0/go.mod h1:uZG1FhGx848Sqfsq4/DlJr3xGGsYMu/L5GW4abiaEPQ= +golang.org/x/term v0.40.0 h1:36e4zGLqU4yhjlmxEaagx2KuYbJq3EwY8K943ZsHcvg= +golang.org/x/term v0.40.0/go.mod h1:w2P8uVp06p2iyKKuvXIm7N/y0UCRt3UfJTfZ7oOpglM= golang.org/x/text v0.22.0/go.mod h1:YRoo4H8PVmsu+E3Ou7cqLVH8oXWIHVoX0jqUWALQhfY= golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58= golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg= golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk= +golang.org/x/tools v0.41.0/go.mod h1:XSY6eDqxVNiYgezAVqqCeihT4j1U2CCsqvH3WhQpnlg= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7 h1:9zdDQZ7Thm29KFXgAX/+yaf3eVbP7djjWp/dXAppNCc= gopkg.in/fsnotify.v1 v1.4.7 h1:xOHLXZwVvI9hhs+cLKq5+I5onOuwQLhQwiu63xxlHs4= gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=