diff --git a/context/skills/mcp-analytics/config.yaml b/context/skills/mcp-analytics/config.yaml index 673c3fb4..697638c0 100644 --- a/context/skills/mcp-analytics/config.yaml +++ b/context/skills/mcp-analytics/config.yaml @@ -2,7 +2,7 @@ type: skill template: description.md category: mcp-analytics -description: Add PostHog MCP analytics to a TypeScript, JavaScript, or Python MCP server. Instruments the server so every tool call, agent intent, and failure is captured as a $mcp_* event — the @posthog/mcp Node SDK for TS/JS, or posthog.mcp (shipped inside the posthog package) for Python. Detects the server style (official MCP SDK Server/McpServer/FastMCP, mcp-handler, @rekog/mcp-nest, jlowin fastmcp, or a custom HTTP/edge dispatcher) and wires in the matching instrumentation, credentials, and graceful shutdown. +description: Add PostHog MCP analytics to a TypeScript, JavaScript, or Python MCP server. Captures tool calls, agent intent, failures, and protocol revision, plus self-reported model identity on supported TypeScript wrappers. Both SDKs support legacy sessions and the handshake-free 2026-07-28 revision. Detects the server style and wires in compatible instrumentation, credentials, conversation correlation, and graceful shutdown. tags: [mcp-analytics, mcp] cli: role: command @@ -28,11 +28,8 @@ variants: tags: [python, mcp] docs_urls: - https://posthog.com/docs/mcp-analytics/installation.md - # Bundled here too, even though it's a TypeScript page: description.md is - # shared by both variants and points at it, so omitting it would leave a - # dangling reference. Its own callout tells a Python reader what applies. - url: https://posthog.com/docs/mcp-analytics/sdk-v2.md - title: MCP TypeScript SDK v1 vs v2 — read when the project uses @modelcontextprotocol/server + title: MCP SDK v2 and the 2026-07-28 revision - https://posthog.com/docs/mcp-analytics/custom-servers.md - https://posthog.com/docs/mcp-analytics/intent.md - https://posthog.com/docs/mcp-analytics/identifying-users.md diff --git a/context/skills/mcp-analytics/description.md b/context/skills/mcp-analytics/description.md index 2c63f019..abc543df 100644 --- a/context/skills/mcp-analytics/description.md +++ b/context/skills/mcp-analytics/description.md @@ -1,6 +1,6 @@ # Add PostHog MCP analytics -Use this skill to instrument a user's own **MCP server** with PostHog MCP analytics. Once instrumented, every tool call, agent intent, and failure the server handles is captured as a `$mcp_*` event in PostHog — so the user can see which tools get used, what agents are trying to do, error rates, and latency. +Use this skill to instrument a user's own **MCP server** with PostHog MCP analytics. Once instrumented, every tool call, agent intent, and failure the server handles is captured as a `$mcp_*` event in PostHog. Supported TypeScript wrappers can also capture the agent's self-reported model, so the user can compare quality, errors, and latency by model. There are two SDKs and this skill handles both: - **TypeScript / JavaScript** — the [`@posthog/mcp`](https://posthog.com/docs/mcp-analytics) Node package. @@ -45,15 +45,17 @@ Determine the language first, then route to the matching instructions throughout **Record which SDK major the project is on.** STEP 2, STEP 3, and STEP 4 each contain a **For `@modelcontextprotocol/sdk` (v1)** and a **For `@modelcontextprotocol/server` (v2)** section — follow only the one matching the major found here. `sdk-v2.md` is the reference for everything v2-specific. - **Python** — there's a `pyproject.toml`, `requirements.txt`, or `setup.py`, or `.py` sources. Look for MCP signals: - - the official `mcp` package — `from mcp.server.fastmcp import FastMCP` or `from mcp.server.lowlevel import Server`. + - the official `mcp` package — `from mcp.server.fastmcp import FastMCP` on 1.x, `from mcp.server.mcpserver import MCPServer` on 2.x, or `from mcp.server.lowlevel import Server` on either major. - jlowin's standalone `fastmcp` 2.0 — `from fastmcp import FastMCP`. - a custom HTTP/edge dispatcher (FastAPI / Starlette / Flask / edge) speaking the MCP protocol directly with no server object to wrap. Determine the installer (pip / uv / poetry) from the lockfile / `pyproject.toml`. + Record which official `mcp` major the project uses. The wrapper is tested against `mcp>=1.26,<3`; don't confuse this with jlowin's separately versioned `fastmcp` package. + - If it's neither TS/JS nor Python, apply the guardrail above and stop. -Then identify the file and the exact place where the server is constructed or where MCP requests are dispatched, and read it before editing. If PostHog MCP analytics is already wired in (an `instrument(` call, or a `PostHogMCP` client — in either language), don't duplicate it: verify it's correct and skip to STEP 7. +Then identify the file and the exact place where the server is constructed or where MCP requests are dispatched, and read it before editing. If PostHog MCP analytics is already wired in (an `instrument(` call, or a `PostHogMCP` client in either language), don't duplicate it. Verify the existing setup against STEP 4, add supported modern options that are missing, then continue through STEP 7. ### STEP 2: Choose the instrumentation path @@ -78,14 +80,14 @@ The server object comes from `@modelcontextprotocol/server`. Follow the **v2** s #### Python -- **Path P1 — a FastMCP or low-level Server** (the official `mcp` package's `FastMCP`/`Server`, or jlowin's `fastmcp` 2.0): wrap it with `instrument(server, posthog)`. One line — the SDK detects which framework it is. +- **Path P1 — a high-level or low-level server** (the official `mcp` package's 1.x `FastMCP`, 2.x `MCPServer`, or `Server` from either major; or jlowin's standalone `fastmcp` package): wrap it with `instrument(server, posthog)`. One line — the SDK detects the framework and major. - **Path P2 — custom dispatcher** (FastAPI / Starlette / Flask / edge with no server object to wrap): use the `PostHogMCP` client and call `capture_tool_call` / `capture_initialize` yourself at the dispatch points. ### STEP 3: Install the SDK #### TypeScript / JavaScript -Install `@posthog/mcp` and `posthog-node` with the project's package manager, pinning `@posthog/mcp` to its current published version (it's pre-1.0) — e.g. `pnpm add @posthog/mcp@ posthog-node`. Read the installed version back from `package.json` / the lockfile rather than guessing. +Install `@posthog/mcp` and `posthog-node` with the project's package manager, pinning `@posthog/mcp` to its current published version (it's pre-1.0) — e.g. `pnpm add @posthog/mcp@ posthog-node`. Read the installed version back from `package.json` / the lockfile rather than guessing. Self-reported model capture requires `@posthog/mcp>=0.12.0`; upgrade an older installed version before enabling it. **Never install an MCP SDK.** Both majors are *optional* peer dependencies of `@posthog/mcp`, and the project already has the one it uses. Adding the other pulls in a whole SDK the code never imports. @@ -99,7 +101,7 @@ No extra constraint — pinning the current published `@posthog/mcp` release is #### Python -The SDK ships inside `posthog`, so install (or require) `posthog>=7.21` with the project's installer — e.g. `pip install "posthog>=7.21"`, `uv add posthog`, `poetry add posthog`. The MCP SDK itself (`mcp` / `fastmcp`) is a peer dependency you already have — you built the server with it — so don't add it. A custom-dispatcher (path P2) project needs nothing beyond `posthog`. +The SDK ships inside `posthog`, so install (or require) `posthog>=7.40.0` with the project's installer — e.g. `pip install "posthog>=7.40.0"`, `uv add "posthog>=7.40.0"`, `poetry add "posthog>=7.40.0"`. Version 7.40.0 added official MCP SDK 2.x and `2026-07-28` support. The MCP SDK is a peer dependency tested across `mcp>=1.26,<3`; don't add or change it as part of this command. jlowin's standalone `fastmcp` package is also supported. A custom-dispatcher (path P2) project needs nothing beyond `posthog`. ### STEP 4: Instrument the server @@ -124,7 +126,9 @@ import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js" import { instrument } from "@posthog/mcp" const server = new McpServer({ name: "my-mcp-server", version: "1.0.0" }) -const analytics = instrument(server, posthog) // wrap immediately after constructing the server +const analytics = instrument(server, posthog, { + captureModel: true, +}) // wrap immediately after constructing the server // register tools as usual — tools added after instrument() are still captured ``` @@ -135,7 +139,10 @@ import { McpServer } from "@modelcontextprotocol/server" import { instrument } from "@posthog/mcp" const server = new McpServer({ name: "my-mcp-server", version: "1.0.0" }) -const analytics = instrument(server, posthog) // wrap immediately after constructing the server +const analytics = instrument(server, posthog, { + captureModel: true, + enableConversationId: true, +}) // wrap immediately after constructing the server // register tools with registerTool() as usual — tools added after instrument() are still captured ``` @@ -152,6 +159,8 @@ import { instrument, getRequestHeaders } from "@posthog/mcp" const handler = createMcpHandler((server) => { instrument(server, posthog, { + captureModel: true, + enableConversationId: true, identify: async (request, extra) => { const token = getRequestHeaders(extra)?.["authorization"] return token ? { distinctId: await resolveUserId(token) } : null @@ -172,8 +181,8 @@ const posthog = new PostHogMCP(process.env.POSTHOG_PROJECT_TOKEN, { host: process.env.POSTHOG_HOST, }) -// on the initialize handshake: -posthog.captureInitialize({ clientName, clientVersion, distinctId }) +// only on a 2025-11-25 initialize handshake: +posthog.captureInitialize({ clientName, clientVersion, distinctId, protocolVersion: "2025-11-25" }) // after each tools/call resolves (wrap the existing handler, time it): const start = Date.now() @@ -186,10 +195,11 @@ posthog.captureToolCall({ isError: false, distinctId, // who the request is from, if known sessionId, // your transport/session id, if you have one + protocolVersion, // read this from the current request }) ``` -Resolve `distinctId` / `sessionId` from whatever auth/session the dispatcher already has; omit them rather than inventing values. These calls are fire-and-forget and never throw, so they can't take down a tool. +Resolve `distinctId` / `sessionId` from whatever auth/session the dispatcher already has; omit them rather than inventing values. Pass `protocolVersion` on every capture. On `2025-11-25`, use the revision the dispatcher's existing session state negotiated during initialize. On `2026-07-28`, read `MCP-Protocol-Version` from the current request because there is no initialize handshake or protocol session. If the dispatcher exposes neither source, omit the property rather than hardcoding a revision. Don't fabricate `$mcp_initialize` on `2026-07-28`. `captureModel` and conversation-id injection aren't available on the custom-dispatcher path, so don't add `$mcp_llm_model` manually. These calls are fire-and-forget and never throw, so they can't take down a tool. **Path D — `@rekog/mcp-nest` (NestJS):** the framework builds the server, so pass a `serverMutator` to `McpModule.forRoot(...)`. Prefer the `instrumentMutator` helper — it instruments the server and returns it, so it drops straight into the hook: @@ -207,7 +217,9 @@ const posthog = new PostHog(process.env.POSTHOG_PROJECT_TOKEN, { McpModule.forRoot({ name: "my-mcp-server", version: "1.0.0", - serverMutator: instrumentMutator(posthog), + serverMutator: instrumentMutator(posthog, { + enableConversationId: true, + }), }), ], }) @@ -216,9 +228,11 @@ class AppModule {} `instrumentMutator` returns the server (not `instrument()`'s handle), so it slots straight into the hook. Compose with an existing `serverMutator` if there is one, and handlers nest registers after the mutator runs are still captured. For [custom events](https://posthog.com/docs/mcp-analytics/custom-events), call `instrument()` directly inside your own mutator and keep its handle, returning the server yourself. +If mcp-nest keeps one persistent server instance, also set `captureModel: true`. Don't enable it when `statelessMode: true` creates a fresh low-level server per request: that path advertises `llm_model` but can't confirm ownership before the call, so the model property stays empty. + ##### For @modelcontextprotocol/server (v2), any path — sessions and protocol revisions -Protocol revision is a property of each *request*, not of the server — a v2 server serves `2025-11-25` traffic too — so instrument once and never branch on the major. The `2026-07-28` revision removed the `initialize` handshake and the `Mcp-Session-Id` header, so on that traffic **every request becomes its own `$session_id`** unless `enableConversationId: true` is set. If the project targets it, offer that option and say what it buys. `sdk-v2.md` has the rest, including why `$mcp_client_name` can be absent on `2025-11-25` traffic behind `createMcpHandler`, and which `2026-07-28` features aren't instrumented yet. +Protocol revision is a property of each *request*, not of the server. A v2 server serves `2025-11-25` traffic too, so instrument once and never branch on the major. The `2026-07-28` revision removed the `initialize` handshake and the `Mcp-Session-Id` header, so on that traffic **every request becomes its own `$session_id`** unless `enableConversationId: true` is set. Enable it on supported v2 wrapping paths. `captureModel: true` works on both revisions, and its value is self-reported and unverified. `sdk-v2.md` has the rest, including MCP Apps compatibility and the current instrumentation gaps. #### Python @@ -242,7 +256,19 @@ analytics = instrument(server, posthog) # wrap right after constructing the ser # register tools as usual — tools added after instrument() are still captured ``` -`instrument()` is idempotent per server and returns an analytics handle (used later for custom events). The same call works on the official `FastMCP`, the low-level `Server`, and jlowin's `fastmcp` 2.0. +`instrument()` is idempotent per server and returns an analytics handle (used later for custom events). The same call works on the official MCP SDK 1.x `FastMCP`, 2.x `MCPServer`, the low-level `Server` from either major, and jlowin's standalone `fastmcp` package. + +For the official `mcp` 2.x SDK, enable conversation IDs because `2026-07-28` has no protocol session: + +```python +from posthog.mcp.types import MCPAnalyticsOptions + +analytics = instrument( + server, + posthog, + MCPAnalyticsOptions(enable_conversation_id=True), +) +``` **Path P2 — custom dispatcher:** swap the existing client for `PostHogMCP` (a drop-in `posthog` client subclass) and call the capture helpers at the dispatch points. Read `custom-servers.md` for the full field reference before editing. @@ -252,8 +278,13 @@ from posthog.mcp import PostHogMCP posthog = PostHogMCP(os.environ["POSTHOG_PROJECT_TOKEN"], host=os.environ["POSTHOG_HOST"]) -# on the initialize handshake: -posthog.capture_initialize(client_name=client_name, client_version=client_version, distinct_id=distinct_id) +# only on a 2025-11-25 initialize handshake: +posthog.capture_initialize( + client_name=client_name, + client_version=client_version, + distinct_id=distinct_id, + protocol_version="2025-11-25", +) # after each tools/call resolves (time it): start = time.monotonic() @@ -266,10 +297,11 @@ posthog.capture_tool_call( is_error=False, distinct_id=distinct_id, # who the request is from, if known session_id=session_id, # your transport/session id, if you have one + protocol_version=protocol_version, # read this from the current request ) ``` -Resolve `distinct_id` / `session_id` from whatever auth/session the dispatcher already has; omit them rather than inventing values. These calls are fire-and-forget and never throw, so they can't take down a tool. +Resolve `distinct_id` / `session_id` from whatever auth/session the dispatcher already has; omit them rather than inventing values. Pass `protocol_version` on every capture. On `2025-11-25`, use the revision the dispatcher's existing session state negotiated during initialize. On `2026-07-28`, read `MCP-Protocol-Version` from the current request because there is no initialize handshake or protocol session. If the dispatcher exposes neither source, omit the property rather than hardcoding a revision. Don't call `capture_initialize` on `2026-07-28`. Python doesn't support self-reported model capture yet, so don't add an `llm_model` field. These calls are fire-and-forget and never throw, so they can't take down a tool. ### STEP 5: Wire up credentials @@ -305,13 +337,17 @@ The PostHog client batches events; the user owns the client's lifecycle. - **TypeScript / JavaScript:** run the project's type-check and/or build script (e.g. `tsc --noEmit`, `pnpm build`) and fix any errors your changes introduced. Run any linter/formatter the project uses on the files you touched. - **Python:** run the project's type-check / tests if present (`mypy`, `pytest`) and fix any errors your changes introduced. Run any formatter the project uses (`ruff`, `black`) on the files you touched. +- For a TypeScript wrapping path with model capture, verify `tools/list` advertises a required `llm_model` string, the tool handler doesn't receive it, and a non-`unknown` answer lands on `$mcp_tool_call` as `$mcp_llm_model` with `$mcp_llm_model_source = "self_reported"`. +- For a `2026-07-28` wrapping path, verify the first tool call captures without an initialize request. When conversation IDs are enabled, verify the returned handle is echoed on the next call and produces the same `$session_id`. +- Don't expect automatic `$mcp_resources_list`, `$mcp_resource_read`, `$mcp_prompts_list`, or `$mcp_prompt_get` events. Those names are reserved, but the wrappers don't emit them yet. +- Check every event name in the final report against `events.md`. Failed tools remain `$mcp_tool_call` events with `$mcp_is_error = true` and can emit a sibling `$exception`; there is no `$mcp_tool_failed` event. - Summarize for the user: which path you used, the files you changed, the env vars to set, and that they'll see `$mcp_*` events in PostHog once the server handles its next request. Link them to https://posthog.com/docs/mcp-analytics for the dashboard and event reference. ## Reference files {references} -`installation.md` is the source of truth for the wrapping paths (A/B and Python P1) and the full `instrument()` options table (`identify`, `context`/intent, `enableConversationId`/`enable_conversation_id`, `reportMissing`/`report_missing`, `beforeSend`/`before_send`, `eventProperties`/`event_properties`). `custom-servers.md` is the source of truth for the custom-dispatcher paths (C and P2) — `PostHogMCP`, `captureToolCall`/`capture_tool_call`, `captureInitialize`/`capture_initialize`, and the per-call field mapping. `sdk-v2.md` (TypeScript only) is the source of truth for everything that differs between MCP TypeScript SDK v1 and v2 — which package is which major, the `getRequestHeaders` migration, sessions on `2026-07-28`, and the revision features not instrumented yet. `intent.md`, `identifying-users.md`, and `conversation-id.md` cover optional enrichment; `events.md` and `custom-events.md` describe what gets captured. The event/property vocabulary is identical across both SDKs. +`installation.md` is the source of truth for the wrapping paths (A/B and Python P1) and the full `instrument()` options table (`identify`, `context`/intent, TypeScript-only `captureModel`, `enableConversationId`/`enable_conversation_id`, `reportMissing`/`report_missing`, `beforeSend`/`before_send`, `eventProperties`/`event_properties`). `custom-servers.md` is the source of truth for the custom-dispatcher paths (C and P2), including the `2026-07-28` rule that no initialize event exists. `sdk-v2.md` is the source of truth for both SDK-major splits, sessions on `2026-07-28`, MCP Apps compatibility, and current instrumentation gaps. `intent.md`, `identifying-users.md`, and `conversation-id.md` cover optional enrichment; `events.md` and `custom-events.md` describe what gets captured. ## Key principles @@ -320,4 +356,4 @@ The PostHog client batches events; the user owns the client's lifecycle. - **Env, never hardcode.** The project token and host come from environment variables. - **Additive only.** Don't change tool behavior or restructure the server — just wrap/capture. - **Don't break STDIO.** No `console.*` (JS) or `print()` (Python) on STDIO transports; use a `logger` instead. -- **Pin the beta SDK** and tell the user it's pre-1.0. (Python: `posthog.mcp` ships inside `posthog`; pin `posthog>=7.21`.) +- **Pin the beta SDK** and tell the user it's pre-1.0. (Python: `posthog.mcp` ships inside `posthog`; require `posthog>=7.40.0` for MCP SDK 2.x support.)