fix(web-analytics): default percentile and thresholds for query-web-vitals

[posthog]

Problem

The query-web-vitals MCP tool required three fields: metric, percentile, and thresholds. But two of those have obvious defaults. percentile should be p75 (the percentile the Google Core Web Vitals bands are defined at), and thresholds are the standard Google band constants that are fully determined by which metric you pick. It's the only web-analytics query wrapper with any required field beyond kind.

So agents doing ordinary read-only per-page vitals analysis (LCP / INP / CLS / FCP) kept calling it with just { "metric": "LCP" } and getting a schema validation error. Over a recent seven-day window this tool failed 23.0% of calls (80 / 348), across many agent clients, and every failure was a validation error rather than an upstream or quota issue. The project-wide baseline is ~4.1%.

I confirmed the failure mode locally: unknown extra keys are stripped by the tool's Zod schema (so those don't cause failures), while a minimal metric-only or metric+percentile call was rejected before the fix. That points squarely at the two over-required fields.

Changes

• Make percentile and thresholds optional on both AssistantWebVitalsPathBreakdownQuery and the full WebVitalsPathBreakdownQuery (schema source of truth in frontend/src/queries/schema/), and regenerate schema.json, posthog/schema.py, the generated MCP Zod wrapper, and the tool-schema snapshot.
• Default the two fields in WebVitalsPathBreakdownQueryRunner: percentile → p75, thresholds → the metric's standard Google bands (LCP [2500, 4000], INP [200, 500], CLS [0.1, 0.25], FCP [1800, 3000]), mirroring the frontend WEB_VITALS_THRESHOLDS constant. Both the raw and lazy-precompute read paths go through the resolved values.
• metric stays required (there's no sensible default for which vital). This is not a loosening of validation: malformed inputs are still rejected. A metric-only call now produces a fully valid, correctly banded p75 result.
• Update the tool prompt/examples so the minimal { "metric": "LCP" } shape is the lead example and the defaults are documented.

The result rate should move from the 23.0% baseline toward the ~4.1% project baseline, because the two fields agents most commonly omitted are no longer required. This can only be confirmed once deployed, since it depends on live invocation shapes.

How did you test this code?

Automated tests I (Claude) actually ran:

• New MCP schema test (services/mcp/tests/unit/query-web-vitals-schema.test.ts): a metric-only call validates and leaves percentile/thresholds unset for the backend to fill; the explicit p75 shape still validates; and malformed inputs (missing/unknown metric, bad percentile, wrong-length thresholds, non-string date_from) are rejected with an error whose path names the offending field. Passing.
• Full MCP unit suite: 2375 tests passing, including the regenerated tool-schema snapshot and the codegen tests.
• New backend test (TestWebVitalsPathBreakdownDefaults, parameterized over the four metrics): a metric-only query buckets pages into good / needs_improvements / poor using the defaulted p75 + standard bands. This is a ClickHouse-backed test, so it runs in CI; I could not execute it locally (no ClickHouse in the sandbox), but I verified the defaulting logic, ran mypy clean on the changed backend files, and confirmed all four metrics are covered by the defaults map.

I regenerated every derived artifact with the repo's own tooling (schema:build:json, build-schema-python.sh, the MCP tool codegen, and the snapshot updater) rather than hand-editing, so the generated files match what CI regeneration produces.

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

🤖 Agent context

Autonomy: Fully autonomous

Authored by Claude (Opus 4.8) via Claude Code, acting on a PostHog Signals inbox report about the query-web-vitals validation failure rate.

Approach: reproduced the baseline from $mcp_tool_call telemetry (all failures classified validation, no free-text message), then traced the tool through its generated Zod schema and the backend query runner. The root cause was the over-required percentile/thresholds. I chose to make them optional with backend-applied defaults rather than only rewording the prompt, since a prompt change alone wouldn't stop agents from omitting fields that have canonical defaults. Kept metric required so validation still rejects genuinely underspecified queries. Invoked the /writing-tests skill before adding tests.

---

Created with PostHog Code from this inbox report.

[trunk-io]

Merging to master in this repository is managed by Trunk.

• To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

[github-actions]

🤖 CI report

⚠️ Bundle size — 🔺 +621 B (+0.0%)

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 64.66 MiB · 🔺 +621 B (+0.0%)

No file changed by more than 1000 B.

_{Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report}

✅ Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root	Eager (shipped)	Δ vs base	Budget
entry (logged-out pages, app bootstrap) src/index.tsx	1.24 MiB · 22 files	no change	███░░░░░░░ 27.5% of 4.51 MiB
authenticated shell (every logged-in page) src/scenes/AuthenticatedShell.tsx	8.21 MiB · 3,000 files	no change	████████░░ 84.5% of 9.71 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx

Largest files eagerly shipped from src/index.tsx

Size	File
126.8 KiB	../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB	../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB	../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.5 KiB	../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
3.9 KiB	../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.4 KiB	../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.3 KiB	src/RootErrorBoundary.tsx
912 B	../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js
789 B	src/scenes/ChunkLoadErrorBoundary.tsx
762 B	src/index.tsx

Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx

Size	File
281.3 KiB	../node_modules/.pnpm/posthog-js@1.406.2/node_modules/posthog-js/dist/rrweb.js
267.7 KiB	../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
236.0 KiB	src/taxonomy/core-filter-definitions-by-group.json
224.7 KiB	../node_modules/.pnpm/posthog-js@1.406.2/node_modules/posthog-js/dist/module.js
167.1 KiB	src/queries/validators.js
154.3 KiB	../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB	../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
105.8 KiB	src/lib/api.ts
94.0 KiB	../packages/quill/packages/quill/dist/index.js
93.3 KiB	../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js

_{Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479}

✅ Toolbar bundle — eager 2.18 MiB within budget

What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.

Metric	Size	Δ vs base	Budget
Eager (shipped) entry + static imports	2.18 MiB · 17 files	no change	████░░░░░░ 38.1% of 5.72 MiB
Deferred (lazy)	2.07 MiB · 33 files	no change	n/a — loads on demand
Loader dist/toolbar.js	1.1 KiB	no change	█░░░░░░░░░ 5.8% of 19.5 KiB

Largest eagerly-shipped chunks

Size	File
713.8 KiB	dist/toolbar/toolbar-app-HY7HJI4V.css
543.6 KiB	dist/toolbar/chunk-chunk-UG3THN3N.js
484.2 KiB	dist/toolbar/chunk-chunk-QS5AHYGW.js
133.6 KiB	dist/toolbar/chunk-chunk-MCXISDMN.js
131.8 KiB	dist/toolbar/chunk-chunk-T5KY5WYR.js
71.0 KiB	dist/toolbar/toolbar-app-UDDFB6JG.js
69.0 KiB	dist/toolbar/chunk-chunk-27JL52RE.js
35.6 KiB	dist/toolbar/chunk-chunk-XVKSNBZ7.js
20.9 KiB	dist/toolbar/chunk-chunk-CS7W2KTV.js
12.2 KiB	dist/toolbar/chunk-chunk-PIK3PADE.js

_{Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile}

✅ Dist folder size — 🔺 +1.7 KiB (+0.0%)

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 1357.18 MiB · 🔺 +1.7 KiB (+0.0%)