Skip to content

feat(telemetry): add Studio Tool Mount Time Measured event#12708

Open
annez wants to merge 6 commits intomainfrom
telemetry/tool-mount-time
Open

feat(telemetry): add Studio Tool Mount Time Measured event#12708
annez wants to merge 6 commits intomainfrom
telemetry/tool-mount-time

Conversation

@annez
Copy link
Copy Markdown
Contributor

@annez annez commented Apr 20, 2026

Summary

Instruments Gap 2 from the studio performance tracking plan: measure how long each tool takes to render its first commit after activation. Identifies slow custom plugins. Since activeTool is already auto-enriched on every TelemetryContext, we slice by tool without duplicating it into the event payload.

Adds one event under the existing <Feature> <Metric> Measured convention:

Event Fires from Payload
Studio Tool Mount Time Measured ToolMountTimer (inside the active tool's <Suspense>) toolName, durationMs, isFirstMount

Fires on every tool activation, not once per session.

How it works

  • New ToolMountTimer component rendered invisibly inside the active tool's <Suspense> boundary. Its first useEffect runs only after the lazy chunk resolves and the tool's first render commits, giving a fair "tool is on screen" moment.
  • T0 captured via useMemo keyed on activeToolName in StudioLayoutComponent — stable per activation, no render-time ref writes.
  • The existing <StudioErrorBoundary key={activeTool?.name}> already re-keys on tool switch, so the timer naturally unmounts/remounts per tool, producing exactly one event per activation.
  • Module-level mountedTools: Set<string> tracks which tools have already been activated this page session, so isFirstMount distinguishes cold-start (includes lazy-chunk fetch) from warm switches back to an already-mounted tool.

v1 scope note

v1 measures only to first React commit. A plugin-facing Tool.onReady lifecycle hook for tools that do async work post-mount (e.g. Vision fetching CORS origins) is explicitly deferred to v2 — see the telemetry gaps implementation plan, D2.1. Plugins that do significant async work after mount will under-report here; we'll extend the instrumentation if the data motivates it.

Files touched

  • new packages/sanity/src/core/studio/__telemetry__/tools.telemetry.ts — event definition
  • new packages/sanity/src/core/studio/ToolMountTimer.tsx — invisible timer component
  • packages/sanity/src/core/studio/StudioLayout.tsx — capture T0 with useMemo, render <ToolMountTimer> inside <Suspense>
  • new packages/sanity/src/core/studio/__tests__/ToolMountTimer.test.tsx

Tests

Cover:

  • fires once on mount with isFirstMount: true on first activation
  • second mount of the same tool → isFirstMount: false (warm switch)
  • distinct tools each report isFirstMount: true (tracked separately)
  • durationMs correctly reflects the delta from the provided t0
  • renders nothing visible

Manual verification

Run with SANITY_STUDIO_DEBUG_TELEMETRY=true pnpm dev. Open the Studio, switch between structure / vision / presentation / any custom plugin — each switch produces one Studio Tool Mount Time Measured event. Switching back to a previously-visited tool sets isFirstMount: false.

Scope

Draft. Second of four PRs staged from /research/telemetry-gaps-implementation-plan.md (internal); Gap 1 is #12707. Does not change any plugin-facing API (that's deferred to v2). Sequential with Gap 4 (search latency) and Gap 3 (document initial load) to follow.

Instruments Gap 2 from the studio-performance-tracking plan: measure
how long each tool (desk, presentation, vision, custom plugins) takes
to render its first commit after activation. Identifies slow custom
plugins — since `activeTool` is already auto-enriched on every
`TelemetryContext`, we can slice by tool without duplicating it into
the event payload.

Adds one event under the existing `<Feature> <Metric> Measured`
convention:

- `Studio Tool Mount Time Measured` — fires on every tool activation,
  not once per session. Payload: `toolName`, `durationMs`,
  `isFirstMount`.

Implementation:

- New `ToolMountTimer` component rendered invisibly inside the active
  tool's `<Suspense>` boundary. Its first `useEffect` runs only after
  the lazy chunk resolves and the tool's first render commits, giving
  us a fair "tool is on screen" moment.
- T0 captured via `useMemo` keyed on `activeToolName` in
  `StudioLayoutComponent` — stable per activation, no render-time ref
  writes.
- The existing `<StudioErrorBoundary key={activeTool?.name}>` already
  re-keys on tool switch, so the timer naturally unmounts/remounts per
  tool, producing exactly one event per activation.
- Module-level `mountedTools: Set<string>` tracks which tools have
  already been activated this page session, so `isFirstMount`
  distinguishes cold-start (includes lazy-chunk fetch) from warm
  switches back to an already-mounted tool.

v1 measures only to first React commit. A plugin-facing `Tool.onReady`
lifecycle hook for tools that do async work post-mount is explicitly
deferred to v2 (see implementation plan D2.1).
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 20, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
page-building-studio Ready Ready Preview, Comment Apr 21, 2026 8:40am
test-studio Ready Ready Preview, Comment Apr 21, 2026 8:40am

Request Review

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 20, 2026

📦 Bundle Stats — sanity

Compared against main (9060f828) · v5.21.0 (npm)

sanity

Metric Value vs main (9060f82) vs v5.21.0
Internal (raw) 4.42 MB +1.5 KB, +0.0% +5.0 KB, +0.1%
Internal (gzip) 1.02 MB +459 B, +0.0% +1.3 KB, +0.1%
Bundled (raw) 12.09 MB +1.5 KB, +0.0% -37.8 KB, -0.3%
Bundled (gzip) 2.72 MB +458 B, +0.0% -7.1 KB, -0.3%
Import time 1.62s -5ms, -0.3% +84ms, +5.5%

bin:sanity

Metric Value vs main (9060f82) vs v5.21.0
Internal (raw) 7.1 KB - -
Internal (gzip) 2.9 KB - -
Bundled (raw) 7.1 KB - -
Bundled (gzip) 2.8 KB - -
Import time 6ms +0ms, +1.9% +1ms, +12.6% ⚠️

🗺️ View treemap · Artifacts

Details
  • Import time regressions over 10% are flagged with ⚠️
  • Sizes shown as raw / gzip 🗜️. Internal bytes = own code only. Total bytes = with all dependencies. Import time = Node.js cold-start median.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 20, 2026

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 20, 2026

📚 TypeDoc Generation Result

TypeDoc generated successfully!

  • File size: 8.2M
  • Total exports: 1065
  • Artifact: sanity-typedoc-1534be04c3892e1e9cf5ff7cd21c73cd3c13b9a2

The TypeDoc JSON file has been generated and validated. All documentation scripts completed successfully.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 20, 2026

⚡️ Editor Performance Report

Updated Tue, 21 Apr 2026 08:45:30 GMT

Benchmark reference
latency of sanity@latest
experiment
latency of this branch
Δ (%)
latency difference
arrayI18n (simple-en) 66.7 efps (15ms) 60.6 efps (17ms) +2ms (+10.0%)
article (title) 41.7 efps (24ms) 46.5 efps (22ms) -3ms (-10.4%)
article (body) 49.1 efps (20ms) 47.2 efps (21ms) +1ms (+4.2%)
article (string inside object) 42.6 efps (24ms) 44.4 efps (23ms) -1ms (-4.3%)
article (string inside array) 45.5 efps (22ms) 45.5 efps (22ms) +0ms (-/-%)
recipe (name) 99.9+ efps (10ms) 99.9+ efps (10ms) +0ms (-/-%)
recipe (description) 52.6 efps (19ms) 52.6 efps (19ms) +0ms (-/-%)
recipe (instructions) 99.9+ efps (5ms) 99.9+ efps (6ms) +1ms (-/-%)
singleString (stringField) 99.9+ efps (7ms) 99.9+ efps (7ms) +0ms (-/-%)
synthetic (title) 50.0 efps (20ms) 52.6 efps (19ms) -1ms (-5.0%)
synthetic (string inside object) 50.0 efps (20ms) 52.6 efps (19ms) -1ms (-5.0%)

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark latency p75 p90 p99 blocking time test duration
arrayI18n (simple-en) 15ms 20ms 23ms 43ms 1ms 5.8s
article (title) 24ms 31ms 42ms 67ms 15ms 7.5s
article (body) 20ms 31ms 59ms 97ms 186ms 5.9s
article (string inside object) 24ms 29ms 52ms 63ms 28ms 6.6s
article (string inside array) 22ms 28ms 38ms 58ms 1ms 6.8s
recipe (name) 10ms 13ms 17ms 35ms 0ms 5.3s
recipe (description) 19ms 21ms 23ms 36ms 0ms 4.4s
recipe (instructions) 5ms 8ms 11ms 17ms 0ms 3.0s
singleString (stringField) 7ms 9ms 10ms 25ms 0ms 4.5s
synthetic (title) 20ms 26ms 53ms 124ms 1438ms 9.7s
synthetic (string inside object) 20ms 28ms 75ms 133ms 1185ms 9.9s

🧪 Experiment result

The performance result of this branch

Benchmark latency p75 p90 p99 blocking time test duration
arrayI18n (simple-en) 17ms 18ms 28ms 54ms 1ms 5.9s
article (title) 22ms 25ms 31ms 53ms 22ms 7.2s
article (body) 21ms 32ms 58ms 81ms 185ms 5.5s
article (string inside object) 23ms 29ms 34ms 62ms 27ms 6.4s
article (string inside array) 22ms 27ms 31ms 56ms 0ms 6.6s
recipe (name) 10ms 12ms 16ms 40ms 0ms 5.3s
recipe (description) 19ms 22ms 26ms 32ms 0ms 4.5s
recipe (instructions) 6ms 10ms 13ms 32ms 0ms 3.1s
singleString (stringField) 7ms 9ms 12ms 20ms 0ms 4.6s
synthetic (title) 19ms 26ms 81ms 149ms 1585ms 10.3s
synthetic (string inside object) 19ms 26ms 70ms 128ms 1099ms 9.5s

📚 Glossary

column definitions

  • benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
  • latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
  • p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
  • p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
  • p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
  • blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
  • test duration — how long the test run took to complete.

@github-actions
Copy link
Copy Markdown
Contributor

⚡️ Editor Performance Report

Deploying studio and running performance tests…

- Replace impure useMemo(performance.now()) with a ref written
  inside an effect on activeToolName change. React Compiler forbids
  calling impure functions during render; the effect still runs
  before Suspense resolves (the fallback renders first), so the
  delta measured in ToolMountTimer still includes lazy-chunk time.
- Pass t0 via RefObject rather than a value prop, so the timer
  reads it in its own effect (post-render).
- Remove dead 'eslint-disable-next-line react-hooks/exhaustive-deps'
  on ToolMountTimer's effect; use proper deps [telemetry, toolName,
  t0Ref], all stable within a single mount (parent re-keys the
  StudioErrorBoundary per tool, so one mount = one tool activation).
- Remove two unused 'eslint-disable-next-line react-hooks/static-
  components' comments (ESLint reported them as unused).
- Replace 'typeof import("...")' type annotations in the test file
  with top-of-file 'import type' to satisfy
  @typescript-eslint/consistent-type-imports.
- Add a new test case: does not fire when t0Ref.current is null
  (covers the new guard path).
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 21, 2026

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 30.22% 20271 / 67062
🔵 Statements 24.5% 26546 / 108336
🔵 Functions 20.16% 3717 / 18437
🔵 Branches 16.37% 14278 / 87178
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
packages/sanity/src/core/studio/StudioLayout.tsx 1.69% 1.32% 0% 3.57% 27-29, 75-241
packages/sanity/src/core/studio/ToolMountTimer.tsx 71.42% 57.14% 50% 90.9% 49, 35-51
Generated in workflow #55117 for commit 4e33e9e by the Vitest Coverage Report Action

…iveToolLayout

Per @studiolead: my earlier patch removed two eslint-disable comments
that ESLint had flagged as 'unused' at pre-patch line numbers. The
disables were actually still needed — just moved line numbers because
my diff shifted them. Post-patch they fired:

  205:10  Cannot create components during render  (on <Navbar />)
  232:18  Cannot create components during render  (on <ActiveToolLayout ... />)

Both components come from hooks (useNavbarComponent,
useActiveToolLayoutComponent) that return middleware-resolved
component references. That's the intended pattern but trips
react-hooks/static-components.

Restored with '-- reason' explanations so the disables are
self-documenting.

Also: oxfmt formatting on ToolMountTimer.test.tsx (import order +
inline one render() call that fit under 100 chars).
Satisfies import/consistent-type-specifier-style. The repo convention
is '{type X}' inline, not 'import type {X}'.
@annez
Copy link
Copy Markdown
Contributor Author

annez commented Apr 21, 2026

Rebased onto current main. No conflicts (this branch already had a merge-from-main before; the rebase preserved the 6-commit history cleanly).

@annez
Copy link
Copy Markdown
Contributor Author

annez commented Apr 21, 2026

Correction on my previous comment — checking more carefully, this branch was already up to date with main at the tip (4e33e9e7b6 Merge branch 'main' into telemetry/tool-mount-time). No rebase needed; nothing pushed. Status: 6 ahead, 0 behind.

The stale-base problem only applied to #12707, #12709, #12710.

@jordanl17 jordanl17 marked this pull request as ready for review April 29, 2026 15:50
@jordanl17 jordanl17 requested review from a team as code owners April 29, 2026 15:50
@jordanl17 jordanl17 requested review from juice49 and removed request for a team April 29, 2026 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants