Skip to content

test(web): 400 ms latency gate for send, thread switch and launch - #117

Merged
nullStack65 merged 11 commits into
release/native-0.1.10from
feat/clo-868-feature-map-perf
Nov 21, 2026
Merged

nullStack65 merged 11 commits into
release/native-0.1.10from
feat/clo-868-feature-map-perf

Conversation

@nullStack65

@nullStack65 nullStack65 commented Nov 20, 2026 •

Copy link
Copy Markdown
Owner

Kameron wants interactions under 400 ms. Nothing measured that, so slow paths such as the transcribe button's 1–3 s "starting" shipped unnoticed.

  • New latency vitest browser project (apps/web/vite.config.ts). Run it with vp run test:latency in apps/web; it uses production React, and a warm-up test fails if it finds the development build.
  • apps/web/test/interactionLatency.ts sets a 400 ms budget and gates on the median of five samples.
    • Each sample runs from the input event's timestamp until a task after the first frame that paints the result.
    • The result must not already show before the input.
    • Every sample is printed to the CI log.
  • apps/web/test/app.latency.tsx mounts the whole app on the in-process fake environment. Medians on an idle ASUS:
    • composer send (Enter until the message shows in the thread): about 66 ms. Gated.
    • thread switch (sidebar row click until the other thread is open): about 150 ms. Gated.
    • app launch (mount until a thread and its composer show, warm JS, no server startup): about 350 ms. Reported, but not gated, because it is too close to the budget for a shared runner.
  • Latency tests are named *.latency.ts(x), so only this project runs them. The transcribe press-to-recording test on fix/clo-919-transcribe-latency joins the gate once it is renamed to src/speech/voiceInput.latency.ts.
  • ARC CI installs Playwright's headless Chromium. It then extracts the Chromium system libraries without root and fails if ldd still finds a missing one. The package list is copied from Playwright 1.64.1's Ubuntu 24.04 table, so update it if the Playwright version or runner image changes.

Not measured: desktop Electron cold launch. Earlier numbers from a heavily loaded ASUS (thread switch about 510 ms, launch about 1.5 s) came from host load and from a timing check that scanned the whole page on every DOM change. The check now runs once per frame.

Deploy: none

Model: claude-opus-5-5
Task: CLO-868 feature map lane (perf), CLO-919 item 4 gate

🤖 Generated with Claude Code

nullStack65 and others added 6 commits November 20, 2026 01:38
Adds a latency vitest browser project run on production React, timed from the
input event to the frame after the UI shows the result, median of five
samples. Composer send is gated. Thread switch and mount to an open thread are
measured and reported but not gated yet because they currently miss the
budget. ARC CI installs headless Chromium unprivileged and runs the gate.

Model: claude-opus-5-5
Task: implementation
Agent: Claude Code (CLO-868 feature map lane)
Lane: magpie
Harness: t3/claude-code
Model-family: Claude
Model-generation: 5.5
Skills-SHA: df60fa7be83ccc7a8e3c64e3c58ba9ed7a84cfc4
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… own list

The dry-run parse found nothing on the ARC runner, so the gate launched
Chromium without libnspr4. The step now resolves Playwright's Ubuntu 24.04
Chromium packages directly and fails with the missing names if ldd still
reports any.

Model: claude-opus-5-5
Task: implementation
Agent: Claude Code (CLO-868 feature map lane)
Lane: magpie
Harness: t3/claude-code
Model-family: Claude
Model-generation: 5.5
Skills-SHA: df60fa7be83ccc7a8e3c64e3c58ba9ed7a84cfc4
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…imports it

Model: claude-opus-5-5
Task: implementation
Agent: Claude Code (CLO-868 feature map lane)
Lane: magpie
Harness: t3/claude-code
Model-family: Claude
Model-generation: 5.5
Skills-SHA: df60fa7be83ccc7a8e3c64e3c58ba9ed7a84cfc4
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review found the done check ran a full-page text scan on every mutation inside
the timed window, the end time was taken before the final paint, a missing
input event hung until the test timeout, and the transcribe test name would
also run in the dev-React browser project. Each frame now checks once and the
end is read in a task after that frame paints; the result must not already
show before the input, and an action without input fails at once. Latency
tests are named *.latency.ts so only the latency project runs them, and the
report goes to stdout through a browser command so CI keeps every sample.
Measured on an idle ASUS: composer send about 66 ms and thread switch about
150 ms, both gated now; mount to an open thread about 350 ms, reported but not
gated.

Model: claude-opus-5-5
Task: implementation
Agent: Claude Code (CLO-868 feature map lane)
Lane: magpie
Harness: t3/claude-code
Model-family: Claude
Model-generation: 5.5
Skills-SHA: df60fa7be83ccc7a8e3c64e3c58ba9ed7a84cfc4
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rst mount

On the ARC runner the first mount's connection failed transiently and the
thread never showed. The mount stayed in the page, so later tests found two
copies of every thread. A thread that fails to open now unmounts, and the
warm-up's first mount may fail before the measured mounts start.

Model: claude-opus-5-5
Task: implementation
Agent: Claude Code (CLO-868 feature map lane)
Lane: magpie
Harness: t3/claude-code
Model-family: Claude
Model-generation: 5.5
Skills-SHA: df60fa7be83ccc7a8e3c64e3c58ba9ed7a84cfc4
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runner image has no fonts, so text laid out with no size and every
visibility wait in the latency gate timed out, while the text-only app launch
check passed. The step now extracts DejaVu and fontconfig, points Chromium at
them with FONTCONFIG_FILE, and fails early when no font is found.

Model: claude-opus-5-5
Task: implementation
Agent: Claude Code (CLO-868 feature map lane)
Lane: magpie
Harness: t3/claude-code
Model-family: Claude
Model-generation: 5.5
Skills-SHA: df60fa7be83ccc7a8e3c64e3c58ba9ed7a84cfc4
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

nullStack65 and others added 2 commits November 20, 2026 18:05
The simulation wrote its log where the runner cannot, so the 400 ms latency
gate died before running. Download-only is the path the voice gate already uses.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
fonts-dejavu-core depends on fonts-dejavu-mono, and resolving that
dependency makes apt clean the root archive dir, which this runner cannot
write. The library list now matches the voice step that already passes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@nullStack65

Copy link
Copy Markdown
Owner Author
{"reviewed_head":"3d4e72b99ed340d11709ba4b13daed4a484a664a","reviewer":"grok-4.7-build (claudeAgent, read-only)","findings":[]}

The review of the CI step at 3d4e72b was clean. CI then failed again in "Gate interaction latency at 400 ms" with rm: cannot remove '/var/cache/apt/archives/partial/*.deb': Permission denied. The download list included fonts-dejavu-core, which depends on fonts-dejavu-mono; resolving that dependency sends apt back to the root archive dir. Dropped the font package so the list matches the voice step that already passes, in a0641c2. This is fix attempt 2 of the same failure. If it fails again it goes back to the caller rather than a third attempt.

libfontconfig1 depends on fontconfig-config, which depends on
fonts-dejavu-core and fonts-dejavu-mono. Resolving those sends apt back to
the root archive dir, which this runner cannot write. The package list now
matches the voice step that already passes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@nullStack65

Copy link
Copy Markdown
Owner Author
{"reviewed_head":"59067a6a6233061bc84b996cd9daf5fd853466fd","reviewer":"grok-4.7-build (claudeAgent, read-only)","findings":[]}

Quick re-check of 59067a6a62 (parent a0641c2aea2d), the commit that drops libfontconfig1 from the "Gate interaction latency at 400 ms" step. CLEAN: the only change is that package line, the remaining 21 packages match the passing voice step token for token, the workflow parses, and the fontconfig cache, font-count check and ldd check are untouched. The reviewer reported it ran as grok-4.7-build rather than grok/grok-4.7.

nullStack65 and others added 2 commits November 20, 2026 20:12
apt-get update runs APT::Update::Post-Invoke from the runner image, and that
hook removes /var/cache/apt/archives/partial/*.deb, which this runner cannot
write. Job 136517003866 died there before the 400 ms gate ran. Pointing
Dir::Etc::Main and Dir::Etc::Parts at empty files in the runner temp dir
drops those hooks while the download still uses the system source lists.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
apt applies -o Dir::Etc only after it has already read
/etc/apt/apt.conf.d, so the runner image Post-Invoke still removes
/var/cache/apt/archives/partial/*.deb and fails the 400 ms gate.
APT_CONFIG is read first. Point it at an empty config under the job
temp dir, keep the image dpkg status, and download only the Chromium
libraries that image does not already have.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@nullStack65
nullStack65 merged commit 7e9326e into release/native-0.1.10 Nov 21, 2026
3 of 4 checks passed
@nullStack65
nullStack65 deleted the feat/clo-868-feature-map-perf branch November 21, 2026 00:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant