Repository navigation
test(ci): Validate Namespace runners for CI benchmarks - #26328
nicohrubec wants to merge 13 commits into
Conversation
Co-Authored-By: OpenAI Codex <codex@openai.com>
size-limit report 📦Compared: 084f289 (PR test merge using develop at f49009b) Against: f49009b (develop) · base run #65866 ✅ Baseline matches: this diff contains only this PR's changes.
|
| key: playwright-${{ runner.os }}-iteration-1-${{ steps.playwright-version.outputs.version }} | ||
| key: | ||
| ${{ env.CI_BENCHMARK_CACHE_PREFIX }}playwright-${{ runner.os }}-${{ runner.arch }}-iteration-1-${{ | ||
| steps.playwright-version.outputs.version }} |
There was a problem hiding this comment.
Playwright always reinstalls on Namespace
Medium Severity
On Namespace the Playwright restore step is skipped, so cache-hit is never true and npx playwright install always runs. Warm volume hits still fetch the Playwright package from the registry, so warm runs stay network-dependent and the cold/warm comparison is skewed.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 22e8ffe. Configure here.
…olicy Co-AuthoredBy: OpenAI Codex <codex@openai.com> Co-Authored-By: OpenAI Codex <codex@openai.com>
Optional E2E testsSome optional checks were not successful. They do not block the merge. 5 not successful, 55 successful. See the run.
Successful checks
|
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 2882be8. Configure here.
| env: | ||
| HEAD_COMMIT: ${{ github.event.inputs.commit || github.sha }} | ||
| CI_BENCHMARK_PROVIDER: namespace | ||
| CI_BENCHMARK_CACHE_PREFIX: bench-sentry-js-20261112-namespace-b- |
There was a problem hiding this comment.
Playwright install ignores volume cache hits
Medium Severity
Browser install still keys off steps.playwright-cache.outputs.cache-hit, but that restore step is skipped when CI_BENCHMARK_PROVIDER is namespace. A skipped step’s output is empty, so != 'true' is always true and every job runs npx -y playwright install. Warm Namespace runs therefore still fetch and invoke Playwright even when ~/.cache/ms-playwright is already on the volume, which distorts the warm-1 benchmark this commit is meant to measure.
Reviewed by Cursor Bugbot for commit 2882be8. Configure here.
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>


Why: Validate today’s CI on our previous Namespace runner profiles and persistent dependency volumes while retaining the recent build, sharding and browser dependency improvements. This draft uses isolated caches and temporarily permits cache saves on the benchmark branch so we can verify the setup before comparing repeated cold and warm runs.