Skip to content

test(ci): Validate Namespace runners for CI benchmarks - #26328

Closed
nicohrubec wants to merge 13 commits into
developfrom
feat/namespace-ci-smoke
Closed

nicohrubec wants to merge 13 commits into
developfrom
feat/namespace-ci-smoke

Conversation

@nicohrubec

Copy link
Copy Markdown
Member

Why: Validate today’s CI on our previous Namespace runner profiles and persistent dependency volumes while retaining the recent build, sharding and browser dependency improvements. This draft uses isolated caches and temporarily permits cache saves on the benchmark branch so we can verify the setup before comparing repeated cold and warm runs.

Co-Authored-By: OpenAI Codex <codex@openai.com>
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

size-limit report 📦

Compared: 084f289 (PR test merge using develop at f49009b)

Against: f49009b (develop) · base run #65866

✅ Baseline matches: this diff contains only this PR's changes.

Path Size % Change Change
@sentry/browser 30.04 kB - -
@sentry/browser - with treeshaking flags 28.23 kB - -
@sentry/browser - with treeshaking flags tracing without tracing 28.12 kB - -
@sentry/browser (incl. Tracing) 52.2 kB - -
@sentry/browser (incl. Tracing + Span Streaming) 52.2 kB - -
@sentry/browser (incl. Tracing, Profiling) 55.17 kB - -
@sentry/browser (incl. Tracing, Replay) 92.73 kB - -
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 81.52 kB - -
@sentry/browser (incl. Tracing, Replay with Canvas) 97.08 kB - -
@sentry/browser (incl. Tracing, Replay, Feedback) 110.51 kB - -
@sentry/browser (incl. Feedback) 47.66 kB - -
@sentry/browser (incl. sendFeedback) 30.62 kB - -
@sentry/browser (incl. FeedbackAsync) 35.62 kB - -
@sentry/browser (incl. Metrics) 31.36 kB - -
@sentry/browser (incl. Logs) 31.64 kB - -
@sentry/browser (incl. Metrics & Logs) 32.3 kB - -
@sentry/react 32.04 kB - -
@sentry/react (incl. Tracing) 54.86 kB - -
@sentry/vue 38.21 kB - -
@sentry/vue (incl. Tracing) 55.14 kB - -
@sentry/svelte 30.05 kB - -
@sentry/remix (Remix 3 client bundle) 57.96 kB - -
CDN Bundle 31.79 kB - -
CDN Bundle (incl. Tracing) 52.58 kB - -
CDN Bundle (incl. Logs, Metrics) 34.58 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) 55.14 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) 75.61 kB - -
CDN Bundle (incl. Tracing, Replay) 90.63 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) 93.25 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) 96 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) 98.6 kB - -
CDN Bundle - uncompressed 92.69 kB - -
CDN Bundle (incl. Tracing) - uncompressed 155.13 kB - -
CDN Bundle (incl. Logs, Metrics) - uncompressed 101.03 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) - uncompressed 162.76 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) - uncompressed 231.11 kB - -
CDN Bundle (incl. Tracing, Replay) - uncompressed 275.27 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) - uncompressed 282.9 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 287.07 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) - uncompressed 294.69 kB - -
@sentry/nextjs (client) 58.23 kB - -
@sentry/sveltekit (client) 53.26 kB - -
@sentry/core/server 44.53 kB - -
@sentry/core/browser 13.99 kB - -
@sentry/node 149.35 kB -0.01% -4 B 🔽
@sentry/node/import (ESM hook with diagnostics-channel injection) 83.18 kB - -
@sentry/node - without tracing 88.27 kB -0.01% -4 B 🔽
@sentry/node - without channel injection 138.45 kB -0.01% -3 B 🔽
@sentry/aws-serverless 97.19 kB -0.01% -7 B 🔽
@sentry/cloudflare (withSentry) - minified 204.35 kB - -
@sentry/cloudflare (withSentry) 474.32 kB - -
@sentry/nextjs/cloudflare (withSentry) - minified 229.66 kB - -

View base workflow run

if (stat.isSymbolicLink()) {
target = bin;
} else if (stat.isFile() && stat.size < 64 * 1024) {
const match = readFileSync(bin, 'utf8').match(SHIM_TARGET);

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

key: playwright-${{ runner.os }}-iteration-1-${{ steps.playwright-version.outputs.version }}
key:
${{ env.CI_BENCHMARK_CACHE_PREFIX }}playwright-${{ runner.os }}-${{ runner.arch }}-iteration-1-${{
steps.playwright-version.outputs.version }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Playwright always reinstalls on Namespace

Medium Severity

On Namespace the Playwright restore step is skipped, so cache-hit is never true and npx playwright install always runs. Warm volume hits still fetch the Playwright package from the registry, so warm runs stay network-dependent and the cold/warm comparison is skewed.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 22e8ffe. Configure here.

Comment thread .github/workflows/build.yml Resolved
…olicy

Co-AuthoredBy: OpenAI Codex <codex@openai.com>

Co-Authored-By: OpenAI Codex <codex@openai.com>
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Optional E2E tests

Some optional checks were not successful. They do not block the merge. 5 not successful, 55 successful. See the run.

Check Status
❌ CI: Optional E2E Tests / E2E cloudflare-workers-send-to-sentry Test (optional) Failing after 6m Details
❌ CI: Optional E2E Tests / E2E node-eve Test (optional) Failing after 2m Details
❌ CI: Optional E2E Tests / E2E node-eve (latest) Test (optional) Failing after 2m Details
❌ CI: Optional E2E Tests / E2E nextjs-16-cacheComponents (cloudflare) Test (optional) Failing after 10m Details
❌ CI: Optional E2E Tests / E2E node-eve (orchestrion) Test (optional) Failing after 2m Details
Successful checks
Check Status
✅ CI: Optional E2E Tests / E2E astro-7-static Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E gen-ai-libraries Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nestjs-12 (deno) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nestjs-12 (bun) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E cloudflare-pi-durable (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E debug-id-sourcemaps Test (optional) Successful in 3m Details
✅ CI: Optional E2E Tests / E2E astro-7 Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E gen-ai-libraries (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E cloudflare-mastra Test (optional) Successful in 3m Details
✅ CI: Optional E2E Tests / E2E cloudflare-pi-durable Test (optional) Successful in 49s Details
✅ CI: Optional E2E Tests / E2E node-fastify-6 Test (optional) Successful in 41s Details
✅ CI: Optional E2E Tests / E2E cloudflare-flue-send-to-sentry Test (optional) Successful in 3m Details
✅ CI: Optional E2E Tests / E2E deno (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-14 (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E cloudflare-think Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (vinext, bun) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (cloudflare, latest) Test (optional) Successful in 4m Details
✅ CI: Optional E2E Tests / E2E nextjs-15 (latest) Test (optional) Successful in 4m Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (deno) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E cloudflare-agent Test (optional) Successful in 50s Details
✅ CI: Optional E2E Tests / E2E node-flue Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E cloudflare-think (ai v6) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nestjs-12 (cloudflare) Test (optional) Successful in 53s Details
✅ CI: Optional E2E Tests / E2E nextjs-15 (turbo) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-flue (latest) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E tanstackstart-react-cloudflare Test (optional) Successful in 3m Details
✅ CI: Optional E2E Tests / E2E node-express-send-to-sentry Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-pi-durable (latest) Test (optional) Successful in 37s Details
✅ CI: Optional E2E Tests / E2E supabase-nextjs Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps Test (optional) Successful in 37s Details
✅ CI: Optional E2E Tests / E2E tanstackstart-react-sourcemaps Test (optional) Successful in 40s Details
✅ CI: Optional E2E Tests / E2E gen-ai-libraries (cloudflare) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nuxt-5 Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nextjs-orpc Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (vinext) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-express-mcp-v1 Test (optional) Successful in 3m Details
✅ CI: Optional E2E Tests / E2E node-pi-durable Test (optional) Successful in 49s Details
✅ CI: Optional E2E Tests / E2E node-openai-decisions Test (optional) Successful in 47s Details
✅ CI: Optional E2E Tests / E2E solidstart-2 Test (optional) Successful in 46s Details
✅ CI: Optional E2E Tests / E2E node-mastra Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nuxt-5 (tunnel-dev) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-mastra (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-fastify-6 (tracer provider) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E react-send-to-sentry Test (optional) Successful in 3m Details
✅ CI: Optional E2E Tests / E2E tanstackstart-react (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E react-router-7-framework-instrumentation Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-express-send-to-sentry-static Test (optional) Successful in 3m Details
✅ CI: Optional E2E Tests / E2E nuxt-5-cloudflare Test (optional) Successful in 33s Details
✅ CI: Optional E2E Tests / E2E vue-3 (no Options API) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nestjs-12 (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-express-mcp-v2 Test (optional) Successful in 50s Details
✅ CI: Optional E2E Tests / E2E node-openai-decisions (latest) Test (optional) Successful in 35s Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps (client source maps kept) Test (optional) Successful in 43s Details
✅ CI: Optional E2E Tests / E2E nextjs-16-tunnel (cloudflare) Test (optional) Successful in 56s Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps (static / nuxt generate) Test (optional) Successful in 31s Details

Comment thread .github/workflows/optional-e2e.yml Resolved
Comment thread .github/workflows/build.yml Resolved
nicohrubec and others added 5 commits November 12, 2026 21:08
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread .github/workflows/build.yml Resolved
Co-Authored-By: OpenAI Codex <codex@openai.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 2882be8. Configure here.

Comment thread .github/workflows/build.yml Outdated
env:
HEAD_COMMIT: ${{ github.event.inputs.commit || github.sha }}
CI_BENCHMARK_PROVIDER: namespace
CI_BENCHMARK_CACHE_PREFIX: bench-sentry-js-20261112-namespace-b-

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Playwright install ignores volume cache hits

Medium Severity

Browser install still keys off steps.playwright-cache.outputs.cache-hit, but that restore step is skipped when CI_BENCHMARK_PROVIDER is namespace. A skipped step’s output is empty, so != 'true' is always true and every job runs npx -y playwright install. Warm Namespace runs therefore still fetch and invoke Playwright even when ~/.cache/ms-playwright is already on the volume, which distorts the warm-1 benchmark this commit is meant to measure.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 2882be8. Configure here.

nicohrubec and others added 5 commits November 12, 2026 22:04
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
@nicohrubec nicohrubec closed this Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants