ci(e2e): add runner comparison telemetry

[cjagwani]

Summary

This PR stacks on #7388 and leaves that PR unchanged. It adds diagnostic-only E2E telemetry so maintainers can distinguish queue delay from execution time, compare heavy-lane runner pressure, and see current semantic-test timing against a bounded nightly history without adding normal terminal-log noise or changing runner routing.

Related Issue

Related to #7145. Follow-up to #7388.

Changes

• Record lightweight CPU, memory, and disk samples at semantic phase boundaries for the five larger-runner candidates, then upload one numeric job-level summary with existing E2E artifacts.
• Add a scorecard table that separates GitHub Actions queue time from execution time and reports only a normalized runner class, never the runner name.
• Publish one bounded e2e-runtime-summary.json artifact and compare the current run with up to ten prior completed scheduled runs; manual runs and historical raw artifacts are excluded.
• Share one exact-root artifact ZIP reader between trace timing and runtime history. A direct duplicate parser would create two security-sensitive archive boundaries; artifact-zip.test.ts protects the shared size, duplicate-entry, and exact-name contract.
• Keep measurement, history lookup, and summary publication advisory so telemetry failures cannot change a live test result.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Quality Gates

• Tests added or updated for changed behavior
• Existing tests cover changed behavior — justification:
• Tests not applicable — justification:
• Docs updated for user-facing behavior changes
• Docs not applicable — justification: This changes maintainer-only E2E diagnostics; the internal contract is documented in test/e2e/README.md.
• Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
• Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Self-reviewed workflow permissions, immutable artifact provenance, schema and size bounds, raw-log exclusion, and best-effort failure isolation; workflow boundary tests cover those contracts.
• Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

DGX Station Hardware Evidence

• Tested on DGX Station
• Tested commit: Not applicable
• Station profile/scenario: Not applicable
• Result: Not applicable
• Supporting evidence: Not applicable

Verification

• PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
• Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
• Targeted behavior tests pass for the current change set, or tests are marked not applicable above — npx vitest run --project e2e-support with the 12 changed-boundary suites: 199 tests passed
• Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — build stages passed, but the local all-project run was stopped after widespread unrelated timeout failures under concurrent load; CI is the broad gate
• Quality Gates section completed with required justifications or waivers
• No secrets, API keys, or credentials committed
• npm run docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

---

Signed-off-by: Charan Jagwani cjagwani@nvidia.com

[github-code-quality]

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 4b47854 in the codex/e2e-telemetry-... branch remains at 96%, unchanged from commit 1bb15a0 in the codex/publish-e2e-ru... branch.

---

_{Updated July 22, 2026 22:31 UTC}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4c3a8631-396a-4c21-b98d-7c7da3261eea

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/e2e-telemetry-followups

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

PR Review Advisor — Informational

Advisor assessment: Informational / high confidence
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions
Status: No actionable findings remain in the canonical review ledger.

Model lanes

• GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
• Nemotron 3 Ultra (second opinion): Failed after a partial review · low confidence · 0 blockers · 3 warnings · 0 suggestions

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-onboard, credential-sanitization, security-posture

4 optional E2E recommendations

• common-egress-agent
• rebuild-hermes
• rebuild-hermes-stale-base
• mcp-bridge

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

[prekshivyas]

Reviewed the incremental stacked diff at exact head a21a6cc against #7388 head 25a1775. Security PASS across all nine review categories. Telemetry is numeric/allowlisted and uses bounded private files; process names, argv, runner names, environment values, and secrets are excluded. The shared ZIP reader validates exact root names, duplicates, file type, encryption, entry/byte bounds, CRC, and never extracts to disk. Historical reads are limited to completed scheduled runs with read-only Actions access. Focused validation: 152 tests passed. No actionable findings. Parent #7388 and exact-SHA credentialed E2E remain required.

[cv]

Addressed PRA-1 in signed commit 877e360: the artifact ZIP fixture now emits real method-8 deflated entries, verifies valid extraction, and verifies corrupt compressed data is rejected. The reader now catches decompression errors and returns null under its existing fail-closed contract. Fresh GitHub checks are running on the new head.

[apurvvkumaria]

Closing this implementation as superseded by #7399 for the core #7145 runner-comparison telemetry.

I compared both current heads against #7145 and the merged #7100 work. #7399 is the safer canonical collector: it is based on main, has a focused five-execution scope, enforces trusted-main/alternate-checkout boundaries, uses bounded canonical JSONL with duplicate-key rejection, and has private-file plus symlink/hardlink defenses. Its ordinary checks are green and its review findings are resolved.

This PR is currently stacked on #7388 rather than main, has failing static/CLI checks, and duplicates phase/resource collection already covered by #7100 and #7399. Merging or rebasing it wholesale would leave two evidence formats and two overlapping collectors.

The unique ideas remain useful and should be preserved as narrow follow-ups under #7145 after the core collector lands:

• queue time, execution time, and normalized runner class, with direct tests;
• bounded scheduled semantic-runtime/pass-rate history;
• a shared bounded ZIP artifact reader.

Those follow-ups should consume the canonical #7399 artifact rather than retain this second collector. The branch and commits remain recoverable if targeted logic needs to be adapted later.