perf(e2e): consolidate Hermes dashboard coverage

[apurvvkumaria]

Summary

Consolidates Hermes dashboard E2E assertions into the canonical hermes-e2e lane instead of rebuilding and onboarding the same Hermes fixture in a standalone job. Legacy hermes-dashboard selectors remain compatible, while scheduled sweeps avoid one complete duplicate Hermes execution and its BuildKit memory exposure.

Related Issue

Related #7140
Related #7144

Changes

• Always enable the existing dashboard assertions in hermes-e2e and remove the duplicate hermes-dashboard workflow job.
• Normalize legacy jobs=hermes-dashboard and targets=hermes-dashboard inputs to hermes-e2e before workflow planning and validation.
• Keep trusted swap eligibility, larger-runner routing, PR result reporting, and manual inference modes working through the canonical lane.
• Update workflow-boundary contracts and operational E2E documentation for the consolidated 13-identity / 16-execution telemetry inventory.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Quality Gates

• Tests added or updated for changed behavior
• Existing tests cover changed behavior — justification:
• Tests not applicable — justification:
• Docs updated for user-facing behavior changes
• Docs not applicable — justification:
• Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
• Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Exact-final-head product and nine-category security review at d84de9116 passed with no findings. The refreshed tree has the same stable aggregate patch identity (e50d9b04152d6c0715b8d64498018e540e99b771) as the previously reviewed patch; selector validation, trusted pre-checkout swap, and inference-secret boundaries remain intact.
• Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

• Documentation writer subagent reviewed the completed changes
• Result: docs-updated
• Evidence: test/e2e/README.md documents both legacy selector aliases, canonical hermes-e2e reporting/dashboard coverage, and the routing and telemetry inventory; inherited maintainer guidance remains correct for exact release-qualification evidence.
• Agent: Codex Desktop

DGX Station Hardware Evidence

• Tested on DGX Station
• Tested commit:
• Station profile/scenario:
• Result:
• Supporting evidence:

Verification

• PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
• Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
• Targeted behavior tests pass for the current change set, or tests are marked not applicable above — exact refreshed tree b3ccff5a051aabaf7d12548546e3ccc37b0bd44e: 187 tests passed across the eight changed E2E support suites; hosted-runner recovery passed 6 tests; semantic E2E phase coverage passed for 125 tests across 82 files; build:cli and typecheck:cli passed.
• Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: the original broad checks passed; exact refresh d84de9116 additionally passed signed hooks, npm run check:diff, and stable aggregate patch-identity verification.
• Quality Gates section completed with required justifications or waivers
• No secrets, API keys, or credentials committed
• npm run docs builds without warnings (doc changes only) — build passed with 0 errors; two pre-existing warnings are outside the changed E2E README.
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

---

Signed-off-by: Apurv Kumaria akumaria@nvidia.com

Summary by CodeRabbit

• New Features
  • Added compatibility aliasing so the retired hermes-dashboard selector maps to the canonical hermes-e2e lane across planning, routing, and reporting.
• Bug Fixes
  • Removed the standalone hermes-dashboard execution path; Hermes dashboard coverage, trusted swap behavior, telemetry, and PR comment reporting now consolidate on hermes-e2e.
  • Normalized jobs/targets selector inputs for consistent request/report contents.
• Tests
  • Updated and expanded E2E coverage to validate legacy selector handling and revised routing/telemetry counts.
• Documentation
  • Refreshed E2E lane naming and routing semantics to match the canonical hermes-e2e behavior.

[coderabbitai]

📝 Walkthrough

Walkthrough

The E2E workflow consolidates Hermes dashboard coverage into hermes-e2e, preserves hermes-dashboard as a selector alias, normalizes selectors across planning and reporting, and updates routing, trusted-swap validation, telemetry, documentation, and boundary tests.

Changes

Hermes E2E lane consolidation

Layer / File(s)	Summary
Selector aliasing and propagation tools/e2e/selector-aliases.mts, tools/e2e/workflow-plan.mts, tools/e2e/workflow-boundary.mts, tools/e2e/report-e2e-results.mts, test/e2e/support/*	Legacy hermes-dashboard selectors normalize to hermes-e2e during planning, dispatch evaluation, report rendering, and boundary tests.
Canonical Hermes workflow and boundary validation .github/workflows/e2e.yaml, tools/e2e/hermes-dashboard-workflow-boundary.mts, test/e2e/support/hermes-dashboard-workflow-boundary.test.ts, test/e2e/support/e2e-progress-fixture.test.ts	Dashboard coverage is enabled on hermes-e2e, the standalone dashboard job is removed, and canonical environment, live-test, checkout, and reporting dependency rules are validated.
Routing, trusted swap, and telemetry updates tools/e2e/trusted-hermes-swap-workflow-boundary.mts, tools/e2e/runner-comparison-workflow-boundary.mts, .github/workflows/e2e.yaml, test/e2e/support/*, test/e2e/README.md	Runner routing, trusted Hermes selection, comparison telemetry counts, boundary expectations, and documentation no longer treat hermes-dashboard as a standalone execution lane.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant workflow_dispatch
  participant workflow_plan
  participant workflow_boundary
  participant report_e2e_results
  workflow_dispatch->>workflow_plan: submit jobs and targets
  workflow_plan->>workflow_boundary: normalize legacy selectors
  workflow_boundary->>report_e2e_results: provide canonical selections
  report_e2e_results->>workflow_dispatch: render hermes-e2e report

Loading

Possibly related PRs

• NVIDIA/NemoClaw#7464: Modifies trusted Hermes swap eligibility and related boundary tests.
• NVIDIA/NemoClaw#7515: Modifies runner-comparison jobs and telemetry expectations.
• NVIDIA/NemoClaw#7534: Modifies Hermes timeout contract and workflow validation plumbing.

Suggested labels: chore

Suggested reviewers: cv, cjagwani, prekshivyas

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: consolidating Hermes dashboard E2E coverage into the canonical lane.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/7144-consolidate-hermes-dashboard

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-code-quality]

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit d84de91 in the codex/7144-consolida... branch remains at 96%, unchanged from commit 1b796e6 in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit d84de91 in the codex/7144-consolida... branch is 80%. The coverage in commit 1b796e6 in the main branch is 81%.

Show a code coverage summary of the most impacted files.

File	main 1b796e6	codex/7144-consolida... d84de91	+/-
src/lib/actions...-add-restart.ts	19%	10%	-9%
src/lib/actions...on-readiness.ts	100%	91%	-9%
src/lib/actions...x/mcp-bridge.ts	43%	36%	-7%
src/lib/actions...lution-probe.ts	93%	88%	-5%
src/lib/platform.ts	89%	84%	-5%
src/lib/state/o...oint-migrate.ts	96%	92%	-4%
src/lib/actions...e-validation.ts	84%	81%	-3%
src/lib/onboard...shboard-port.ts	93%	90%	-3%
src/lib/actions...dbox/destroy.ts	95%	93%	-2%
src/lib/onboard...eway-service.ts	82%	81%	-1%

---

_{Updated July 26, 2026 20:22 UTC}

[github-actions]

PR Review Advisor — Informational

Advisor assessment: Informational / low confidence
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions
Status: PR review advisor failed: PR review advisor SDK execution failed: session: correctness-state-analysis omitted required analysis; turn: correctness-state-analysis: correctness-state-analysis omitted required analysis

Model lanes

• GPT-5.6 Terra (primary): Failed
• Nemotron 3 Ultra (second opinion): Failed after a partial review · low confidence · 0 blockers · 4 warnings · 6 suggestions

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-onboard, credential-sanitization, security-posture

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

[coderabbitai]

🧹 Nitpick comments (1)

tools/e2e/workflow-boundary.mts (1)

613-662: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Minor: redundant normalization of jobs.

normalizeE2eSelectorIds(splitSelector(jobs)) is computed once for validation (line 631) and again for selectedFreeStandingJobs (line 660). Hoisting it into a local const normalizedJobs = ... and reusing it would avoid the duplicate work, though the input size makes this negligible in practice.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/e2e/workflow-boundary.mts` around lines 613 - 662, Hoist the normalized
job selector computation into a local normalizedJobs constant before validation
in evaluateE2eWorkflowDispatchSelectors, then reuse it for both unknown-job
validation and selectedFreeStandingJobs construction. Preserve the existing
behavior for empty or invalid jobs input.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tools/e2e/workflow-boundary.mts`:
- Around line 613-662: Hoist the normalized job selector computation into a
local normalizedJobs constant before validation in
evaluateE2eWorkflowDispatchSelectors, then reuse it for both unknown-job
validation and selectedFreeStandingJobs construction. Preserve the existing
behavior for empty or invalid jobs input.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d67ec293-ec24-427d-99d1-88122cadb9b8

📥 Commits

Reviewing files that changed from the base of the PR and between d4a859a and 3562db5.

📒 Files selected for processing (18)

• .github/workflows/e2e.yaml
• test/e2e/README.md
• test/e2e/support/e2e-progress-fixture.test.ts
• test/e2e/support/e2e-report-to-pr-workflow-boundary.test.ts
• test/e2e/support/e2e-workflow.test.ts
• test/e2e/support/hermes-dashboard-workflow-boundary.test.ts
• test/e2e/support/larger-runner-routing-workflow-boundary.test.ts
• test/e2e/support/runner-comparison-workflow-boundary.test.ts
• test/e2e/support/trusted-hermes-swap-workflow-boundary.test.ts
• test/e2e/support/workflow-plan.test.ts
• tools/e2e/hermes-dashboard-workflow-boundary.mts
• tools/e2e/hermes-timeout-contract.mts
• tools/e2e/report-e2e-results.mts
• tools/e2e/runner-comparison-workflow-boundary.mts
• tools/e2e/selector-aliases.mts
• tools/e2e/trusted-hermes-swap-workflow-boundary.mts
• tools/e2e/workflow-boundary.mts
• tools/e2e/workflow-plan.mts

💤 Files with no reviewable changes (2)

• tools/e2e/hermes-timeout-contract.mts
• test/e2e/support/larger-runner-routing-workflow-boundary.test.ts

[apurvvkumaria]

Current-head status for 36872f422:

• All ordinary CI and automated review checks passed.
• The credentialed E2E child did not start.
• Controller run 30213114264 failed closed because GitHub released the approval job without recording a required reviewer.
• The repository environment currently has protection_rules: [], deployment_branch_policy: null, and administrator bypass enabled.
• No E2E child was created and no repository secret was exposed.

This is the existing repository-configuration blocker tracked in #7451, not a #7592 code or runner failure. I added the exact run/API evidence there.

Safe next action: configure the protected environment as specified in #7451, then use the documented manual exact-SHA run-control-plane fallback for this current head, or create a fresh PR SHA after configuration. Rerunning this unprotected approval job cannot supply the missing reviewer record, and the controller correctly refuses it.

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

tools/e2e/workflow-boundary.mts (1)

667-672: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Add the mapped job ID, not the target ID.

freeStandingTargetToJob.get(target) returns the free-standing job identifier, but this branch adds target to selectedFreeStandingJobs. When a target ID differs from its job ID, the requested free-standing job is omitted or misidentified downstream.

Proposed fix

-    if (job) selectedFreeStandingJobs.add(target);
+    if (job) selectedFreeStandingJobs.add(job);

Add a test covering a target-to-job mapping with different IDs.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/e2e/workflow-boundary.mts` around lines 667 - 672, Update the
free-standing target handling around freeStandingTargetToJob so the mapped job
identifier returned by get(target) is added to selectedFreeStandingJobs instead
of the target identifier; preserve registryTargets behavior for unmapped targets
and add coverage for differing target and job IDs.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@tools/e2e/workflow-boundary.mts`:
- Around line 667-672: Update the free-standing target handling around
freeStandingTargetToJob so the mapped job identifier returned by get(target) is
added to selectedFreeStandingJobs instead of the target identifier; preserve
registryTargets behavior for unmapped targets and add coverage for differing
target and job IDs.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f22cee3e-d91d-4134-be81-549c9c269f54

📥 Commits

Reviewing files that changed from the base of the PR and between 36872f4 and d84de91.

📒 Files selected for processing (4)

• .github/workflows/e2e.yaml
• test/e2e/README.md
• test/e2e/support/e2e-workflow.test.ts
• tools/e2e/workflow-boundary.mts

🚧 Files skipped from review as they are similar to previous changes (1)

• test/e2e/support/e2e-workflow.test.ts