Skip to content

perf(e2e): focus retained coverage - #7931

Merged
prekshivyas merged 22 commits into
mainfrom
codex/e2e-optimize-7917-7920-7921-post-7665
Jul 30, 2026
Merged

perf(e2e): focus retained coverage#7931
prekshivyas merged 22 commits into
mainfrom
codex/e2e-optimize-7917-7920-7921-post-7665

Conversation

@prekshivyas

@prekshivyas prekshivyas commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

Reduce retained E2E cost without changing the workflow schedule or lane structure, and finalize the rebuild-selector retirement staged by #7665. This PR deletes the sandbox-rebuild and upgrade-stale-sandbox live jobs and tests after the compatibility controller and focused replacement coverage land in #7665.

This PR builds on #7665, merged as 818a62f2b.

The trusted E2E workflow keeps its existing daily 0 0 * * * schedule and its single five-row openshell-gateway-upgrade job unchanged.

Related Issue

Fixes #7615
Fixes #7917
Fixes #7921

Changes

  • Delete the sandbox-rebuild and upgrade-stale-sandbox workflow jobs, live tests, helper, mock-parity entries, and job-specific workflow validators. Keep their job and target selectors routed through the compatibility controller to focused rebuild, route-preflight, batch-upgrade, and integration replacements. rebuild-openclaw remains the canonical live rebuild and upgrade target.
  • Stop openclaw-plugin-runtime-exdev after the v2 recreate and cross-device probe instead of repeating the same state-replacement boundary with a v3 rebuild. The lifecycle command helper lets the e2e-support test protect the retained onboard, restart, and recreate sequence without a live runner; openclaw-plugin-runtime-exdev-fixture.test.ts verifies that command map on the current source-aware fixture boundary.
  • Map workflow-wired live tests and their owning helpers to canonical focused jobs, leave test/e2e/support/ changes in PR CI, and preserve the broad cloud floor for unknown, renamed, security, workflow, or shared-tool paths. The risk-plan and workflow-inventory tests cover additions, renames, support-only changes, owning helpers, and cross-cutting changes.
  • Let trusted controller selections containing only retired job IDs emit an empty live plan so the compatibility job can verify their replacements; per-selector tests protect both retired job paths.
  • Reject step-scoped NEMOCLAW_E2E_USE_HOSTED_INFERENCE overrides in the inference-switch workflow boundary, with focused regression coverage.
  • Preserve the existing daily workflow schedule and the existing single openshell-gateway-upgrade lane with all five migration rows.
  • Document retired-selector compatibility, the retained EXDEV lifecycle, and changed-area selection in test/e2e/README.md.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: No supported product behavior changes. The internal E2E README documents the retired selectors and retained topology.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: A maintainer approved the stacked E2E optimization and test(e2e): consolidate priority 2 live seams #7665 transition. The current merge preserves the reviewed compatibility boundaries, and their focused security/workflow tests pass.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: No waiver requested.

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: docs-updated
  • Evidence: test/e2e/README.md; the review verified retired-selector compatibility, rebuild-openclaw as the canonical target, the retained EXDEV lifecycle, focused PR-gate selection, and explanatory text across the full diff. No user-facing Fern or AI-route documentation is required.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit: Not applicable.
  • Station profile/scenario: Not applicable.
  • Result: Not applicable.
  • Supporting evidence: Not applicable.

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: The post-test(e2e): consolidate priority 2 live seams #7665 resolved state passed 59 focused CLI tests, 99 focused E2E-support tests, and 9 mock-parity integration tests. The inherited Launchable integration passed 5/5 with a 30-second local macOS timeout. Semantic E2E phase coverage found 114 tests across 71 files, exact Vitest membership found 1,938 files across seven projects, repository checks passed, and merge hooks passed.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: Earlier full E2E-support diagnostics on the stacked changes passed more than 1,800 tests, with known macOS timing/systemd fixture failures under high parallelism. The directly affected combined subset passes 92/92.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Prekshi Vyas prekshiv@nvidia.com

Summary by CodeRabbit

  • New Features
    • Added/updated current OpenClaw plugin runtime lifecycle coverage in E2E (onboarding, restart, and sandbox recreation).
    • Expanded retired-selector compatibility to support target-based selection and broader replacement coverage.
  • Bug Fixes
    • Updated deterministic risk-plan and PR E2E recommendations to use rebuild-openclaw as the coverage floor.
    • Tightened “canonical” Anthropic-compatible inference/mode validation and reduced hosted coverage.
  • Refactor
    • Removed obsolete E2E live lanes/tests for sandbox rebuild and stale-sandbox upgrade.
  • Documentation
    • Refreshed E2E operational docs for retired selector compatibility and updated runner routing/telemetry expectations.
  • Tests
    • Updated and added E2E and workflow boundary tests to match revised gates and expected job/shard mappings.

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
@prekshivyas prekshivyas self-assigned this Jul 30, 2026
@copy-pr-bot

copy-pr-bot Bot commented Jul 30, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR retires overlapping E2E lanes, narrows inference and Bedrock matrices, adds target-aware retired-selector compatibility, refines risk-plan ownership, and shortens the OpenClaw EXDEV lifecycle to onboarding, restart, recreation, and dependency replacement.

Changes

E2E workflow consolidation

Layer / File(s) Summary
Workflow inventory and risk mapping
.github/workflows/e2e.yaml, tools/advisors/*, tools/e2e/workflow-boundary.mts, test/e2e/README.md, test/pr-risk-plan.test.ts, test/e2e-recommendations.test.ts
Retired lanes, ownership mappings, risk-plan rules, schedules, reporting coverage, documentation, and deterministic expectations are updated.
Selector compatibility and empty plans
tools/e2e/retired-selector-compatibility.mts, tools/e2e/workflow-plan.mts, test/e2e/support/*
Retired job and target selectors are filtered, replacement coverage accepts target selections, workflow drift is tested, and retired-only selections emit empty plans.
Inference and runtime matrix contracts
.github/workflows/e2e.yaml, tools/e2e/*workflow-boundary.mts, test/e2e/support/*, test/pr-e2e-gate*
Hosted inference rows and credentials are removed, Bedrock-compatible coverage keeps OpenClaw, and reduced matrix contracts are enforced.
OpenClaw EXDEV lifecycle
test/e2e/live/openclaw-plugin-runtime-exdev*, test/e2e/support/openclaw-plugin-runtime-exdev-fixture.test.ts, test/e2e/README.md
Lifecycle commands and phases are centralized; the final v3 rebuild phase is removed while restart, recreation, EXDEV replacement, and state checks remain.
Rebuild recovery contracts
src/lib/actions/sandbox/rebuild-route-preflight.test.ts, src/lib/actions/upgrade-sandboxes-recovery.test.ts
Focused tests cover sequential credential migration and automatic stale-sandbox rebuild invocation.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Controller
  participant WorkflowPlan
  participant E2EWorkflow
  participant LiveE2E
  Controller->>WorkflowPlan: provide jobs and targets
  WorkflowPlan->>E2EWorkflow: filter retired selectors and build plan
  E2EWorkflow->>LiveE2E: run retained matrix lanes
  LiveE2E->>LiveE2E: onboard, restart, recreate, and verify EXDEV replacement
Loading

Possibly related issues

  • #7917 — Removes the duplicate rebuild phase from the retained OpenClaw EXDEV lifecycle.
  • #7921 — Maps changed files to the smallest retained live E2E set.
  • #7912 — Covers the broader E2E retirement and inference-matrix consolidation.

Possibly related PRs

Suggested labels: area: e2e, area: ci, chore, area: performance

Suggested reviewers: cv

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 2.94% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR matches the linked goals by retiring duplicate live seams, keeping rebuild-openclaw canonical, shortening EXDEV lifecycle, and tightening mapping tests.
Out of Scope Changes check ✅ Passed No clearly unrelated changes stand out; the workflow, tests, helpers, and docs all support the retained E2E consolidation goals.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title concisely and accurately describes the PR’s main change: focusing retained E2E coverage for improved performance.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/e2e-optimize-7917-7920-7921-post-7665

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 3 blockers · 5 warnings · 1 suggestion
  • Model comparison: normalized findings differ; normalized E2E selections differ; Nemotron reported 3 more blockers, 5 more warnings, 1 more suggestion.

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-inference, cloud-onboard, security-posture, inference-routing, network-policy, openclaw-plugin-runtime-exdev, openclaw-plugin-runtime-exdev-release, sandbox-rebuild, upgrade-stale-sandbox

1 optional E2E recommendation
  • rebuild-openclaw

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
@github-code-quality

github-code-quality Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit c579d39 in the codex/e2e-optimize-7... branch remains at 96%, unchanged from commit 818a62f in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit c579d39 in the codex/e2e-optimize-7... branch remains at 81%, unchanged from commit 818a62f in the main branch.

Show a code coverage summary of the most impacted files.
File main 818a62f codex/e2e-optimize-7... c579d39 +/-
src/lib/onboard...flight-ports.ts 67% 33% -34%
src/lib/onboard...box-prebuild.ts 92% 74% -18%
src/lib/actions...ocker-health.ts 82% 65% -17%
src/lib/actions...confirmation.ts 79% 69% -10%
src/lib/domain/.../connect-env.ts 97% 89% -8%
src/lib/onboard...box-gpu-mode.ts 97% 92% -5%
src/lib/actions...orchestrator.ts 87% 83% -4%
src/lib/onboard/preflight.ts 80% 80% 0%
src/lib/onboard...host-anchors.ts 90% 94% +4%
src/lib/onboard/docker-cdi.ts 70% 80% +10%

Updated July 30, 2026 23:17 UTC

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
@prekshivyas
prekshivyas marked this pull request as ready for review July 30, 2026 19:21
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (4)
tools/advisors/risk-plan.mts (1)

393-397: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Rule-id special case embedded in the generic matcher.

Hard-coding rule.id === "e2e-control-plane" inside the shared flatMap couples the generic family loop to one rule. A declarative field on the rule (e.g. yieldsToFocusedLive: true) keeps the loop rule-agnostic and makes the narrowing visible next to the rule definition.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/advisors/risk-plan.mts` around lines 393 - 397, Replace the hard-coded
"e2e-control-plane" check in the staticFamilies matcher with a declarative rule
property such as yieldsToFocusedLive, define that property on the corresponding
RISK_RULES entry, and use it to exclude focused live files while keeping the
generic flatMap rule-agnostic.
test/e2e/support/workflow-plan.test.ts (1)

227-263: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Near-duplicate of the retired-job case above.

The two blocks differ only in JOBS/TARGETS. Folding them into one it.each over [{ jobs, targets }] pairs (or extracting an expectEmptyLivePlan({ jobs, targets }) helper) removes ~35 duplicated lines and keeps the expected output schema in one place.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/support/workflow-plan.test.ts` around lines 227 - 263, Consolidate
the empty live-plan tests, including the current P2 selector case and the
retired-job case above, by parameterizing them with it.each over their
JOBS/TARGETS pairs or by extracting an expectEmptyLivePlan({ jobs, targets })
helper. Keep the shared planner invocation and expected output/schema assertions
in one place while preserving each case’s distinct environment values.
tools/e2e/workflow-boundary.mts (1)

580-603: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

LIVE_E2E_OWNING_FILE_JOBS is a second hand-maintained path→job mirror.

The map duplicates ownership knowledge that already lives in the free-standing inventory (inventory.liveTestToJobs) and will silently drift when a helper is renamed or a lane retires — the inventory.allowedJobs.includes(job) guard makes drift silent rather than loud. Consider deriving these helper→job edges from the inventory (or asserting the map's keys still exist on disk and its jobs are all allowed) so a rename fails a test instead of quietly dropping coverage.

As per path instructions, "Derive inventories and limits from a canonical source where possible; flag duplicated lists that can silently drift."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/e2e/workflow-boundary.mts` around lines 580 - 603, The
LIVE_E2E_OWNING_FILE_JOBS map duplicates ownership data and can silently lose
coverage when helpers or jobs change. Update focusedE2eJobsForChangedFiles and
the surrounding inventory setup to derive these helper-to-job relationships from
the canonical inventory.liveTestToJobs data where possible; otherwise validate
every map key exists and every mapped job is in inventory.allowedJobs, failing
loudly instead of silently skipping stale entries.

Source: Path instructions

tools/e2e/openshell-gateway-upgrade-workflow-boundary.mts (1)

21-24: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Hardcoded cron literal duplicates WEEKLY_CRON.

COMPATIBILITY_CONDITION hardcodes '0 0 * * 0' instead of interpolating the WEEKLY_CRON constant defined two lines above. If the weekly cron is ever changed, this condition string (and the real workflow's if: expression it must match byte-for-byte) will silently drift out of sync with the schedule check.

♻️ Proposed fix
 const COMPATIBILITY_CONDITION =
-  "${{ (github.event_name == 'schedule' && github.event.schedule == '0 0 * * 0') || contains(format(',{0},', inputs.jobs), ',openshell-gateway-upgrade-compatibility,') || contains(format(',{0},', inputs.targets), ',openshell-gateway-upgrade-compatibility,') }}";
+  `\${{ (github.event_name == 'schedule' && github.event.schedule == '${WEEKLY_CRON}') || contains(format(',{0},', inputs.jobs), ',openshell-gateway-upgrade-compatibility,') || contains(format(',{0},', inputs.targets), ',openshell-gateway-upgrade-compatibility,') }}`;
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/e2e/openshell-gateway-upgrade-workflow-boundary.mts` around lines 21 -
24, Update COMPATIBILITY_CONDITION to interpolate the existing WEEKLY_CRON
constant instead of hardcoding "0 0 * * 0", preserving the generated workflow’s
schedule comparison and keeping it synchronized with the defined weekly cron
value.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/e2e.yaml:
- Around line 4333-4353: Update the compatibility matrix rows used by the
openshell-gateway-upgrade-compatibility job to define current_openclaw_version
and openclaw_state_upgrade for every weekly entry, matching the existing matrix
naming and value conventions. Ensure the shared env references
NEMOCLAW_CURRENT_OPENCLAW_VERSION and NEMOCLAW_OPENCLAW_STATE_UPGRADE_PROOF
resolve to populated values for each row, without changing unrelated jobs.

In `@tools/e2e/inference-switch-workflow-boundary.mts`:
- Around line 102-112: Update the validation loop over job.steps in the
inference-switch boundary check to reject NEMOCLAW_E2E_USE_HOSTED_INFERENCE
whenever it is defined in step.env, alongside the existing NVIDIA key checks.
Add a regression case proving a step-scoped hosted-inference override is
reported as invalid, while preserving the current job-level validation.

---

Nitpick comments:
In `@test/e2e/support/workflow-plan.test.ts`:
- Around line 227-263: Consolidate the empty live-plan tests, including the
current P2 selector case and the retired-job case above, by parameterizing them
with it.each over their JOBS/TARGETS pairs or by extracting an
expectEmptyLivePlan({ jobs, targets }) helper. Keep the shared planner
invocation and expected output/schema assertions in one place while preserving
each case’s distinct environment values.

In `@tools/advisors/risk-plan.mts`:
- Around line 393-397: Replace the hard-coded "e2e-control-plane" check in the
staticFamilies matcher with a declarative rule property such as
yieldsToFocusedLive, define that property on the corresponding RISK_RULES entry,
and use it to exclude focused live files while keeping the generic flatMap
rule-agnostic.

In `@tools/e2e/openshell-gateway-upgrade-workflow-boundary.mts`:
- Around line 21-24: Update COMPATIBILITY_CONDITION to interpolate the existing
WEEKLY_CRON constant instead of hardcoding "0 0 * * 0", preserving the generated
workflow’s schedule comparison and keeping it synchronized with the defined
weekly cron value.

In `@tools/e2e/workflow-boundary.mts`:
- Around line 580-603: The LIVE_E2E_OWNING_FILE_JOBS map duplicates ownership
data and can silently lose coverage when helpers or jobs change. Update
focusedE2eJobsForChangedFiles and the surrounding inventory setup to derive
these helper-to-job relationships from the canonical inventory.liveTestToJobs
data where possible; otherwise validate every map key exists and every mapped
job is in inventory.allowedJobs, failing loudly instead of silently skipping
stale entries.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4ed7fe0f-6e58-44a6-894a-1e1d4250e09f

📥 Commits

Reviewing files that changed from the base of the PR and between 1ce71fa and 5e00ea7.

📒 Files selected for processing (38)
  • .github/workflows/e2e.yaml
  • scripts/scorecard/summarize-jobs.mts
  • test/e2e-recommendations.test.ts
  • test/e2e/README.md
  • test/e2e/live/openclaw-plugin-runtime-exdev-lifecycle.ts
  • test/e2e/live/openclaw-plugin-runtime-exdev.test.ts
  • test/e2e/live/sandbox-rebuild.test.ts
  • test/e2e/live/upgrade-stale-sandbox-helpers.ts
  • test/e2e/live/upgrade-stale-sandbox.test.ts
  • test/e2e/mock-parity.json
  • test/e2e/support/e2e-report-to-pr-workflow-boundary.test.ts
  • test/e2e/support/e2e-scorecard.test.ts
  • test/e2e/support/e2e-workflow.test.ts
  • test/e2e/support/inference-switch-workflow-boundary.test.ts
  • test/e2e/support/openclaw-plugin-runtime-exdev-fixture.test.ts
  • test/e2e/support/openshell-gateway-upgrade-workflow-boundary.test.ts
  • test/e2e/support/retired-selector-compatibility-workflow-boundary.test.ts
  • test/e2e/support/retired-selector-compatibility.test.ts
  • test/e2e/support/runner-comparison-workflow-boundary.test.ts
  • test/e2e/support/trusted-hermes-swap-workflow-boundary.test.ts
  • test/e2e/support/workflow-plan.test.ts
  • test/pr-e2e-gate-shards.test.ts
  • test/pr-e2e-gate-signal-shards.test.ts
  • test/pr-e2e-gate.test.ts
  • test/pr-review-advisor-security-boundaries.test.ts
  • test/pr-review-advisor.test.ts
  • test/pr-risk-plan.test.ts
  • test/release-e2e-evidence.test.ts
  • tools/advisors/risk-plan.mts
  • tools/e2e/inference-switch-workflow-boundary.mts
  • tools/e2e/openshell-gateway-upgrade-workflow-boundary.mts
  • tools/e2e/report-e2e-results.mts
  • tools/e2e/retired-selector-compatibility.mts
  • tools/e2e/runner-comparison-workflow-boundary.mts
  • tools/e2e/trusted-hermes-swap-workflow-boundary.mts
  • tools/e2e/upload-e2e-artifacts-workflow-boundary.mts
  • tools/e2e/workflow-boundary.mts
  • tools/e2e/workflow-plan.mts
💤 Files with no reviewable changes (6)
  • tools/e2e/trusted-hermes-swap-workflow-boundary.mts
  • test/e2e/live/upgrade-stale-sandbox.test.ts
  • test/e2e/live/sandbox-rebuild.test.ts
  • test/e2e/live/upgrade-stale-sandbox-helpers.ts
  • test/e2e/support/trusted-hermes-swap-workflow-boundary.test.ts
  • test/e2e/mock-parity.json

Comment thread .github/workflows/e2e.yaml Outdated
Comment thread tools/e2e/inference-switch-workflow-boundary.mts

@apurvvkumaria apurvvkumaria left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved under a critical-only blocking threshold. I found no critical correctness or security issue in the retained-coverage changes.

Non-blocking fast-follow: matrix scorecard aggregation currently collapses sibling rows by job name and can retain the latest-completing result rather than the worst conclusion. Please track worst-conclusion/full-span aggregation with a mixed-result test; this should not block this PR.

Operationally, land #7665 first and then rebase/fast-forward this stacked PR. The failed Nemotron advisor lane is an analysis-infrastructure failure with no product-test assertion; the primary advisor reported merge_as_is.

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Comment thread .github/workflows/e2e.yaml Outdated
Comment on lines +9 to +10
- cron: "0 0 * * 1-6"
- cron: "0 0 * * 0"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the intent behind changing our sched?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intent was to reduce daily runner minutes by running the current gateway-migration row daily and the historical/architecture rows weekly. That scheduling change is not required for the retained-coverage work and would alter lane cadence, so I removed it. #7931 now keeps the existing daily schedule and the single five-row openshell-gateway-upgrade job unchanged; #7920 is no longer part of this PR.

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
tools/e2e/workflow-boundary.mts (1)

992-1003: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Require exactly one daily schedule entry.

Line 1001 accepts a workflow containing 0 0 * * * plus extra schedules, so this validator does not enforce its single-cron contract.

Proposed fix
-  if (!cronEntries.includes("0 0 * * *")) {
+  if (schedule.length !== 1 || cronEntries.length !== 1 || cronEntries[0] !== "0 0 * * *") {
     errors.push("workflow schedule must run daily at 00:00 UTC");
   }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/e2e/workflow-boundary.mts` around lines 992 - 1003, Update
requireScheduledRun so the workflow must contain exactly one valid cron entry,
and that entry must be "0 0 * * *". Reject schedules with additional cron
entries while preserving the existing error reporting for missing or invalid
schedules.
.github/workflows/e2e.yaml (1)

4213-4248: 🚀 Performance & Scalability | 🟠 Major | 🏗️ Heavy lift

Do not run historical gateway rows on every daily schedule.

For scheduled runs, Line 4206 is always true, so all four historical/architecture rows run daily alongside the current row. This defeats the required daily-current/weekly-or-direct historical tiering and retains the cost this PR intends to remove. Add schedule-aware matrix selection or split the current and weekly compatibility lanes. Based on PR objectives: historical and architecture-specific rows must run weekly or by direct selection.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/e2e.yaml around lines 4213 - 4248, Update the workflow’s
matrix selection around the schedule condition near line 4206 so scheduled daily
runs include only the current row. Route the historical v0.0.36/v0.0.55/v0.0.74
and architecture-specific rows to weekly schedules or explicit/direct matrix
selection, preserving their ability to run when directly requested.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In @.github/workflows/e2e.yaml:
- Around line 4213-4248: Update the workflow’s matrix selection around the
schedule condition near line 4206 so scheduled daily runs include only the
current row. Route the historical v0.0.36/v0.0.55/v0.0.74 and
architecture-specific rows to weekly schedules or explicit/direct matrix
selection, preserving their ability to run when directly requested.

In `@tools/e2e/workflow-boundary.mts`:
- Around line 992-1003: Update requireScheduledRun so the workflow must contain
exactly one valid cron entry, and that entry must be "0 0 * * *". Reject
schedules with additional cron entries while preserving the existing error
reporting for missing or invalid schedules.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f893d324-6c03-449a-8cb2-b97dbdff9cb8

📥 Commits

Reviewing files that changed from the base of the PR and between 5e00ea7 and 4f64091.

📒 Files selected for processing (6)
  • .github/workflows/e2e.yaml
  • test/e2e/README.md
  • test/e2e/support/e2e-workflow.test.ts
  • test/pr-e2e-gate-signal-shards.test.ts
  • test/pr-risk-plan.test.ts
  • tools/e2e/workflow-boundary.mts
💤 Files with no reviewable changes (1)
  • test/e2e/support/e2e-workflow.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/e2e/README.md

@cv
cv enabled auto-merge (squash) July 30, 2026 21:27
# Conflicts:
#	.github/workflows/e2e.yaml
#	test/e2e/live/upgrade-stale-sandbox-helpers.ts
#	test/e2e/live/upgrade-stale-sandbox.test.ts
#	test/e2e/support/workflow-plan.test.ts
#	tools/e2e/workflow-boundary.mts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/support/workflow-plan.test.ts`:
- Around line 32-35: Update retiredControllerSelectorIds() to validate that
filtering RETIRED_CONTROLLER_SELECTOR_IDS does not produce an empty list before
the parameterized test consumes it; assert that at least one retired selector
remains, or that the expected retired selectors are present, while preserving
the existing filtering behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8663bd88-3b77-4c75-a5ea-162addd0fb31

📥 Commits

Reviewing files that changed from the base of the PR and between d8cd881 and 2b81d2a.

📒 Files selected for processing (6)
  • src/lib/actions/sandbox/rebuild-route-preflight.test.ts
  • src/lib/actions/upgrade-sandboxes-recovery.test.ts
  • test/e2e/README.md
  • test/e2e/support/retired-selector-compatibility.test.ts
  • test/e2e/support/workflow-plan.test.ts
  • tools/e2e/retired-selector-compatibility.mts
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/e2e/README.md

Comment thread test/e2e/support/workflow-plan.test.ts
@prekshivyas
prekshivyas disabled auto-merge July 30, 2026 22:25
apurvvkumaria and others added 3 commits July 30, 2026 15:56
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
# Conflicts:
#	.github/workflows/e2e.yaml
#	test/e2e/README.md
#	test/e2e/support/retired-selector-compatibility-workflow-boundary.test.ts
#	test/e2e/support/workflow-plan.test.ts
#	tools/e2e/inference-switch-workflow-boundary.mts
#	tools/e2e/workflow-boundary.mts
#	tools/e2e/workflow-plan.mts
@prekshivyas
prekshivyas enabled auto-merge (squash) July 30, 2026 23:28
@prekshivyas
prekshivyas merged commit 4767f58 into main Jul 30, 2026
72 of 75 checks passed
@prekshivyas
prekshivyas deleted the codex/e2e-optimize-7917-7920-7921-post-7665 branch July 30, 2026 23:39
@sandl99 sandl99 mentioned this pull request Jul 31, 2026
23 tasks
sandl99 added a commit that referenced this pull request Jul 31, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Adds the canonical July 30 release entry for `v0.0.99` before the
release tag is captured.
The entry covers all 37 merged PRs since `v0.0.98` and bounds
experimental or dormant work without presenting it as supported
behavior.

## Changes

- Adds `docs/changelog/2026-07-30.mdx` with the exact `## v0.0.99`
heading, parser-safe MDX SPDX comment, summary, detailed release
bullets, and published documentation routes.
- Records user-visible recovery, snapshot, shared-route, Hermes,
readiness, inference, image, documentation, and release E2E changes.
- States that the managed-image selection and startup-profile contracts
remain dormant and do not activate buildless onboarding.

Source summary:

- [#7972](#7972) ->
`docs/changelog/2026-07-30.mdx`: Records restored managed OpenClaw
configuration modes during recovery.
- [#7834](#7834) ->
`docs/changelog/2026-07-30.mdx`: Records clone-bound pairing
verification after snapshot restore.
- [#7975](#7975) ->
`docs/changelog/2026-07-30.mdx`: Records managed startup recovery
coverage.
- [#7960](#7960) ->
`docs/changelog/2026-07-30.mdx`: Records dormant startup-profile
coordination without activating a supported surface.
- [#7856](#7856) ->
`docs/changelog/2026-07-30.mdx`: Records persistence of the
credential-free OpenClaw startup command.
- [#7959](#7959) ->
`docs/changelog/2026-07-30.mdx`: Records dormant startup-profile
construction without changing onboarding.
- [#7946](#7946) ->
`docs/changelog/2026-07-30.mdx`: Records the internal startup-profile
schema and transport contract.
- [#7951](#7951) ->
`docs/changelog/2026-07-30.mdx`: Records platform-pull cleanup before
managed-image validation.
- [#7949](#7949) ->
`docs/changelog/2026-07-30.mdx`: Records rejection of retained Hermes
`uv` build cache metadata.
- [#7597](#7597) ->
`docs/changelog/2026-07-30.mdx`: Records separate command and agent
first-turn latency evidence.
- [#7931](#7931) ->
`docs/changelog/2026-07-30.mdx`: Records focused E2E replacement
evidence for retired selectors.
- [#7950](#7950) ->
`docs/changelog/2026-07-30.mdx`: Records exclusion of build-only
BuildKit telemetry from the Deep Agents Code probe.
- [#7665](#7665) ->
`docs/changelog/2026-07-30.mdx`: Records consolidated priority 2 E2E
coverage.
- [#7911](#7911) ->
`docs/changelog/2026-07-30.mdx`: Records the corrected NVIDIA DORI
installation pin.
- [#7934](#7934) ->
`docs/changelog/2026-07-30.mdx`: Records the staging image-family wait
before Brev Launchable deployment.
- [#7772](#7772) ->
`docs/changelog/2026-07-30.mdx`: Records dormant managed-image selection
contracts without activating buildless onboarding.
- [#7941](#7941) ->
`docs/changelog/2026-07-30.mdx`: Records corrected agent-specific
provider and policy guidance.
- [#7819](#7819) ->
`docs/changelog/2026-07-30.mdx`: Records removal of empty Deep Agents
Code provider-switch sections.
- [#7932](#7932) ->
`docs/changelog/2026-07-30.mdx`: Records independent
credential-generation E2E execution.
- [#7840](#7840) ->
`docs/changelog/2026-07-30.mdx`: Records shared-route preservation and
pre-delete peer validation during upgrades.
- [#7874](#7874) ->
`docs/changelog/2026-07-30.mdx`: Records the split between pre-tag
release entries and post-tag Announcements.
- [#7876](#7876) ->
`docs/changelog/2026-07-30.mdx`: Records the writable Hermes runtime
root within lockdown.
- [#7756](#7756) ->
`docs/changelog/2026-07-30.mdx`: Records validated multi-platform
managed-image publication.
- [#7914](#7914) ->
`docs/changelog/2026-07-30.mdx`: Records accepted `uv` version metadata
in Hermes image validation.
- [#7686](#7686) ->
`docs/changelog/2026-07-30.mdx`: Records the explicitly experimental
Microsoft Entra runtime identity reference.
- [#7869](#7869) ->
`docs/changelog/2026-07-30.mdx`: Records classified gateway relaunch
quarantine and rebuild guidance.
- [#7814](#7814) ->
`docs/changelog/2026-07-30.mdx`: Records state restore into replacement
sandboxes and SQLite write verification.
- [#7839](#7839) ->
`docs/changelog/2026-07-30.mdx`: Records quieter onboarding test
execution without a user-facing behavior claim.
- [#7854](#7854) ->
`docs/changelog/2026-07-30.mdx`: Records generalized agent-selection
guidance.
- [#7845](#7845) ->
`docs/changelog/2026-07-30.mdx`: Records isolated CDI test evidence
without a user-facing behavior claim.
- [#7843](#7843) ->
`docs/changelog/2026-07-30.mdx`: Records the corrected Omni sub-agent
model ID.
- [#7908](#7908) ->
`docs/changelog/2026-07-30.mdx`: Records reviewed Hermes and Deep Agents
Code dependency pins.
- [#7887](#7887) ->
`docs/changelog/2026-07-30.mdx`: Records rejection of a symlinked DGX
Station release marker.
- [#7747](#7747) ->
`docs/changelog/2026-07-30.mdx`: Records the internal compute-driver
separation without a user-facing behavior claim.
- [#7660](#7660) ->
`docs/changelog/2026-07-30.mdx`: Records atomic publication of rebuild
recovery manifests.
- [#7661](#7661) ->
`docs/changelog/2026-07-30.mdx`: Records bounded local inference
health-response retention.
- [#7654](#7654) ->
`docs/changelog/2026-07-30.mdx`: Records state preservation across
supervisor relaunch recovery.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [x] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [ ] Tests added or updated for changed behavior
- [x] Existing tests cover changed behavior — justification:
`test/changelog-docs.test.ts` validates the dated changelog contract,
SPDX comment, version heading, and published routes.
- [ ] Tests not applicable — justification:
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Documentation Writer Review

- [x] Documentation writer subagent reviewed the completed changes
- Result: `docs-updated`
- Evidence: `docs/changelog/2026-07-30.mdx`; the documentation-only diff
passed review against `WRITING.md`, the controlled word list, and
`docs/CONTRIBUTING.md`. The review covered terminology, structure,
active voice, release meaning, product-scope boundaries, and link and
code presentation. Changelog tests passed 6/6, and the docs build
reported 0 errors with 2 pre-existing warnings.
- Agent: Codex CLI
<!-- docs-review-head-sha: 200940f -->
<!-- docs-review-agents-blob-sha: c052d60 -->

## DGX Station Hardware Evidence

- [ ] Tested on DGX Station
- Tested commit:
- Station profile/scenario:
- Result:
- Supporting evidence:

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run validate:pr` passed after refreshing `origin/main` when hooks
were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: `npx
vitest run test/changelog-docs.test.ts` passed 6/6 tests.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result: Not applicable to this
documentation-only release entry.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) —
result: Build passed with 0 errors and 2 pre-existing warnings.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

---
Signed-off-by: San Dang <sdang@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Added release notes for v0.0.99 covering snapshot restoration, sandbox
recovery, gateway route upgrades, and Hermes security updates.
* Documented experimental Microsoft Entra runtime identity support and
enhanced readiness checks.
* Added details on managed image validation, trusted CI image promotion,
and end-to-end release evidence.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
@coderabbitai coderabbitai Bot mentioned this pull request Jul 31, 2026
23 tasks
@wscurran wscurran added area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure labels Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure

Projects

None yet

4 participants