perf(e2e): avoid second Hermes channel rebuild - #7580
Conversation
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThe changes add compiler coverage for persisted channel stop/start restoration and update live E2E expectations for queued changes, agent-scoped rebuilds, and lifecycle validation. ChangesChannel lifecycle validation
Estimated code review effort: 3 (Moderate) | ~20 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage in commit 324ee02 in the TypeScript / code-coverage/cliThe overall coverage in commit 324ee02 in the Show a code coverage summary of the most impacted files.
Updated |
PR Review Advisor — No blocking findings reportedAdvisor assessment: No blocking advisor findings reported Model lanes
Nemotron output stays in workflow artifacts and does not change the assessment above. E2E guidanceAdvisory only. E2E / PR Gate selects and runs jobs independently. Recommended E2E: This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge. |
|
Exact-head status for
Evidence: controller run 30195580897, pending coordination check. This needs the repository's authorized environment-review path (or its documented manual control-plane fallback), not a code change or blind workflow rerun. |
|
The OpenClaw Please hold another #7580 E2E dispatch until both #7582 and #7585 merge and #7580 is refreshed. One selected E2E run after that should validate the three previously failing lanes without spending another runner cycle on known prerequisites. |
<!-- markdownlint-disable MD041 --> ## Summary Current OpenClaw messaging extensions can link their OpenClaw peer dependency to the locked runtime package path. Backup and Shields audits treated that exact image-owned link as unsafe, so the channels stop/start E2E failed before backup even though the link came from the shipped image. This change accepts only the exact legacy and current image package targets while arbitrary targets and wrong source paths still fail closed. ## Related Issue Part of #7140. Unblocks the OpenClaw channels stop/start coverage in #7580. ## Changes - Keep the TypeScript backup/extraction policy and privileged Python state guard on the same two-target contract. - Accept extension peer links only at `extensions/<safe-id>/node_modules/openclaw` and only when they target the legacy global package or current locked runtime package. - Cover pre-backup audit, post-extraction audit, Shields transitions, current-runtime sibling tampering, and wrong-source rejection. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: this restores the existing documented backup/rebuild behavior; the internal image path and narrow audit exception are not user-facing configuration. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: an independent Codex security review passed after verifying that the TypeScript and Python guards share the exact two-target allowlist and retain path-shape, state-root, sibling-target, and wrong-source rejection. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `no-docs-needed` - Evidence: `docs/manage-sandboxes/backup-restore.mdx`, `docs/manage-sandboxes/set-up-whatsapp.mdx`, and `docs/reference/host-files-and-state.mdx` already describe the restored durable-state behavior and unsafe-link rejection. No documentation files changed. - Agent: Codex Desktop <!-- docs-review-head-sha: 702917d --> <!-- docs-review-agents-blob-sha: be20a09 --> ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: - Station profile/scenario: - Result: - Supporting evidence: ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, including the source-shape and 1,500-line test-file budget gates - [x] Targeted behavior tests pass for the current change set — 24 CLI policy tests, 82 backup/extraction tests, and 5 focused guard rejection tests passed; an offline Linux container also passed both trusted targets through preflight/lock/unlock and rejected sibling/wrong-source cases. - [x] Applicable broad gate passed — not applicable to this narrow exact-target compatibility fix; `npm run build:cli`, `npm run typecheck:cli`, source-shape, title-style, project-overlap, and full changed-file hooks passed. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Apurv Kumaria <akumaria@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Improved handling of OpenClaw extension links across supported installation locations. - Preserved valid extension links during state backups, restores, and transitions. - Continued rejecting untrusted or tampered links outside approved package paths. - **Tests** - Expanded coverage for multiple extensions and trusted runtime package locations. - Added regression checks for archive extraction, snapshots, and state-directory transitions. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- markdownlint-disable MD041 --> ## Summary When an exact managed Hermes image build on a local Docker-driver gateway starts in host-side BuildKit and exits unsuccessfully, retrying the same BuildKit-only Dockerfile through the gateway builder hides the useful error behind `RUN --mount requires BuildKit`. This change preserves that attempted BuildKit failure while retaining the existing fallback for every path that cannot prove the same managed build provenance. ## Related Issue Part of #7140 and #7144. This complements #7582, which fixes the stale Hermes base digest observed in #7580. It does not broaden #7253's OpenClaw compatibility behavior. ## Changes - Issue an in-memory capability only for a generated Hermes build on the local Docker-driver path. - Bind the capability to the staged context and Dockerfile identities plus the build ID, and reject copied, malformed, or drifted provenance. - Preserve the original failure only after that exact BuildKit attempt returns a nonzero or missing exit status. - Retain gateway fallback when prebuild is disabled or unavailable, the build cannot start, trust validation fails, provenance changes, the image is custom, the agent is OpenClaw, or the gateway is remote. - Cover capability issuance, provenance drift, retry behavior, exact failure preservation, and all unchanged fallback paths. - Document the narrow managed-Hermes recovery path. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: two independent exact-diff reviews found no security or correctness blocker; the final review verified the production relocation, canonical custom-image exclusion, exact context/Dockerfile/build binding, and unchanged fallback paths. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `docs-updated` - Evidence: `docs/reference/troubleshooting.mdx` documents nonzero and missing BuildKit exit statuses, the recovery step, and unchanged optional/custom/OpenClaw/remote behavior. - Agent: Codex Desktop <!-- docs-review-head-sha: 77856d8 --> <!-- docs-review-agents-blob-sha: be20a09 --> ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: - Station profile/scenario: - Result: - Supporting evidence: ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed - [x] Targeted behavior tests pass for the current change set — 42/42 focused CLI tests passed across `sandbox-prebuild.test.ts` and `sandbox-create-launch.test.ts`. - [x] Applicable broad gate passed — `npm run checks`, `npm run typecheck:cli`, project-membership, source-shape, title-style, test-size, and Biome checks passed. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — Fern completed with 0 errors and 2 pre-existing hidden warnings. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Apurv Kumaria <akumaria@nvidia.com> --------- Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
<!-- markdownlint-disable MD041 --> ## Summary Add the canonical `docs/changelog/2026-07-25.mdx` release entry with the exact `## v0.0.96` heading. The entry reconciles all 90 first-parent commits since v0.0.95 with all 92 merged PRs in the live `v0.0.96` label ledger and groups the user-visible changes by operator journey. ## Changes - Add the parser-safe dated MDX changelog entry for v0.0.96 with root-absolute links to the focused user guides. - Source summary: - [#7194](#7194) -> `docs/changelog/2026-07-25.mdx`: Document persistent baseline network policy exclusions and their inspection, rebuild, and snapshot behavior. - [#7188](#7188), [#7427](#7427), and [#7546](#7546) -> `docs/changelog/2026-07-25.mdx`: Document DNS-backed HTTPS inference routing, keyless loopback endpoints, and provider-marker isolation. - [#7238](#7238) -> `docs/changelog/2026-07-25.mdx`: Document blueprint sandbox and provider identifier validation before state writes or OpenShell calls, with bounded terminal-safe rejection previews. - [#7319](#7319), [#7274](#7274), [#7528](#7528), [#7353](#7353), and [#7560](#7560) -> `docs/changelog/2026-07-25.mdx`: Document the managed default gateway service, onboarding readiness, and container-runtime identity safeguards. - [#7349](#7349), [#7498](#7498), [#7406](#7406), [#7196](#7196), [#7559](#7559), [#7421](#7421), [#7510](#7510), [#7295](#7295), and [#7565](#7565) -> `docs/changelog/2026-07-25.mdx`: Document gateway-scoped status, lifecycle diagnostics, managed MCP recovery, delete-edge safeguards, and fail-closed CLI prompt and command output. - [#7591](#7591) -> `docs/changelog/2026-07-25.mdx`: Document opt-in authenticated MCP tool-name discovery, its bounded and names-only contract, probe interaction, and rebuild requirement. - [#7305](#7305), [#7480](#7480), [#7471](#7471), [#7365](#7365), and [#7541](#7541) -> `docs/changelog/2026-07-25.mdx`: Document installer version checks, version-tag reporting, license guidance, WSL Ollama selection, and DGX Station vLLM detection. - [#7482](#7482), [#7466](#7466), [#7208](#7208), [#7434](#7434), and [#7586](#7586) -> `docs/changelog/2026-07-25.mdx`: Document Ollama resource details, reasoning precedence, Hermes onboarding behavior, and preserved managed Hermes BuildKit failures. - [#6830](#6830), [#7492](#7492), [#7563](#7563), and [#7582](#7582) -> `docs/changelog/2026-07-25.mdx`: Document the authoritative OpenClaw production lock, fixed managed-image dependencies, immutable Hermes base adoption, and Hermes image-size reduction. - [#7505](#7505), [#7530](#7530), [#7547](#7547), [#7508](#7508), [#7548](#7548), [#7549](#7549), [#7537](#7537), [#7534](#7534), [#7515](#7515), [#7511](#7511), [#7551](#7551), [#7562](#7562), [#7575](#7575), [#7496](#7496), [#7594](#7594), [#7595](#7595), and [#7599](#7599) -> `docs/changelog/2026-07-25.mdx`: Summarize release validation, transient and bounded dispatch reconciliation, exact pre-tag qualification, identity revalidation, npm-audit retry, sharding, image reuse, timeout, telemetry, and workflow-hardening changes. - Reconciled without separate changelog prose: - [#7539](#7539), [#7526](#7526), [#7507](#7507), [#7506](#7506), [#7519](#7519), [#7516](#7516), [#7396](#7396), [#7254](#7254), [#7583](#7583), [#7596](#7596), and [#7598](#7598): Test-harness or fixture-only changes. - [#7403](#7403), [#7161](#7161), [#6877](#6877), [#7531](#7531), [#7525](#7525), [#7522](#7522), [#7536](#7536), [#7552](#7552), [#7566](#7566), [#7553](#7553), [#7561](#7561), [#7577](#7577), [#7569](#7569), [#7585](#7585), [#7584](#7584), [#7592](#7592), [#7580](#7580), [#7571](#7571), [#7517](#7517), [#7589](#7589), [#7402](#7402), [#7558](#7558), [#7544](#7544), and [#7601](#7601): Dependency, internal recovery, validation, contributor-workflow, E2E optimization, telemetry, or CI trust changes with no separate user-facing release claim. - [#7556](#7556), [#7573](#7573), [#7576](#7576), and [#7578](#7578): Experimental repository-maintainer conflict automation with no canonical user documentation surface. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [x] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [ ] Tests added or updated for changed behavior - [x] Existing tests cover changed behavior — justification: `test/changelog-docs.test.ts` validates dated changelog structure, version headings, and published links. - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `docs-updated` - Evidence: Reviewed `docs/changelog/2026-07-25.mdx` at exact head `0f5dedb47` against 90 first-parent release commits and 92 merged PRs labeled `v0.0.96`. Verified parser-safe MDX SPDX, the exact version heading, literal CLI names, writing style, skip terms, all 20 root-absolute published links, and the accepted #7591 opt-in authenticated discovery bounds. #7544, #7599, and #7601 remain internal or CI-only release-ledger entries. Changelog tests passed 6/6, the docs build passed with 0 errors and two pre-existing Fern warnings, and `npm run check:diff` plus the final diff check passed. - Agent: Codex Desktop documentation-writer subagent <!-- docs-review-head-sha: 0f5dedb --> <!-- docs-review-agents-blob-sha: be20a09 --> ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: - Station profile/scenario: - Result: - Supporting evidence: ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — `npx vitest run test/changelog-docs.test.ts`: 6/6 passed. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: Not applicable to this prose-only changelog entry. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — the build passed with 0 errors and 2 existing Fern warnings; the published-route check passed. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) — native changelog files use the required parser-safe MDX SPDX comment and no frontmatter. --- Signed-off-by: Carlos Villela <cvillela@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Persistent network policy exclusions with consistent restore/exclusion reporting across rebuilds/snapshots. * Opt-in MCP tool discovery via `mcp status --tools` with bounded, redacted authenticated traffic. * Improved HTTPS inference switching for custom endpoints and refreshed onboarding/model menu details. * Refined OpenShell gateway defaults for port `8080`, including more reliable readiness checks. * **Bug Fixes** * Prevent incorrect provider/model restoration after compatible-provider update failures. * Preserve managed MCP state after exec loss and tighten gateway/doctor status scoping. * **Tests** * Stronger, fail-closed release validation with hardened evidence/artifact handoff and bounded timeouts/retries. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> Co-authored-by: Prekshi Vyas <prekshiv@nvidia.com>
## Summary The rebuild-Hermes lanes now prepare the published current base, trusted gateway, hosted inference route, and dashboard port directly instead of onboarding and deleting a disposable current-Hermes sandbox. This removes one generated image build and its high-memory export while preserving the real historical-to-current rebuild and state-migration assertions. The branch is refreshed onto exact `main` SHA `c4c020ca5` after benchmark prerequisites #7571, #7580, #7582, #7586, and #7589 merged. Their changes collapse out of the PR diff; the remaining five changed files are limited to `test/e2e/**`. ## Related Issue Part of #7144 Parent epic: #7140 ## Changes - Resolve Hermes through production `ensureAgentBaseImage`, require the published immutable metadata, and fail if the lane constructs or overrides a base. - Start the `nemoclaw` gateway through the production recovery path, configure the exact compatible-endpoint route, and allocate the dashboard port through the production allocator. - Keep the real rebuild credentialless and retain old-base provenance, backup/restore, messaging placeholders, token rotation, final image identity, readiness, and inference validation. - Preserve the existing eight-phase contract with truthful setup wording and consistently numbered artifacts. - List forward ownership before cleanup and use sandbox-scoped stops so a reused port cannot terminate another sandbox's forward. - Run exit-capable production bootstrap functions in captured child processes so failures cannot terminate the Vitest worker. - Use the workflow-selected absolute OpenShell executable for every provider, readiness, sandbox, and validation operation so PATH drift cannot split the lane across binaries. - Record malformed persisted dashboard-port state, attempt every known sandbox-owned forward even when one stop fails, write cleanup evidence, and only then propagate one or aggregated cleanup failures. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: the diff only changes internal live-E2E orchestration and exposes no user-facing command, configuration, runtime default, or output. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: independent Codex maintainers reviewed the full exact diff through signed/Verified head `c067fb6ce` for credentials, gateway authority, immutable base identity, cleanup ownership, dashboard allocation, deterministic failure aggregation, evidence preservation, child-process failure propagation, and retained real rebuild/state/token/inference assertions. Final verdict: PASS with no findings. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `no-docs-needed` - Evidence: Exact diff `c4c020ca5..c067fb6` changes only `test/e2e/**` test infrastructure. No documentation paths or user-facing behavior changed. `git diff --check` passed. - Agent: Codex Desktop <!-- docs-review-head-sha: c067fb6 --> <!-- docs-review-agents-blob-sha: be20a09 --> ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: not applicable - Station profile/scenario: not applicable - Result: not applicable; `scripts/prepare-dgx-station-host.sh` is unchanged. - Supporting evidence: ## Verification - [x] PR description includes a `Signed-off-by:` line and every pushed commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — the prepared current-main sync passed its 17/17 focused support tests; the prior broader exact suite passed all ten rebuild-Hermes support files (10 files, 63 tests), including the fail-closed markerless-bootstrap and all-forwards cleanup regressions. - [ ] Applicable broad gate passed — not applicable; this is scoped to one live target and its focused support contracts. A prior full local e2e-support attempt passed 154 files and hit nine unrelated macOS host/process timing failures outside the changed files. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) Additional validation on the refreshed head: - `npm run test:e2e-phases:check` - `npm run typecheck:cli` - `npm run source-shape:check` - `npm run test-size:check` - `npm run test:titles:check` - `npm run test:projects:check` - `npm run test-conditionals:scan -- --top 25` - `npx prek run --from-ref origin/main --to-ref HEAD --stage pre-commit` - `npx prek run --from-ref origin/main --to-ref HEAD --stage pre-push` Five-run benchmark baseline: - Exact main SHA: `0b185498155a0a51a3f682a3e2b57f80c95eeaaa` - Runner routing: `E2E_LARGER_RUNNER_LABEL` unset; standard `ubuntu-latest`, Linux/x64, 4 CPU, about 16 GB memory - Protocol: five sequential selective `e2e.yaml` dispatches; attempt 1 only; both lanes required to pass; zero Docker build cache at scenario start | Sample | Workflow run | Normal total | Stale-base total | | --- | --- | ---: | ---: | | 1 | [30241442305](https://github.com/NVIDIA/NemoClaw/actions/runs/30241442305) | 322,906 ms | 323,358 ms | | 2 | [30241954029](https://github.com/NVIDIA/NemoClaw/actions/runs/30241954029) | 324,529 ms | 324,778 ms | | 3 | [30242448958](https://github.com/NVIDIA/NemoClaw/actions/runs/30242448958) | 321,993 ms | 322,012 ms | | 4 | [30242980946](https://github.com/NVIDIA/NemoClaw/actions/runs/30242980946) | 325,750 ms | 327,602 ms | | 5 | [30243543432](https://github.com/NVIDIA/NemoClaw/actions/runs/30243543432) | 323,497 ms | 443,959 ms | | **Median** | | **323,497 ms** | **324,778 ms** | Median phase evidence: - Disposable current-Hermes onboard removed by this PR: normal 138,276 ms; stale-base 138,599 ms - Historical fixture pull: normal 46,237 ms; stale-base 50,446 ms - Historical sandbox creation: normal 27,257 ms; stale-base 31,883 ms - Actual Hermes rebuild retained by this PR: normal 90,521 ms; stale-base 80,321 ms - The removed onboard alone represents 42.7% of each baseline median. This identifies the expected gain but is not substituted for the required post-change measurement. Resource evidence: - All ten lanes began with zero build cache, used zero swap, recorded zero memory-full PSI, passed semantic validation, uploaded artifacts, and completed cleanup without failures. - Peak BuildKit RSS ranged from 3,124,420 to 4,075,160 KiB while minimum available memory stayed at or above 10,371,420 KiB; the baseline does not show memory exhaustion. - Sample 5 stale-base is an I/O/runner-class outlier: its onboard phase took 256,419 ms on Intel Xeon 6973P-C with 40.84% peak I/O-full PSI. The outlier does not move the five-run median. Remaining acceptance evidence: - Run the exact-head trusted two-lane smoke through the refreshed PR controller after E2E authorization. - After merge, run the matching five-sequential-run cohort on `main` and confirm at least 25% median wall-time improvement independently for both lanes, as required by #7144. --- Signed-off-by: Apurv Kumaria <akumaria@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added end-to-end helper coverage for rebuilding Hermes, including trusted current-base resolution and hosted inference gateway bootstrapping with readiness markers and bootstrap artifacts. * Introduced stricter validation for OpenShell selection, inference route/provider-model matching, and dashboard/forward port handling. * **Bug Fixes** * Improved error messaging when current-base evidence validation fails during rebuild. * **Tests** * Added a dedicated “rebuild Hermes direct bootstrap” e2e suite with marker, environment, routing, and cleanup assertions. * Updated the live rebuild e2e flow to use the dynamically selected OpenShell and enhanced forward-port tracking/cleanup behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Apurv Kumaria <akumaria@nvidia.com> Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> Co-authored-by: Prekshi Vyas <prekshiv@nvidia.com>
Summary
The Hermes
channels-stop-startlane rebuilt once after disabling all channels and then rebuilt the same active configuration again after re-enabling them. This keeps the uniquely valuable disabled-state rebuild, replaces only the second Hermes rebuild with exact queued-state and persisted-plan assertions, and leaves the OpenClaw two-rebuild path unchanged.In the representative E2E run 30181872151, the removed rebuild took about 2m07s, roughly 20% of the 10m46s Hermes job.
Related Issue
Part of #7144.
Parent epic: #7140.
Changes
The persistence-aware test asserts semantic plan/render parity. It intentionally does not claim byte-identical Base64 cache input because compact-plan hydration can change JSON key insertion order.
Type of Change
Quality Gates
324ee020c.Documentation Writer Review
no-docs-neededDGX Station Hardware Evidence
Verification
Signed-off-by:line and every commit appears asVerifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run check:diffpassed when hooks were skipped or unavailableworkflow-planner.test.ts23/23; E2E workflow boundary 43/43; semantic phase coverage 125 tests across 82 files.npm run checks, CLI typecheck, plugin source/test typecheck, CLI build, and plugin build passed.npm run docsbuilds without warnings (doc changes only)Signed-off-by: Apurv Kumaria akumaria@nvidia.com
Summary by CodeRabbit
Enhancements
Tests