Skip to content

fix(rebuild): journal same-name sandbox replacement - #7788

Merged
senthilr-nv merged 41 commits into
mainfrom
fix/rebuild-recreate-journal
Jul 31, 2026
Merged

fix(rebuild): journal same-name sandbox replacement#7788
senthilr-nv merged 41 commits into
mainfrom
fix/rebuild-recreate-journal

Conversation

@laitingsheng

@laitingsheng laitingsheng commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

rebuild and non-resumed onboard previously deleted a same-name sandbox before durably recording its replacement. They now open the canonical recreation transaction before the first destructive mutation, bind it to the exact source, target, and gateway, and recover or fail closed after interruption instead of guessing.

Related Issue

Fixes #7734
Fixes #7735

Product Scope Approval

Product scope is approved independently of GitHub mergeStateStatus: accepted maintainer-authored issues #7734 and #7735 define the same-name replacement recovery contract and ownership. Maintainer direction in #6492 also establishes lifecycle recovery as supported NemoClaw behavior. This PR does not add a new third-party integration, custom image, or solution recipe.

Changes

  • Open and reconcile the existing CheckpointSandboxRecreateTransaction before rebuild or onboard replacement mutates sandbox state.
  • Bind recovery to the recorded gateway, source registry generation and fingerprint, hashed OpenShell identity, and validated non-secret target fingerprint.
  • Record deleting before deletion and deleted only after exact-gateway absence is proved; ambiguous observations fail closed and preserve recovery evidence.
  • Preserve the journaled source registry row until replacement registration commits, and resume safely from every pre- and post-creation journal phase.
  • Share strict gateway-scoped recreation observation between rebuild and onboard, and explicitly select the gateway for replacement get, list, and delete operations.
  • Document interrupted replacement recovery and command behavior in docs/manage-sandboxes/recover-rebuild-sandboxes.mdx and docs/reference/commands.mdx.
  • Update focused lifecycle, restart, authority, integration, and architecture-budget coverage.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification:
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: independent Codex Desktop security specialist review for exact head 3e6c3d0391f9fd7e58ce9775871807b9b3c2ff8e is PASS; this is agent evidence, not human review.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: docs-updated
  • Evidence: docs/manage-sandboxes/recover-rebuild-sandboxes.mdx; docs/reference/commands.mdx. Exact head 3e6c3d0391f9fd7e58ce9775871807b9b3c2ff8e, base 299050fc0563db0cd3a803298a6fb986dcfbb745; PASS with no blocking writing findings. npm run docs completed with 0 errors and two unchanged advisory warnings.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit: Not applicable; scripts/prepare-dgx-station-host.sh is unchanged.
  • Station profile/scenario: Not applicable.
  • Result: Not applicable.
  • Supporting evidence: Not applicable. No hardware review is claimed.

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: exact head 3e6c3d0391f9fd7e58ce9775871807b9b3c2ff8e; 405 focused CLI and integration tests passed across 27 files; npm run typecheck:cli passed; source architecture passed with 1,425 files, 4,201 edges, one allowed cycle, max fan-in 103, and max fan-out 225.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: local npm run check did not provide valid evidence on this unsupported host (Node 23.7.0 instead of the repository's Node 22.19+, missing Python yaml, and missing the official OpenShell Homebrew formula caused unrelated failures and timeouts). Fresh supported GitHub CI for this exact head/base pair is required before merge.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only) — 0 errors; two unchanged advisory warnings remain (Fern redirect-auth availability and an existing light-mode accent-contrast warning).
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only) — no new documentation pages.

Signed-off-by: Tinson Lai tinsonl@nvidia.com

`rebuild` deleted the sandbox and only then handed off to resumed onboarding,
so nothing durable proved the source, target, or completed phase. A termination
in that window left a later command with no replacement intent to reconcile,
and the ordinary path had already dropped the registry row.

Open the canonical recreate transaction before the destroy phase, bound to the
recorded gateway, the source registry fingerprint and live identity, and a
target fingerprint built from validated non-secret rebuild inputs. Record
`deleting` before the delete command and `deleted` only after the journal
re-proves absence on that exact gateway; a probe that shows neither a live
sandbox nor explicit absence stops the command with the backup, MCP state, and
registry recovery data preserved.

Keep the journaled source row across the delete for every rebuild, not just the
MCP-bearing and baseline-exclusion cases, so a restart between deletion and
replacement registration still has a source contract to reconcile. Prepared
recovery therefore no longer reverses a default-sandbox transition, because none
happens.

Carry the journal across the session reset the recreate phase performs, rebinding
only the sandbox identity, gateway authority, and transaction onto a checkpoint
derived from the new session, and hand the journaled target fingerprint to the
inner run so it adopts the open transaction instead of starting its own.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Rebuild and non-resumed onboarding now use durable, gateway-bound sandbox replacement journals. Deletion is journaled and confirmed before recreation. Checkpoint state and target fingerprints cross onboarding boundaries. Recovery tests cover failure, restart, identity drift, registry preservation, and multi-gateway behavior.

Changes

Sandbox recreate journaling

Layer / File(s) Summary
Journal contracts and gateway observation
src/lib/onboard/sandbox-recreate-probe.ts, src/lib/actions/sandbox/rebuild-recreate-journal.ts, src/lib/onboard/onboard-recreate-journal.ts, src/lib/onboard/sandbox-recreate-transaction.ts
Adds target fingerprinting, gateway-scoped observation, phase-aware progression, durable registry fingerprints, and fail-closed source identity checks.
Rebuild deletion and recreation boundary
src/lib/actions/sandbox/rebuild-destroy-phase.ts, src/lib/actions/sandbox/rebuild-pipeline.ts, src/lib/actions/sandbox/rebuild-recreate-phase.ts, src/lib/onboard.ts
Records deletion before OpenShell mutation, skips deletion when the source is already absent, confirms absence through the journal, preserves the journaled registry row, and carries journal state into recreation.
Onboarding checkpoint and gateway handoff
src/lib/onboard/machine/*, src/lib/onboard/types.ts, src/lib/onboard/sandbox-reuse.ts, src/lib/actions/sandbox/rebuild-gpu-opt-out.ts
Carries checkpoint state, target intent fingerprints, inference-route inputs, and gateway-scoped sandbox operations across recreation.
Recovery and lifecycle validation
src/lib/actions/sandbox/*test.ts, src/lib/onboard/*test.ts, test/helpers/*, test/onboard-*.test.ts, test/rebuild-*.test.ts
Expands journal, restart, identity, gateway, deletion-state, registry-preservation, rollback, and stable-identity coverage.
Documentation and architecture budgets
docs/manage-sandboxes/recover-rebuild-sandboxes.mdx, docs/reference/commands.mdx, ci/source-architecture-budget.json
Documents interrupted replacement recovery and updates architecture budget thresholds and cycle/root-file settings.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

  • NVIDIA/NemoClaw#7369: Extends deletion confirmation from polling-based convergence to journal-driven absence observation.
  • NVIDIA/NemoClaw#7690: Introduced the recreation transaction infrastructure extended by this change.
  • NVIDIA/NemoClaw#7840: Also changes rebuild deletion-edge validation for shared inference route drift.

Suggested labels: area: onboarding

Suggested reviewers: cv

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 11.90% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes implement canonical journaling, gateway-scoped checks, fail-closed recovery, onboarding handoff, and tests for #7734 and #7735.
Out of Scope Changes check ✅ Passed The code, tests, documentation, and budget updates support durable same-name rebuild or onboarding replacement journaling.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: adding journaling for same-name sandbox replacement during rebuild.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/rebuild-recreate-journal

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 3e6c3d0 in the fix/rebuild-recreate... branch remains at 96%, unchanged from commit 299050f in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 3e6c3d0 in the fix/rebuild-recreate... branch remains at 81%, unchanged from commit 299050f in the main branch.

Show a code coverage summary of the most impacted files.
File main 299050f fix/rebuild-recreate... 3e6c3d0 +/-
src/lib/state/o...d-checkpoint.ts 90% 82% -8%
src/lib/onboard...ndbox-create.ts 91% 83% -8%
src/lib/shields/index.ts 69% 69% 0%
src/lib/shields...nsition-lock.ts 86% 87% +1%
src/lib/actions...estroy-phase.ts 84% 87% +3%
src/lib/onboard...-transaction.ts 88% 95% +7%
src/lib/inference/gpu-trust.ts 64% 100% +36%
src/lib/onboard...eate-journal.ts 0% 97% +97%
src/lib/actions...eate-journal.ts 0% 100% +100%
src/lib/onboard...create-probe.ts 0% 100% +100%

Updated July 31, 2026 19:29 UTC

@laitingsheng laitingsheng added area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery bug-fix PR fixes a bug or regression labels Jul 29, 2026
@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — Blocking findings reported

Advisor assessment: Blockers require maintainer review
Next action: Review the blockers below.
Findings: 1 blocker · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 1 blocker · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Model comparison: normalized findings differ; normalized E2E selections differ; Nemotron reported 1 fewer blocker, the same number of warnings, the same number of suggestions.
6 additional E2E selections from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

  • rebuild-hermes-stale-base: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • sandbox-survival: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • gateway-guard-recovery: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • openshell-gateway-upgrade: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • double-onboard: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • gpu-double-onboard: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion E2E selections are advisory. They do not change the primary assessment or E2E / PR Gate.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-onboard, onboard-repair, onboard-resume, rebuild-openclaw, state-backup-restore

2 optional E2E recommendations
  • rebuild-hermes
  • sandbox-operations

Blockers

PRA-1 Blocker — Bind every replacement input in the rebuild journal

  • Location: src/lib/actions/sandbox/rebuild-recreate-journal.ts:43
  • Category: acceptance
  • Problem: The rebuild target fingerprint omits non-secret replacement inputs that affect the created sandbox, including endpoint URL, compatible-endpoint reasoning settings, web-search configuration, messaging intent, and prepared image identity. The inner sandbox handler trusts this supplied fingerprint, so a restart can continue a journal after those target inputs change.
  • Impact: A resumed rebuild can create or accept a replacement whose endpoint, reasoning, web-search, messaging, or image configuration differs from the replacement intent that passed the original destructive boundary.
  • Fix: Derive the rebuild journal fingerprint from the complete validated secret-free create intent that inner onboarding will consume, including endpoint, reasoning, web-search, messaging, and image identity inputs. Reject recovery when any of those inputs differ.
  • Verification: Inspect the journal fingerprint payload and compare it with the complete resolved create intent passed to inner onboarding; run the focused rebuild recreate-journal tests.
  • Test coverage: Add restart tests that change each omitted target input after journaling and assert recovery stops before deletion, creation, provider mutation, or registry commit.
  • Simplification (shrink): Remove The separate partial rebuild fingerprint payload.; use Fingerprint the existing complete validated secret-free create intent.. Net: 0 lines.
  • Keep: Preserve all source-identity, gateway, and target-generation checks.
  • Evidence: src/lib/actions/sandbox/rebuild-recreate-journal.ts:43-85 src/lib/onboard/machine/handlers/sandbox.ts:1248-1253 Linked issue fix(rebuild): journal same-name sandbox replacement #7734, Required behavior and Fail-closed conditions: target fingerprint must use validated non-secret rebuild inputs and a mismatch must stop recovery.

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/actions/sandbox/rebuild-destroy-phase.ts`:
- Line 359: Wrap the recreateJournal.markDeleting() call in the rebuild/destroy
phase with failure handling that invokes reattachMcpAfterDeleteFailure for the
prepared MCP entries, relocks the shields, and exits before any OpenShell
invocation. Preserve propagation or reporting of the original durable-state
error while ensuring the still-running sandbox is restored when markDeleting
throws.

In `@src/lib/actions/sandbox/rebuild-recreate-observability.test.ts`:
- Around line 221-235: Update the test around rebuildOnboardDependencies.onboard
to retain the spy returned by vi.spyOn and restore it in a finally block
surrounding the execution and assertions. Preserve the existing observations and
expectations while ensuring restoration occurs even when an assertion fails.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9827a9f4-ebe9-46fe-bf23-f3e6bcd56219

📥 Commits

Reviewing files that changed from the base of the PR and between eeab81c and 083fa1d.

📒 Files selected for processing (21)
  • ci/source-architecture-budget.json
  • src/lib/actions/sandbox/rebuild-destroy-phase.test.ts
  • src/lib/actions/sandbox/rebuild-destroy-phase.ts
  • src/lib/actions/sandbox/rebuild-gpu-opt-out.ts
  • src/lib/actions/sandbox/rebuild-pipeline.ts
  • src/lib/actions/sandbox/rebuild-prepared-recovery.test.ts
  • src/lib/actions/sandbox/rebuild-recreate-journal.test.ts
  • src/lib/actions/sandbox/rebuild-recreate-journal.ts
  • src/lib/actions/sandbox/rebuild-recreate-observability.test.ts
  • src/lib/actions/sandbox/rebuild-recreate-phase.ts
  • src/lib/actions/sandbox/rebuild-shields-finally.test.ts
  • src/lib/onboard.ts
  • src/lib/onboard/machine/core-flow-phases.ts
  • src/lib/onboard/machine/handlers/sandbox.ts
  • src/lib/onboard/sandbox-recreate-transaction.ts
  • src/lib/onboard/types.ts
  • test/helpers/rebuild-flow-harness.ts
  • test/helpers/rebuild-flow-lifecycle-cases.ts
  • test/helpers/rebuild-flow-recovery-cases.ts
  • test/helpers/rebuild-flow-test-support.ts
  • test/mcp-destroy-lifecycle.test.ts

Comment thread src/lib/actions/sandbox/rebuild-destroy-phase.ts Outdated
Comment thread src/lib/actions/sandbox/rebuild-recreate-observability.test.ts Outdated
`createSandbox` re-derives its own recreation triggers, so a plain onboard run
could delete and replace a live same-name sandbox with no durable replacement
intent. The sandbox handler only opened the canonical journal when resuming, and
without one the lifecycle runtime was an inert stub: the delete boundary, the
post-delete absence proof, and the registry-row reservation all became no-ops.

Open the canonical recreate transaction inside `createSandbox`, after every
required confirmation and non-mutating check and before provider cleanup, the
OpenShell delete, or registry removal. Bind it to the sandbox name, the resolved
gateway name and port, the source registry fingerprint and generation, the
hashed source OpenShell identity, and a target fingerprint built from validated
non-secret create inputs. An active journal is reconciled through
`planSandboxRecreateRecovery` before any mutation, so a later invocation resumes
the replacement without `--resume`.

Fail closed instead of guessing: a missing source registry row, a live sandbox
with no stable OpenShell Id, a changed source identity, a changed target
fingerprint, and a probe that reports neither a live sandbox nor explicit
absence all stop the command before the next mutation.

Name the gateway explicitly on the sandbox get, list, and delete commands that
drive replacement, so a host running several gateways cannot answer or mutate
for a sibling. Extract the shared strict-absence classifier and gateway-scoped
observation so the rebuild and onboard journals prove absence the same way.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard.ts`:
- Around line 2566-2567: The recovery journal must observe the recorded gateway
rather than resolving the ambient gateway at runtime. Update the `observe`
callback passed to `openOnboardRecreateJournal` and the
`getSandboxRecreateObservation` API to propagate `probeTarget.gatewayName`
alongside `sandboxName`, ensuring resumed recovery inspects the journal-scoped
gateway; add coverage for resuming on a non-default gateway.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d483639f-382f-4a76-a303-d7f48f175fd1

📥 Commits

Reviewing files that changed from the base of the PR and between 083fa1d and a2254bd.

📒 Files selected for processing (18)
  • ci/source-architecture-budget.json
  • src/lib/actions/sandbox/gateway-state.ts
  • src/lib/actions/sandbox/rebuild-destroy-phase.ts
  • src/lib/actions/sandbox/rebuild-recreate-journal.ts
  • src/lib/onboard.ts
  • src/lib/onboard/onboard-recreate-journal.test.ts
  • src/lib/onboard/onboard-recreate-journal.ts
  • src/lib/onboard/sandbox-recreate-probe.ts
  • src/lib/onboard/sandbox-reuse.ts
  • test/onboard-custom-dockerfile.test.ts
  • test/onboard-extra-provider-reconciliation.test.ts
  • test/onboard-installer-restore-intent.test.ts
  • test/onboard-messaging.test.ts
  • test/onboard-reservation-recreate.test.ts
  • test/onboard-sandbox-build.test.ts
  • test/onboard-sandbox-recreation.test.ts
  • test/onboard.test.ts
  • test/shellquote-sandbox.test.ts
💤 Files with no reviewable changes (1)
  • src/lib/actions/sandbox/gateway-state.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/lib/actions/sandbox/rebuild-recreate-journal.ts
  • src/lib/actions/sandbox/rebuild-destroy-phase.ts

Comment thread src/lib/onboard.ts Outdated
The journal opened inside createSandbox has no outer owner, so it now
advances to completed and clears itself once the replacement registry row
commits. Without that a later replacement met a stale active transaction
and refused to start.

Read the committed journal back from the session the write returned rather
than through a store that may not observe it yet, keep src/lib/onboard.ts
net-neutral by moving the managed-MCP refusal text into the journal module,
and model post-delete absence in the onboard stubs the guard now requires.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
A restart that observed a registered, ready sandbox carrying the
journaled target generation and identity still entered the destroy
phase, so the rebuild could delete the replacement it had already
proved. Surface that accepted target from the journal, retire the
transaction, and end the rebuild before deletion.

Reattach the prepared MCP entries when the delete boundary cannot be
journaled, observe a resumed replacement on the gateway its journal
records rather than the ambient one, and stop the Docker GPU rollback
tests from probing DNS through a live container.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
src/lib/actions/sandbox/rebuild-destroy-phase.test.ts (2)

932-971: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

Solid coverage of the journal-failure boundary.

Correctly verifies MCP reattachment, shield relock, rejection, and that the destructive OpenShell delete never fires when markDeleting() throws — matching the try/catch in runRebuildDestroyPhase.

One small nit: Line 966 uses not.toHaveBeenCalledWith(specificArgs), which would still pass if runOpenshell were called with different arguments. not.toHaveBeenCalled() would be a strictly tighter guarantee that no OpenShell invocation occurs at all in this failure path.

🔧 Tighten the negative assertion
-    expect(mocks.runOpenshell).not.toHaveBeenCalledWith(
-      ["sandbox", "delete", "-g", "nemoclaw", "alpha"],
-      expect.anything(),
-    );
+    expect(mocks.runOpenshell).not.toHaveBeenCalled();
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/actions/sandbox/rebuild-destroy-phase.test.ts` around lines 932 -
971, In the test “reattaches MCP providers when the delete boundary cannot be
journaled,” replace the argument-specific negative assertion on
mocks.runOpenshell with an assertion that it was never called at all, preserving
the existing failure-path coverage.

76-86: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extract stubRecreateJournal() into a shared test helper.

The identical helper is duplicated verbatim in test/mcp-destroy-lifecycle.test.ts; this PR had to update both copies with the same three new fields. Centralizing it (e.g., in test/helpers/) would avoid future drift.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/actions/sandbox/rebuild-destroy-phase.test.ts` around lines 76 - 86,
Extract the duplicated stubRecreateJournal helper into a shared test utility,
then import and reuse it in both rebuild-destroy-phase.test.ts and
mcp-destroy-lifecycle.test.ts. Preserve the current returned
RebuildRecreateJournal shape, including all three mock methods and target
fields, while removing the local duplicate definitions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/lib/actions/sandbox/rebuild-destroy-phase.test.ts`:
- Around line 932-971: In the test “reattaches MCP providers when the delete
boundary cannot be journaled,” replace the argument-specific negative assertion
on mocks.runOpenshell with an assertion that it was never called at all,
preserving the existing failure-path coverage.
- Around line 76-86: Extract the duplicated stubRecreateJournal helper into a
shared test utility, then import and reuse it in both
rebuild-destroy-phase.test.ts and mcp-destroy-lifecycle.test.ts. Preserve the
current returned RebuildRecreateJournal shape, including all three mock methods
and target fields, while removing the local duplicate definitions.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 37030ec8-d631-4fa5-8192-06c59eca3332

📥 Commits

Reviewing files that changed from the base of the PR and between 3109fe7 and 3d15a73.

📒 Files selected for processing (13)
  • src/lib/actions/sandbox/rebuild-destroy-phase.test.ts
  • src/lib/actions/sandbox/rebuild-destroy-phase.ts
  • src/lib/actions/sandbox/rebuild-pipeline.ts
  • src/lib/actions/sandbox/rebuild-recreate-journal.test.ts
  • src/lib/actions/sandbox/rebuild-recreate-journal.ts
  • src/lib/actions/sandbox/rebuild-recreate-observability.test.ts
  • src/lib/onboard.ts
  • src/lib/onboard/docker-gpu-patch-rollback.test.ts
  • src/lib/onboard/sandbox-reuse.test.ts
  • src/lib/onboard/sandbox-reuse.ts
  • test/helpers/rebuild-flow-recovery-cases.ts
  • test/mcp-destroy-lifecycle.test.ts
  • test/onboard-messaging.test.ts
🚧 Files skipped from review as they are similar to previous changes (8)
  • src/lib/actions/sandbox/rebuild-pipeline.ts
  • src/lib/actions/sandbox/rebuild-recreate-observability.test.ts
  • test/mcp-destroy-lifecycle.test.ts
  • src/lib/onboard/sandbox-reuse.ts
  • test/onboard-messaging.test.ts
  • src/lib/actions/sandbox/rebuild-recreate-journal.test.ts
  • src/lib/actions/sandbox/rebuild-destroy-phase.ts
  • src/lib/onboard.ts

Rebuild the proven-replacement helper without a guard branch so changed
test files add no conditionals, and tighten the shell-quote fan-in budget
to the count the current tree reports.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/lib/actions/sandbox/rebuild-recreate-journal.test.ts (1)

346-353: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Constrain the registry mock to the expected lookup.

mockReturnValue ignores every getSandbox argument, so a regression that queries the wrong sandbox name or gateway still receives the alpha fixture and may pass. Use a narrow mockImplementation that validates the requested identity and gateway. As per path instructions, broad mocks that bypass the behavior under test should be flagged.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/actions/sandbox/rebuild-recreate-journal.test.ts` around lines 346 -
353, Replace the broad mockReturnValue on registry.getSandbox with a
mockImplementation that checks the requested sandbox identity and gateway
against the expected alpha and gateway values, returning the fixture only for
that lookup. Preserve the existing fixture fields and make unexpected arguments
fail rather than silently returning alpha.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@src/lib/actions/sandbox/rebuild-recreate-journal.test.ts`:
- Around line 346-353: Replace the broad mockReturnValue on registry.getSandbox
with a mockImplementation that checks the requested sandbox identity and gateway
against the expected alpha and gateway values, returning the fixture only for
that lookup. Preserve the existing fixture fields and make unexpected arguments
fail rather than silently returning alpha.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: cc1c11eb-b89e-4928-8653-c21aeb369ac0

📥 Commits

Reviewing files that changed from the base of the PR and between 3d15a73 and a1af774.

📒 Files selected for processing (2)
  • ci/source-architecture-budget.json
  • src/lib/actions/sandbox/rebuild-recreate-journal.test.ts

Drive the rebuild pipeline from persisted creating, created,
registry_committing, and completed journals. Assert a restart accepts
only a ready replacement whose identity and generation match the
journal, and that a foreign same-name sandbox stops the rebuild before
any destructive mutation.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
…journal

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@cjagwani

Copy link
Copy Markdown
Collaborator

Babysitting status for exact head e8e8a28 (plain comment; no Changes Requested review from me): the current deterministic gate reports failed E2E / PR Gate and coordination, and the branch is based on 3aea38c rather than current main da1b103. Maintainer edits are disabled. This large state/rebuild patch needs a refreshed quiet head and clean exact-head protected evidence before approval; I will re-gate it when available.

…dy proved

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
@cjagwani

Copy link
Copy Markdown
Collaborator

Correction to my prior handoff: conflict-free base refreshes are explicitly waived. Please do not merge main solely for base currency; preserving exact-head evidence is preferred unless GitHub reports a real conflict or reviewed behavior requires a change. The substantive blocker or missing evidence described in the earlier handoff remains, but base age by itself is not a blocker. This is a plain coordination comment, not Changes Requested.

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

@senthilr-nv

Copy link
Copy Markdown
Collaborator

Security review for exact head dfb14639cc9c87e48f5ab30c694f775949c52e0e against base 57f73a509cd7e950e01af1423b2514ef352bec82.

This is an agent review by Codex Desktop, not independent human sensitive-path approval.

Result: PASS — no actionable security finding in the complete 48-file diff.

Category Result Evidence
Secrets and credentials PASS The journal stores non-secret target fingerprints and hashed OpenShell identities; credentials and temporary handoffs are excluded. Gitleaks passed.
Input validation and injection PASS Sandbox observations use argument arrays, bounded stable IDs, strict absence classification, and fail closed on ambiguous output.
Authentication and authorization PASS Delete/recreate effects are bound to checkpointed gateway authority and revalidated before mutation.
Dependencies and supply chain PASS No dependency or workflow files changed.
Logging and error handling PASS Diagnostics expose journal identifiers/phases but not credential material or raw OpenShell IDs; destructive ambiguity preserves recovery state.
Cryptography PASS SHA-256 fingerprints and randomUUID are used for identity binding and transaction IDs.
Configuration security PASS No security defaults, policy presets, or headers were weakened.
Security tests PASS Regression tests cover gateway/source/target drift, ambiguous absence, identity mismatch, restart phases, delete-edge ordering, and TOCTOU behavior.
System security PASS Exact gateway scoping and source/target identity proofs prevent sibling-gateway or same-name sandbox confusion at destructive boundaries.

Product scope is approved independently of mergeStateStatus: maintainer-authored issues #7734 and #7735 define the required behavior, security constraints, acceptance tests, and validation, building on maintainer approval in #6492 for the durable recreate-journal slice.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/helpers/rebuild-flow-harness.ts (1)

338-349: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extract one shared resolveGatewayTeardownAuthority test fixture. All three sites hardcode the identical fixed GatewayOwner result (mode: "nemoclaw-managed", source: "standalone", null endpoint/stateDir/supervisor, empty requiredCapabilities). The shared root cause is the lack of one canonical test fixture for this authority contract, so a future field addition or rename needs three synchronized edits.

  • test/helpers/rebuild-flow-harness.ts#L338-L349: replace the inline vi.spyOn(...).mockImplementation(...) object with a call to a new shared fixture helper (for example exported from a common test-helpers module) that returns this standalone-authority shape.
  • src/lib/actions/sandbox/rebuild-resume-snapshot.test.ts#L95-L106: use the same shared fixture helper instead of repeating the literal.
  • test/helpers/onboard-script-mocks.cjs#L162-L176: keep mockStandaloneGatewayTeardownAuthority as the CommonJS entry point, but have it source its returned shape from the same canonical fixture definition (or vice versa) so all three stay in sync.
♻️ Example shared fixture
// test/helpers/gateway-teardown-authority-fixture.ts
export function standaloneGatewayTeardownAuthority({
  gatewayName,
  gatewayPort,
}: {
  gatewayName: string;
  gatewayPort: number;
}) {
  return {
    gatewayName,
    gatewayPort,
    mode: "nemoclaw-managed" as const,
    source: "standalone" as const,
    endpoint: null,
    stateDir: null,
    supervisor: null,
    requiredCapabilities: [],
  };
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/helpers/rebuild-flow-harness.ts` around lines 338 - 349, Extract one
canonical standalone gateway teardown authority fixture and reuse it at all
three sites: update test/helpers/rebuild-flow-harness.ts lines 338-349 and
src/lib/actions/sandbox/rebuild-resume-snapshot.test.ts lines 95-106 to call the
shared helper, and update test/helpers/onboard-script-mocks.cjs lines 162-176 so
mockStandaloneGatewayTeardownAuthority sources the same definition while
remaining the CommonJS entry point. Preserve the existing gateway name/port
inputs and authority fields.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/helpers/rebuild-flow-harness.ts`:
- Around line 338-349: Extract one canonical standalone gateway teardown
authority fixture and reuse it at all three sites: update
test/helpers/rebuild-flow-harness.ts lines 338-349 and
src/lib/actions/sandbox/rebuild-resume-snapshot.test.ts lines 95-106 to call the
shared helper, and update test/helpers/onboard-script-mocks.cjs lines 162-176 so
mockStandaloneGatewayTeardownAuthority sources the same definition while
remaining the CommonJS entry point. Preserve the existing gateway name/port
inputs and authority fields.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 38d70202-5657-4c81-b0e7-c7b2321e0059

📥 Commits

Reviewing files that changed from the base of the PR and between 09810f1 and dfb1463.

📒 Files selected for processing (14)
  • docs/manage-sandboxes/recover-rebuild-sandboxes.mdx
  • docs/reference/commands.mdx
  • src/lib/actions/sandbox/rebuild-destroy-phase.test.ts
  • src/lib/actions/sandbox/rebuild-recreate-phase.ts
  • src/lib/actions/sandbox/rebuild-recreate-reasoning.test.ts
  • src/lib/actions/sandbox/rebuild-resume-snapshot.test.ts
  • test/helpers/onboard-script-mocks.cjs
  • test/helpers/rebuild-flow-harness.ts
  • test/helpers/rebuild-flow-lifecycle-cases.ts
  • test/helpers/rebuild-flow-test-harness.ts
  • test/onboard-installer-restore-intent.test.ts
  • test/onboard-reservation-recreate.test.ts
  • test/onboard-sandbox-build.test.ts
  • test/onboard-sandbox-recreation.test.ts
🚧 Files skipped from review as they are similar to previous changes (8)
  • test/helpers/rebuild-flow-lifecycle-cases.ts
  • src/lib/actions/sandbox/rebuild-recreate-phase.ts
  • test/onboard-sandbox-build.test.ts
  • test/onboard-reservation-recreate.test.ts
  • test/helpers/rebuild-flow-test-harness.ts
  • test/onboard-installer-restore-intent.test.ts
  • test/onboard-sandbox-recreation.test.ts
  • src/lib/actions/sandbox/rebuild-destroy-phase.test.ts

@senthilr-nv

Copy link
Copy Markdown
Collaborator

Exact-head maintainer preparation receipt

  • Head: dbd695bde46dd2c9041fe177649e44457c7d4454
  • Base: 1e5700085b3f5bd768ca8d1c065689ceb07c6434
  • Tree: a6e909f419dd476093a27bd894a37b961a3f4e18

Security specialist review: PASS across all nine repository categories, with no actionable findings. The exact-head security-focused run passed 114/114 tests across four files. The review covered the complete 48-file diff, including exact-gateway authority, strict presence/absence classification, source identity and target fingerprint binding, monotonic journal recovery, delete-edge revalidation, redacted errors, and adversarial fail-closed tests. This was an independent Codex Desktop agent review; it is not human or sensitive-path human approval.

Product scope approval is recorded independently of GitHub mergeStateStatus: accepted maintainer-authored issues #7734 and #7735 define the supported same-name replacement recovery contract and ownership, and maintainer direction in #6492 establishes lifecycle recovery as supported NemoClaw behavior. The PR adds no third-party integration, custom image, or solution recipe.

Automated-review dispositions carried onto this exact tree:

  • CodeRabbit suggested extracting a shared standalone gateway-authority fixture across TypeScript and CommonJS tests. Disposition: non-actionable maintainability preference. The fixtures exercise independent loader surfaces; sharing them would add cross-module loader coupling without changing behavior or security coverage.
  • A Nemotron artifact claimed rebuild recreation fails to record the created identity and that the machine handler skips the created phase. Disposition: false positive. The inner createSandbox path calls recreateRuntime.recordCreated() before registration, and the same transaction then advances through registry_committing and completed before clearing.
  • The artifact also warned that rebuild/onboard ownership was undocumented. Disposition: no action. Ownership is explicit in the call paths, comments, recovery diagnostics, focused tests, and the two updated user-facing documentation pages; duplicating prose in implementation would not protect another contract.

All previously published review threads were resolved. Fresh automated advisors and exact-pair CI/E2E remain merge gates for this head/base pair.

@senthilr-nv senthilr-nv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed Files changed for exact head 12a41cc against base 299050f. No blocking findings. Approve, subject to required CI and exact-pair E2E gates passing.

@senthilr-nv

Copy link
Copy Markdown
Collaborator

Exact-head maintainer preparation receipt

  • Head: 3e6c3d0391f9fd7e58ce9775871807b9b3c2ff8e
  • Base: 299050fc0563db0cd3a803298a6fb986dcfbb745
  • Tree: 7b7c963b5666f68186e9be6527f3a47704eaac12
  • Base-to-head binary patch SHA-256: 43160e2036016d574fcab0ca48fccb4509a2e9acd8d019017e6010a5035bb080

Security specialist review: PASS across all nine repository categories, with no actionable findings. The exact-head security-focused run passed 114/114 tests across four files. The complete exact-head focused run passed 405/405 tests across 27 files. The review covered the complete 48-file diff, including exact-gateway authority, strict presence/absence classification, source identity and target fingerprint binding, monotonic journal recovery, delete-edge revalidation, redacted errors, and adversarial fail-closed tests. This was an independent Codex Desktop agent review; it is not human or sensitive-path human approval.

Product scope approval is recorded independently of GitHub mergeStateStatus: accepted maintainer-authored issues #7734 and #7735 define the supported same-name replacement recovery contract and ownership, and maintainer direction in #6492 establishes lifecycle recovery as supported NemoClaw behavior. The PR adds no third-party integration, custom image, or solution recipe.

Automated-review dispositions carried onto this exact tree:

  • CodeRabbit suggested extracting a shared standalone gateway-authority fixture across TypeScript and CommonJS tests. Disposition: non-actionable maintainability preference. The fixtures exercise independent loader surfaces; sharing them would add cross-module loader coupling without changing behavior or security coverage.
  • A Nemotron artifact claimed rebuild recreation fails to record the created identity and that the machine handler skips the created phase. Disposition: false positive. The inner createSandbox path calls recreateRuntime.recordCreated() before registration, and the same transaction then advances through registry_committing and completed before clearing.
  • The artifact also warned that rebuild/onboard ownership was undocumented. Disposition: no action. Ownership is explicit in the call paths, comments, recovery diagnostics, focused tests, and the two updated user-facing documentation pages; duplicating prose in implementation would not protect another contract.

All previously published review threads were resolved. Fresh automated advisors and exact-pair CI/E2E remain merge gates for this head/base pair.

@senthilr-nv
senthilr-nv merged commit 90f7391 into main Jul 31, 2026
67 of 68 checks passed
@senthilr-nv
senthilr-nv deleted the fix/rebuild-recreate-journal branch July 31, 2026 20:05
@senthilr-nv senthilr-nv mentioned this pull request Aug 1, 2026
23 tasks
senthilr-nv added a commit that referenced this pull request Aug 1, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Adds the canonical dated changelog entry for `v0.0.100` so the
maintainer release plan can verify the pre-tag documentation
prerequisite. The entry summarizes the user-facing changes merged since
`v0.0.99` and links to the relevant guides.

## Changes

- Add `docs/changelog/2026-07-31.mdx` with the exact `## v0.0.100`
heading.
- Cover restored OpenClaw pairing, transactional replacement, Deep
Agents Code, onboarding recovery, lifecycle cleanup, Hermes builds, host
provenance, documentation, and trusted E2E evidence.
- Distinguish active Docker and Kubernetes runtime-bundle enforcement
from the still-inactive managed shared-state transaction foundation.

## Source Coverage

The release entry maps the doc-impacting merged PRs in the
`v0.0.99..main` release range to `docs/changelog/2026-07-31.mdx`: #8021,
#8024, #7973, #8028, #7947, #7788, #7884, #8023, #7969, #8020, #7989,
#8000, #7907, #7942, #7567, #8013, #7955, #8017, #8014, #8015, #7629,
#7644, #7821, #7971, and #7991.

PR #7974 was reviewed after the final rebase and excluded because it
changes internal maintainer-skill attribution policy and tests only; it
does not change a user-facing product or documentation surface.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [x] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [ ] Tests added or updated for changed behavior
- [x] Existing tests cover changed behavior — justification: the
changelog contract test validates the dated entry, version heading, SPDX
form, and route constraints.
- [ ] Tests not applicable — justification:
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Documentation Writer Review

- [x] Documentation writer subagent reviewed the completed changes
- Result: `docs-updated`
- Evidence: `docs/changelog/2026-07-31.mdx`; exact-head review passed
for `6093f44f`; writing rules and documentation style reviewed; `npx
vitest run test/changelog-docs.test.ts` passed 6/6; `npm run docs`
passed with zero Fern errors and two generic Fern upgrade notices.
- Agent: Codex Desktop
<!-- docs-review-head-sha: 6093f44 -->
<!-- docs-review-agents-blob-sha: 3dd7c24 -->

## DGX Station Hardware Evidence

- [ ] Tested on DGX Station
- Tested commit: Not applicable; no DGX Station host script changed.
- Station profile/scenario: Not applicable.
- Result: Not applicable.
- Supporting evidence: Not applicable.

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run validate:pr` passed after refreshing `origin/main` when hooks
were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: `npx
vitest run test/changelog-docs.test.ts` passed 6/6 at `6093f44f`.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result: Not applicable to a dated
prose-only release entry.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) —
validation passed with zero errors; Fern emitted two generic upgrade
notices.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)
— the changelog entry has the required parser-safe MDX SPDX header;
dated changelog entries intentionally do not use page frontmatter.

---

Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
  * Added release notes for v0.0.100.
* Documented improvements to restore pairing, sandbox replacement,
onboarding recovery, lifecycle cleanup, runtime handling, build support,
host readiness, and end-to-end validation.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
senthilr-nv added a commit that referenced this pull request Aug 1, 2026
## Summary

Rebuild now removes the obsolete owned sandbox image only after the
same-name replacement is registered and its exact journaled generation
and live identity are proven. Interrupted resume preserves shared-image
ownership state, so cleanup retains shared images. This restores the
documented cleanup behavior without weakening the recreate journal added
by #7788.

## Changes

- Capture the source registry entry before journaled same-name
replacement.
- Persist the minimal secret-free source-workload cleanup receipt so
interrupted replacements resume cleanup against the original image.
- Preserve the source workload's sharing state through checkpoint
validation and resume cleanup.
- Reload the same recreate transaction after replacement creation and
bind cleanup to its exact recorded live identity.
- Record post-create transaction and cleanup failures as repair failures
while preserving the original error.
- Remove only a provider-owned obsolete image after the replacement
generation and live identity are proven.
- Retain shared images, reused images, foreign generations, and
workloads whose ownership cannot be proven.
- Use the active launcher name in cleanup guidance, including
`nemohermes gc`.
- Add lifecycle-boundary regressions for removal, reuse, sharing,
missing identity, foreign generation, and journal ordering.

Product scope: this restores the existing supported and documented
`rebuild` image-cleanup contract; it does not add a new integration,
configuration, or product surface.

## Type of Change

- [x] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [ ] Docs updated for user-facing behavior changes
- [x] Docs not applicable — justification: Existing
`docs/reference/commands.mdx` already documents automatic rebuild image
cleanup and the `gc` recovery path; this change restores that contract.
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [x] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification: Final exact-head
nine-category review PASS:
#8039 (comment)
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Documentation Writer Review

- [x] Documentation writer subagent reviewed the completed changes
- Result: `no-docs-needed`
- Evidence: Existing
`docs/manage-sandboxes/recover-rebuild-sandboxes.mdx:172-195` already
requires matching live identity and registry generation and fails closed
on identity mismatch; `docs/reference/commands.mdx:284-287` routes
interrupted replacements to that contract; exact-head writer review
found no command, configuration, output, or workflow change; focused
tests passed 98/98, and `git diff --check` passed.
- Agent: Codex Desktop
<!-- docs-review-head-sha: 7ab0927 -->
<!-- docs-review-agents-blob-sha: 3dd7c24 -->

## DGX Station Hardware Evidence

- [ ] Tested on DGX Station
- Tested commit: Not applicable
- Station profile/scenario: Not applicable
- Result: Not applicable
- Supporting evidence: Not applicable

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run validate:pr` passed after refreshing `origin/main` when hooks
were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification:
Focused tests passed 98/98 at the exact final tree; regressions cover
shared-state loss, same-generation replacement identity mismatch, and
post-create cleanup failure recording; independent exact-head writer
review passed.
- [x] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result: Not a broad runtime or
harness change; `npm run validate:pr` passed repository checks, CLI
type-checking, commitlint, secret scanning, and all applicable hooks.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

---

Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>

---------

Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery bug-fix PR fixes a bug or regression

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(onboard): journal non-resumed same-name recreation fix(rebuild): journal same-name sandbox replacement

4 participants