Skip to content

fix(installer): recover upgrades with user-local OpenShell - #7028

Merged
apurvvkumaria merged 6 commits into
mainfrom
codex/fix-6114-legacy-upgrade
Jul 16, 2026
Merged

fix(installer): recover upgrades with user-local OpenShell#7028
apurvvkumaria merged 6 commits into
mainfrom
codex/fix-6114-legacy-upgrade

Conversation

@cjagwani

@cjagwani cjagwani commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Upgrades from v0.0.55 could miss an existing OpenShell binary installed in ~/.local/bin, skip the pre-upgrade backup, and strand the registered sandbox after the gateway changed. This restores the standard user-local OpenShell path before the backup decision so prepared recovery can preserve and restore the sandbox.

Related Issue

Fixes #6114

Changes

  • Discover an existing user-local OpenShell binary before deciding that no pre-upgrade gateway backup is possible, without overriding an OpenShell already available on PATH.
  • Cover the stale-PATH v0.0.55 installer shape and verify strict backup-all plus restore preparation.
  • Exercise the ordinary interactive v0.0.55 upgrade path without test-only confirmation variables and with ~/.local/bin deliberately absent from PATH.
  • Run the v0.0.55 live upgrade fixture natively on x86_64 and arm64 using its immutable multi-architecture sandbox-base index.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: This restores the documented backup-and-recovery upgrade behavior; it adds no user-facing command, option, or workflow. A documentation review found the existing quickstart, command reference, and sandbox update guidance sufficient.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Independent focused review confirmed the change only restores standard OpenShell discovery and does not relax strict backup, gateway ownership, legacy managed-image provenance, or custom-image fail-closed guards.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification:
    • npx vitest run --project integration test/install-openshell-upgrade-prompt.test.ts test/install-preexisting-sandbox-recovery.test.ts test/install-orphaned-sandbox-recovery.test.ts — 35 passed
    • npx vitest run --project cli src/lib/actions/upgrade-sandboxes-recovery.test.ts src/lib/actions/maintenance.test.ts src/lib/actions/sandbox/stopped-sandbox-backup.test.ts — 75 passed
    • npx vitest run --project e2e-support test/e2e/support/openshell-gateway-upgrade-workflow-boundary.test.ts test/e2e/support/e2e-workflow.test.ts — 39 passed
    • NEMOCLAW_RUN_LIVE_E2E=1 npx vitest run --project e2e-live test/e2e/live/openshell-gateway-upgrade.test.ts — 3 passed; Linux live case skipped locally and covered by the x86_64/arm64 CI matrix
    • npm run source-shape:check — passed
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Charan Jagwani cjagwani@nvidia.com

Summary by CodeRabbit

  • New Features

    • Expanded openshell-gateway-upgrade e2e coverage to run across multiple legacy fixtures, including x86_64 and ARM64, with added workflow-boundary validation for the expected matrix and live-test step.
  • Bug Fixes

    • Improved upgrade preinstall guard to recover an OpenShell binary from user-local locations when it’s not found on standard PATH.
  • Tests

    • Updated and extended e2e and installer-prompt tests to cover interactive vs non-interactive upgrade flows, PATH discovery variations, and fixture-specific environment/log assertions.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@cjagwani cjagwani self-assigned this Jul 16, 2026
@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: fdebe96c-1efa-4b2b-a099-6d38094d37a7

📥 Commits

Reviewing files that changed from the base of the PR and between cc4ce95 and 8e20e19.

📒 Files selected for processing (1)
  • scripts/install.sh
🚧 Files skipped from review as they are similar to previous changes (1)
  • scripts/install.sh

📝 Walkthrough

Walkthrough

The installer now restores user-local OpenShell binaries before gateway recovery checks. Upgrade tests cover interactive and non-interactive flows, hidden local binaries, and missing binaries. E2E coverage adds an arm64 legacy fixture and workflow boundary validation.

Changes

Gateway upgrade recovery and validation

Layer / File(s) Summary
OpenShell recovery discovery and guard tests
scripts/install.sh, test/install-openshell-upgrade-prompt.test.ts
The installer attempts user-local OpenShell recovery when it is absent from PATH, while tests cover PATH, user-local, and missing-binary scenarios.
Interactive upgrade execution
test/e2e/live/openshell-gateway-upgrade-helpers.ts, test/e2e/live/openshell-gateway-upgrade.test.ts, test/e2e/support/openshell-gateway-upgrade-workflow-boundary.test.ts
Installer argument helpers and upgrade tests distinguish interactive current upgrades from non-interactive legacy upgrades and validate prompt behavior.
Cross-architecture regression fixtures
.github/workflows/e2e.yaml, test/e2e/support/openshell-gateway-upgrade-workflow-boundary.test.ts
The legacy gateway-upgrade matrix selects per-fixture runners, adds a v0.0.55 arm64 fixture, and updates the corresponding sandbox image digest.
Workflow boundary validation
tools/e2e/openshell-gateway-upgrade-workflow-boundary.mts, tools/e2e/workflow-boundary.mts, test/e2e/support/openshell-gateway-upgrade-workflow-boundary.test.ts
New validation checks enforce runner selection, pinned v0.0.55 fixtures, and the live Vitest invocation within aggregate E2E workflow validation.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant UpgradeTest
  participant runInstallerPayload
  participant install.sh
  participant OpenShell
  UpgradeTest->>runInstallerPayload: select interactive or non-interactive arguments
  runInstallerPayload->>install.sh: run installer with optional confirmation input
  install.sh->>OpenShell: discover or restore local binary
  OpenShell-->>install.sh: provide executable for recovery
  install.sh-->>runInstallerPayload: complete upgrade preparation
Loading

Possibly related PRs

Suggested labels: area: install, area: e2e, platform: arm64, bug-fix

Suggested reviewers: cv

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Linked Issues check ❓ Inconclusive The changes address the stale-PATH upgrade path, but the summary does not fully verify all sandbox-preservation requirements from #6114. Provide diff details confirming sandbox registration, onboarding suppression, and post-upgrade exec/recovery behavior for existing sandboxes.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately describes the installer recovery fix for upgrades with a user-local OpenShell.
Out of Scope Changes check ✅ Passed The changes stay within upgrade recovery, test coverage, and workflow validation for the reported bug.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-6114-legacy-upgrade

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage remains at 96%, unchanged from the main branch.

TypeScript / code-coverage/cli

The overall coverage in the codex/fix-6114-legac... branch remains at 80%, unchanged from the main branch.

Show a code coverage summary of the most impacted files.
File main d034b7f codex/fix-6114-legac... e55eac2 +/-
src/lib/inferen...lama/process.ts 100% 50% -50%
src/lib/inferen...er-lifecycle.ts 71% 65% -6%

Updated July 16, 2026 17:35 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/e2e.yaml:
- Around line 3531-3539: Update the v0.0.55 fixture in
openshell-gateway-upgrade-workflow-boundary.test.ts to use the workflow’s
sandbox image digest
10433a8cd2f2b809dd0fdf983514679e04c0f8aa1ff5bbff675029046033b108, including the
expected sandboxBaseDigest value, while preserving the existing boundary-test
structure.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ce044bb0-793d-4f06-a82f-b76ca98706dc

📥 Commits

Reviewing files that changed from the base of the PR and between 4168995 and 1148c6b.

📒 Files selected for processing (6)
  • .github/workflows/e2e.yaml
  • scripts/install.sh
  • test/e2e/live/openshell-gateway-upgrade-helpers.ts
  • test/e2e/live/openshell-gateway-upgrade.test.ts
  • test/e2e/support/openshell-gateway-upgrade-workflow-boundary.test.ts
  • test/install-openshell-upgrade-prompt.test.ts

Comment thread .github/workflows/e2e.yaml
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — Informational

Advisor assessment: Informational / high confidence
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions
Status: No actionable findings remain in the canonical review ledger.

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 9 warnings · 0 suggestions
  • Model comparison: normalized findings differ; normalized E2E selections match; Nemotron reported the same number of blockers, 9 more warnings, the same number of suggestions.

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-onboard, credential-sanitization, security-posture, openshell-gateway-upgrade

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@apurvvkumaria
apurvvkumaria enabled auto-merge (squash) July 16, 2026 17:31
@apurvvkumaria apurvvkumaria self-assigned this Jul 16, 2026
@apurvvkumaria apurvvkumaria added bug-fix PR fixes a bug or regression Recommended Blocker Recommended release blocker for maintainer review labels Jul 16, 2026
Co-authored-by: Charan Jagwani <cjagwani@nvidia.com>
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@apurvvkumaria
apurvvkumaria merged commit a423a34 into main Jul 16, 2026
66 of 72 checks passed
@apurvvkumaria
apurvvkumaria deleted the codex/fix-6114-legacy-upgrade branch July 16, 2026 17:56
@cjagwani
cjagwani restored the codex/fix-6114-legacy-upgrade branch July 16, 2026 18:09
@cjagwani
cjagwani deleted the codex/fix-6114-legacy-upgrade branch July 16, 2026 18:09
cv pushed a commit that referenced this pull request Jul 17, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Add the canonical dated changelog entry required before the v0.0.85
release plan can be generated.
The entry summarizes the user-visible OpenShell, DGX Station, inference,
MCP, onboarding, and recovery changes merged since v0.0.84 and links to
their owning guides.

## Changes

- Add `docs/changelog/2026-07-16.mdx` with the exact `## v0.0.85`
heading, parser-safe SPDX comment, release summary, and detailed
bullets.
- Link every documented theme to its most specific published OpenClaw
guide routes.
- Reconcile the release entry with these merged source PRs:
- #6726 -> `docs/changelog/2026-07-16.mdx`: Document the supported
OpenShell v0.0.85 upgrade, immutable consumed artifacts, multiline exec,
credential rewrite diagnostics, and child-process TLS boundary.
- #6986 -> `docs/changelog/2026-07-16.mdx`: Document managed MCP
behavior shared across supported agents.
- #6991 and #7045 -> `docs/changelog/2026-07-16.mdx`: Document qualified
DGX Station host preparation and the interactive-terminal boundary for
`--station-deepseek`.
- #6992, #7001, #7006, and #7044 -> `docs/changelog/2026-07-16.mdx`:
Document managed-model reasoning behavior, safe inference route
mutation, and verified vLLM served aliases.
- #6865, #7010, and #7028 -> `docs/changelog/2026-07-16.mdx`: Document
onboarding DNS recovery, explicit notice acceptance, and upgrades with
user-local OpenShell.
- #7005, #7021, #7029, and #7049 -> `docs/changelog/2026-07-16.mdx`:
Document rebuild backup safety, no-dashboard state, managed gateway
discovery, and Hermes shields topology checks.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [x] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [ ] Tests added or updated for changed behavior
- [x] Existing tests cover changed behavior — justification:
`test/changelog-docs.test.ts` validates the canonical heading,
parser-safe SPDX comment, and detailed entry structure; the docs build
validates published routes.
- [ ] Tests not applicable — justification:
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — `npx vitest run
test/changelog-docs.test.ts` passed 6/6.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — not applicable to this doc-only entry.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) — passed
with 0 errors and 2 pre-existing Fern warnings.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [x] New doc pages include SPDX header and frontmatter (new pages only)
— native changelog entries use the required parser-safe MDX SPDX comment
instead of frontmatter.

---
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
  * Added release notes for NemoClaw v0.0.85.
* Documented improvements to compatibility, credential handling, setup
validation, recovery workflows, endpoint configuration, gateway
discovery, and runtime validation.
  * Added links to relevant user-guide sections.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
cv pushed a commit that referenced this pull request Jul 17, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Follow up #7077 and #7028 by completing direct historical gateway
upgrades. After a strict backup, the installer now retires a running
gateway whenever its installed OpenShell is outside the prepared current
release's supported range, preventing the new CLI from encountering the
old gateway schema during recovery.

## Changes

- Replace the setup-only conditional with an equivalent boolean
invariant assertion.
- Preserve the requirement that the `v0.0.55` fixture records the
original OpenShell directory before hiding it from `PATH`.
- Pin the `v0.0.74` registry contract to both `nemoclawVersion:
"0.0.74"` and `fromDockerfile: null`, as proven by selective live E2E
artifacts.
- Keep earlier historical fixtures pinned to absent registry metadata.
- Resolve the current OpenShell min/max range from the prepared current
source after backup.
- Retire the selected gateway only when the installed OpenShell falls
below or above that range; keep a supported gateway running.
- Fail closed after backup and before gateway retirement when the
supported range is missing, malformed, or inverted.
- Document the expanded gateway-retirement boundary in the quickstart,
command reference, and sandbox-update guide.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [x] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [x] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification: #7028's maintainer
approval established the strict-backup and managed-image recovery
boundary for #6114. This follow-up retains those guards, scopes
retirement to the selected gateway only after backup, preserves
supported gateways, and fails closed on invalid range data.
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — installer recovery integration tests
(38 passed), gateway support tests (5 passed), and final focused
installer test after hook formatting (26 passed)
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result:
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) — passed
with 0 errors; Fern reported the existing unauthenticated-redirect and
accent-contrast warnings
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

---
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Tests**
* Improved end-to-end coverage for OpenShell gateway upgrades by
verifying structured legacy registry metadata (instead of only version
values).
* Expanded upgrade prompt/preinstall guard scenarios to confirm “keep vs
retire” based on the blueprint-supported OpenShell version window,
including invalid/unknown ranges and safer legacy gateway stop behavior.
* **Documentation**
* Updated the quickstart, sandbox update guide, and command reference to
reflect conditional gateway retirement/retention and fail-closed
behavior.
* **Chores**
* Refined the installer’s gateway retirement logic to rely on the
blueprint-defined compatibility range, with a Linux-only PID-validated
fallback to stop the legacy gateway process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-fix PR fixes a bug or regression Recommended Blocker Recommended release blocker for maintainer review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[All Platforms][Upgrade] v0.0.55 → v0.0.76 leaves pre-existing sandboxes stuck in Provisioning/Error — user data inaccessible until manual rebuild

4 participants