feat(installer): add exact Station factory image profiles - #7126
Conversation
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
📝 WalkthroughWalkthroughDGX Station support now recognizes qualified factory profiles, validates BaseOS and AI Developer Tools host runtimes, enforces GB300 GPU checks, preserves expanded Express resume state, and requires detected GB300 devices for Station vLLM containers. ChangesDGX Station factory profiles
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant ExpressInstaller
participant HostPreparation
participant FactoryRuntime
participant ResumeState
ExpressInstaller->>HostPreparation: start Station Express preparation
HostPreparation->>FactoryRuntime: probe Docker and GB300 runtime
FactoryRuntime-->>HostPreparation: return validation and session status
HostPreparation->>ResumeState: persist resume settings when needed
ResumeState-->>ExpressInstaller: provide extended resume command
Possibly related issues
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
PR Review Advisor — InformationalAdvisor assessment: Informational / high confidence Model lanes
Nemotron output stays in workflow artifacts and does not change the assessment above. E2E guidanceAdvisory only. E2E / PR Gate selects and runs jobs independently. Recommended E2E: 1 optional E2E recommendation
1 warning · 0 suggestionsWarningsWarnings do not block.
|
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/prepare-dgx-station-host.sh`:
- Around line 1358-1361: Update the STATION_HOST_PROFILE case handling to
include ai-developer-tools and set expected_driver from the qualified image’s
pinned driver version, ensuring host and container probes reject driver drift.
Add or update the relevant test to verify that a mismatched loaded driver is
rejected.
In `@src/lib/inference/vllm.test.ts`:
- Around line 523-528: Replace the helper-level flag assertion in the Station
GB300 empty-detection test with a public-boundary test through installVllm.
Configure no GB300 GPUs, assert installVllm reports a controlled failure, and
verify dockerRunDetached is not invoked, preserving coverage that startContainer
catches flag-construction errors.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 6b9c9a9b-0f9b-470c-8ba4-b36f71483580
📒 Files selected for processing (8)
scripts/install.shscripts/prepare-dgx-station-host.shsrc/lib/inference/vllm-install-storage.test.tssrc/lib/inference/vllm.test.tssrc/lib/inference/vllm.tstest/install-express-prompt.test.tstest/install-station-dgx-os.test.tstest/install-station-host-preparation.test.ts
| case "$STATION_HOST_PROFILE" in | ||
| generic-ubuntu) expected_driver="$DRIVER_VERSION" ;; | ||
| colossus-baseos) expected_driver="$BASEOS_DRIVER_VERSION" ;; | ||
| esac |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Pin the AI Developer Tools driver.
ai-developer-tools leaves expected_driver empty, so any loaded driver passes the host and container probes. Enforce the qualified image’s driver pin and add a drift-rejection test.
Proposed fix
readonly BASEOS_DRIVER_VERSION="595.58.03"
+readonly AI_DEVELOPER_TOOLS_DRIVER_VERSION="610.43.03" case "$STATION_HOST_PROFILE" in
generic-ubuntu) expected_driver="$DRIVER_VERSION" ;;
colossus-baseos) expected_driver="$BASEOS_DRIVER_VERSION" ;;
+ ai-developer-tools) expected_driver="$AI_DEVELOPER_TOOLS_DRIVER_VERSION" ;;
esac📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| case "$STATION_HOST_PROFILE" in | |
| generic-ubuntu) expected_driver="$DRIVER_VERSION" ;; | |
| colossus-baseos) expected_driver="$BASEOS_DRIVER_VERSION" ;; | |
| esac | |
| case "$STATION_HOST_PROFILE" in | |
| generic-ubuntu) expected_driver="$DRIVER_VERSION" ;; | |
| colossus-baseos) expected_driver="$BASEOS_DRIVER_VERSION" ;; | |
| ai-developer-tools) expected_driver="$AI_DEVELOPER_TOOLS_DRIVER_VERSION" ;; | |
| esac |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/prepare-dgx-station-host.sh` around lines 1358 - 1361, Update the
STATION_HOST_PROFILE case handling to include ai-developer-tools and set
expected_driver from the qualified image’s pinned driver version, ensuring host
and container probes reject driver drift. Add or update the relevant test to
verify that a mismatched loaded driver is rejected.
| it("fails closed instead of exposing all GPUs when Station GB300 detection is empty", () => { | ||
| mocks.getGpuIndicesByName.mockReturnValue([]); | ||
| const profile = detectVllmProfile({ platform: "station", type: "nvidia" }); | ||
|
|
||
| expect(profile).not.toBeNull(); | ||
| expect(() => profile!.buildDockerRunFlags!()).toThrow(/requires an NVIDIA GB300 GPU/); |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Cover the failure through the public install boundary.
This helper-level assertion would still pass if flag construction moved outside startContainer’s catch again. Exercise installVllm with no GB300 and assert a controlled failure plus no dockerRunDetached call.
As per path instructions, “Prefer observable outcomes through the public boundary over source-text, private-shape, or mock-call assertions.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/lib/inference/vllm.test.ts` around lines 523 - 528, Replace the
helper-level flag assertion in the Station GB300 empty-detection test with a
public-boundary test through installVllm. Configure no GB300 GPUs, assert
installVllm reports a controlled failure, and verify dockerRunDetached is not
invoked, preserving coverage that startContainer catches flag-construction
errors.
Source: Path instructions
<!-- markdownlint-disable MD041 --> ## Summary Add the v0.0.87 changelog entry and align the DGX Station, platform-support, and rebuild documentation with behavior merged since v0.0.86. The Station documentation retains the Deferred support status while recording the two exact factory-image qualification profiles and the post-reboot receipt compatibility fix from #7130. ## Changes - Add the v0.0.87 changelog summary, including the merged Station resume receipt fix, with links to the owning documentation pages. - Document the exact April 2026 Colossus BaseOS and June 2026 AI Developer Tools Station identities, validation boundaries, and permitted host preparation. - Synchronize those Station qualification paths into the canonical platform matrix and generated provider/platform pages. - Document how an OpenClaw rebuild clears stale managed-provider session-model pins after an inference switch. ### Source summary - [#7130](#7130) -> `docs/changelog/2026-07-17.mdx`: Document compatibility with current six-field and legacy three-field Station resume receipts after host preparation. - [#7128](#7128) -> `docs/changelog/2026-07-17.mdx`: Document restart-safe managed DCode startup and required Docker resource limits. - [#7126](#7126) -> `docs/changelog/2026-07-17.mdx`, `docs/get-started/dgx-station-preparation.mdx`, `ci/platform-matrix.json`: Document the two bounded Station factory-image qualification profiles without promoting Deferred support and synchronize the generated platform/provider references. - [#6947](#6947) -> `docs/changelog/2026-07-17.mdx`: Document streaming sandbox backup archive creation. - [#7117](#7117) -> `docs/changelog/2026-07-17.mdx`: Document Hermes post-restore gateway and managed MCP health verification. - [#7109](#7109) -> `docs/changelog/2026-07-17.mdx`, `docs/manage-sandboxes/recover-rebuild-sandboxes.mdx`: Document stale managed session-model pin reconciliation after rebuild. - [#7068](#7068) -> `docs/changelog/2026-07-17.mdx`: Document strict-provider compatibility for Hermes tool schemas. - [#6965](#6965) -> `docs/changelog/2026-07-17.mdx`: Document managed vLLM download storage estimation. - [#7114](#7114) -> `docs/changelog/2026-07-17.mdx`: Document preserved, redacted rebuild diagnostics. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [x] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [ ] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [x] Tests not applicable — justification: Documentation-only release-prep update; the changelog, platform-generation contracts, and docs build validate the changed pages and links. - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — `npx vitest run test/generate-platform-docs.test.ts test/station-doc-ownership.test.ts test/changelog-docs.test.ts`: 29 passed; `python3 scripts/generate-platform-docs.py --check`: all generated tables in sync - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [x] `npm run docs` builds without warnings (doc changes only) - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added filesystem-aware managed vLLM storage preflight (cold download sizing; interactive vs non-interactive capacity checks). - Improved tool-schema compatibility for strict OpenAI-compatible providers (including Gemini schema handling) using a strict single envelope. - Enhanced sandbox backup creation with streamed archive generation and incremental entry validation. - **Bug Fixes** - Strengthened rebuild/recovery checks with Hermes sandbox health validation and cleanup of stale managed-provider session pins. - Persisted onboarding startup commands with required `nproc`/`nofile` limits across sandbox recreation. - Improved replacement-image rebuild diagnostics with bounded, redacted output handling. - For OpenCLAW “rebuild while preserving state,” stale model/provider pins are cleared when appropriate. - **Documentation** - Expanded DGX Station GB300 no-OTA factory profile/qualification criteria and clarified managed vLLM provider/sandbox constraints. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Summary
DGX Station Express currently rejects two observed Ubuntu 24.04 factory images because they omit the stock DGX OS OTA identity. This change recognizes only their exact release profiles, preserves the factory driver and package stack, validates or repairs only the bounded runtime prerequisites each profile needs, and fails closed on identity or health drift.
Related Issue
Tracks #7124.
Maintainer review approved this bounded scope in review 4726981533. Full hardware qualification and canonical documentation remain merge blockers. The separate DGX OS 7.5.0 image whose DMI and release identity are
Not Specifiedis intentionally out of scope.Changes
Type of Change
Quality Gates
Verification
Signed-off-by:line and every commit appears asVerifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run check:diffpassed when hooks were skipped or unavailablevitestfocused installer/vLLM suite: 242 passed, 1 skippednpm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes — command/result: pending draft PR CInpm run docsbuilds without warnings (doc changes only)Hardware evidence
v0.0.86pre-fix baseline onpmgb300ws-0044: exact commite7e8b674e0d7fcaa37ad3806d5f39df352e702c4classified the image asunsupported-dgx-osand exited before installation. Normalized before/after host snapshots have the same SHA-256 (6932dde24cf70904dda9112b3bc5aa21ca9dd1edad1474f285231e0a39ae7c21).pmgb300ws-0044(April BaseOS): the exact candidate--checkrecognizedsupported-colossus-baseos, verified package and failed-unit fingerprints, preserved factory packages/driver, and stopped at the expected Docker-login boundary. Apply/resume and full E2E are pending.Signed-off-by: Senthil Ravichandran senthilr@nvidia.com
Summary by CodeRabbit
New Features
Bug Fixes