Skip to content

feat(release): preflight E2E evidence in parallel - #7655

Merged
cv merged 3 commits into
mainfrom
codex/release-preflight-ledger
Jul 27, 2026
Merged

feat(release): preflight E2E evidence in parallel#7655
cv merged 3 commits into
mainfrom
codex/release-preflight-ledger

Conversation

@cv

@cv cv commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

Release preparation now derives the candidate E2E denominator before dispatch and runs independent default and explicit-only coverage concurrently. It preserves each matrix execution across runs and attempts, gates conditional hardware dispatch on authoritative availability, and keeps the final release decision bound to the latest origin/main candidate.

Changes

  • Add a read-only release evidence helper that derives required executions from the candidate workflow and builds a SHA-bound, attempt-aware ledger from workflow-produced dispatch receipts. The cut-release skill is the current consumer; a copied test table is insufficient because workflow matrices and run attempts change, and test/release-e2e-evidence.test.ts protects the contract.
  • Dispatch the ordinary default suite and unconditional explicit-only jobs concurrently, while treating Jetson as conditional on runner inventory and Brev qualification readiness as an independent exception path.
  • Update the cut-release, evening, E2E, and release-policy guidance so late merges trigger one final candidate refresh instead of freezing main during preflight work.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: No user-facing docs/ source changes; the canonical internal maintainer skills and release policy are updated in this PR.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: The helper is read-only and derives coverage only from a workflow-produced receipt bound to the candidate, run, attempt, and selector inputs. Workflow and artifact boundary validators pin the receipt producer and upload contract; focused tests cover candidate binding, matrix identity, terminal state, attempts, skipped and malformed evidence, partial reruns, and conditional dispatch.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: Internal release-maintainer tooling, skills, and policy guidance are the correct documentation surface; no user-facing docs/ source requires an update. The reviewer verified latest-existing receipt discovery for partial reruns, manifest path consistency, exact-Brev validation ownership, SHA-drift reruns, and writing and policy consistency.
  • Agent: Codex Desktop documentation writer subagent

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: npm run test:changed — 30 files and 299 tests passed; focused release and policy integration — 3 files and 27 tests passed; workflow and artifact contracts — 2 files and 57 tests passed; source-shape and test-size budgets passed.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

  • New Features
    • Added release E2E evidence preflight + ledger generation via npm run release:e2e-evidence.
    • Introduced coordinated “coverage” dispatch groups with correlation tracking and concurrent execution.
    • Added readiness-aware protected-qualification behavior and improved evidence/receipt collection for validation.
  • Documentation
    • Updated release procedures for readiness gating, itemized exceptions when protected readiness is disabled, and immediate SHA-drift handling.
  • Tests
    • Expanded ledger/evidence validation (green vs missing, aggregation, and rejection of in-progress/skipped or wrong-candidate evidence).
    • Added E2E release-gate contract validation for trusted dispatch receipt drift.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv cv self-assigned this Jul 27, 2026
@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The release process now performs E2E preflight planning, readiness-based dispatch, candidate-SHA validation, trusted receipt collection, and attempt-aware evidence ledger generation. Release guidance, orchestration, CLI tooling, workflow validation, and tests are updated.

Changes

Release E2E qualification

Layer / File(s) Summary
Preflight and ledger engine
.agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts, package.json
Adds the release:e2e-evidence CLI for matrix expansion, dispatch planning, manifest validation, checkout verification, and green/missing ledger generation.
Release preflight and confirmation policy
.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md, .agents/skills/nemoclaw-maintainer-evening/SKILL.md, .agents/skills/nemoclaw-maintainer-policies/references/*
Documents read-only preflight, readiness-dependent full-mode behavior, concurrent evidence dispatch, candidate drift handling, manifest collection, and final SHA checks.
Release coverage dispatch integration
.agents/skills/nemoclaw-maintainer-e2e/SKILL.md
Adds grouped dispatch sequencing, Jetson runner gating, correlation checks, all-attempt job collection, and reruns when the candidate SHA changes.
Trusted dispatch receipt contract
.github/workflows/e2e.yaml, tools/e2e/*
Records workflow dispatch metadata as an artifact and validates its workflow and upload contracts.
Evidence and policy validation
test/release-e2e-evidence.test.ts, test/maintainer-skills-policy.test.ts, test/e2e-release-gate-workflow.test.ts
Tests dispatch plans, matrix execution preservation, multi-attempt evidence, failure states, SHA mismatches, receipt contracts, and updated policy requirements.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Maintainer
  participant ReleaseE2eEvidence
  participant MaintainerE2E
  participant GitHubActions
  Maintainer->>ReleaseE2eEvidence: build candidate-SHA preflight
  ReleaseE2eEvidence-->>Maintainer: return dispatch groups and expected executions
  Maintainer->>MaintainerE2E: dispatch release coverage group
  MaintainerE2E->>GitHubActions: dispatch default and explicit lanes
  GitHubActions-->>MaintainerE2E: return receipts and job attempts
  Maintainer->>ReleaseE2eEvidence: build ledger from manifest
  ReleaseE2eEvidence-->>Maintainer: return green and missing execution counts
Loading

Possibly related PRs

  • NVIDIA/NemoClaw#7496: Implements related exact pre-tag qualification logic, including trusted receipts, SHA-locked evidence, readiness gating, and Brev Launchable qualification.

Suggested labels: area: docs

Suggested reviewers: brandonpelfrey, ahunnargikar-nvidia, dnandakumar-nv

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately captures the main change: release E2E evidence preflight with parallel execution.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/release-preflight-ledger

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

@github-code-quality

github-code-quality Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 0173d70 in the codex/release-prefli... branch remains at 96%, unchanged from commit f6b0c98 in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 0173d70 in the codex/release-prefli... branch remains at 80%, unchanged from commit f6b0c98 in the main branch.

Show a code coverage summary of the most impacted files.
File main f6b0c98 codex/release-prefli... 0173d70 +/-
src/lib/actions...time-command.ts 100% 82% -18%
src/lib/actions...-add-restart.ts 19% 10% -9%
src/lib/actions...on-readiness.ts 100% 91% -9%
src/lib/actions...lution-probe.ts 95% 88% -7%
src/lib/actions...x/mcp-bridge.ts 41% 35% -6%
src/lib/actions...e-validation.ts 84% 81% -3%
src/lib/onboard...shboard-port.ts 93% 90% -3%
src/lib/actions...dbox/destroy.ts 95% 93% -2%
src/lib/onboard...eway-service.ts 82% 81% -1%
src/lib/domain/.../connect-env.ts 89% 97% +8%

Updated July 27, 2026 21:36 UTC

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (3)
test/release-e2e-evidence.test.ts (1)

78-83: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Avoid asserting selector ordering.

parallelExplicit.jobs represents selected jobs, not an ordered behavior. Compare sets (or sorted arrays) so harmless workflow/inventory reordering does not fail this test.

Proposed fix
-    expect(plan.dispatches.parallelExplicit.jobs.split(",")).toEqual([
+    expect(new Set(plan.dispatches.parallelExplicit.jobs.split(","))).toEqual(new Set([
       "openshell-gateway-auth-contract",
       "mcp-bridge-dev",
       "hermes-gpu-startup",
       "sandbox-rlimits-connect",
-    ]);
+    ]));
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/release-e2e-evidence.test.ts` around lines 78 - 83, Update the assertion
for parallelExplicit.jobs in the release evidence test to compare the selected
job names order-independently, such as by sorting both arrays or comparing sets.
Preserve validation of the same four job identifiers without requiring their
declaration order.

Source: Path instructions

.agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts (1)

258-264: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Fragile job-classification via whole-object string search.

isQualificationJob/requiresConfirmedJetsonRunner stringify the entire job definition and substring-search for input names, rather than inspecting the specific field (e.g. job.if) that actually gates dispatch. A step name, comment, or unrelated script line mentioning include_staging_brev_launchable/allow_jetson_runner_queue would falsely match. The qualification-job case is guarded by the length !== 1 check (Line 284-288), but conditionalJobs has no equivalent safety net — a false match there would silently misclassify a job into the wrong dispatch group.

Consider inspecting job.if (or another specific field) directly instead of stringifying the whole job.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
@.agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts
around lines 258 - 264, Update isQualificationJob and
requiresConfirmedJetsonRunner to inspect the job.if dispatch condition directly,
rather than JSON.stringify(job), and match the relevant input names only within
that field. Preserve the existing classification behavior while preventing
matches from step names, comments, or unrelated job content.
.agents/skills/nemoclaw-maintainer-e2e/SKILL.md (1)

199-206: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Possible duplicate job-attempt collection with cut-release-tag/SKILL.md.

This block paginates jobs?filter=all into jobs-all-$RUN_ID.json per run "for the matrix-preserving ledger." cut-release-tag/SKILL.md (Lines 183-192) separately runs the identical gh api --paginate --slurp ... jobs?filter=all ... call for "every accepted default, explicit, and conditional run" to build jobs-$RUN_ID.json for its manifest. If both steps run against the same release-coverage-group runs, this doubles the paginated GitHub API calls per run and creates two evidence files with overlapping purpose but no clear single source of truth for the ledger.

Please clarify whether this file's jobs-all-$RUN_ID.json output is meant to be handed back to cut-release-tag (avoiding its own re-fetch), or whether the two collection points are intentionally independent for different callers.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.agents/skills/nemoclaw-maintainer-e2e/SKILL.md around lines 199 - 206,
Clarify the ownership of the paginated jobs collection between this
release-coverage flow and cut-release-tag: either make jobs-all-$RUN_ID.json the
shared input handed to cut-release-tag so it reuses the existing response
instead of fetching jobs?filter=all again, or explicitly document why both
collections are independent and preserve distinct evidence purposes. Update the
surrounding instructions and filenames/references consistently.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
@.agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts:
- Around line 299-312: Exclude the special-cased "live" job from defaultJobIds
alongside "shared-e2e" and explicit jobs. Update the filter in the executions
construction so jobExecutions("live", ...) remains the sole source of live
executions and execution IDs stay unique.

In @.agents/skills/nemoclaw-maintainer-policies/references/release-train.md:
- Around line 54-62: Update the protected qualification trigger in the
release-train policy so full-mode nemoclaw-maintainer-e2e dispatch occurs
whenever no applicable exact Brev Launchable evidence exists for the candidate
SHA, rather than merely when no full-mode run exists. Keep the existing
requirements for the successful Exact staging Brev Launchable job and its
receipts unchanged.

---

Nitpick comments:
In
@.agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts:
- Around line 258-264: Update isQualificationJob and
requiresConfirmedJetsonRunner to inspect the job.if dispatch condition directly,
rather than JSON.stringify(job), and match the relevant input names only within
that field. Preserve the existing classification behavior while preventing
matches from step names, comments, or unrelated job content.

In @.agents/skills/nemoclaw-maintainer-e2e/SKILL.md:
- Around line 199-206: Clarify the ownership of the paginated jobs collection
between this release-coverage flow and cut-release-tag: either make
jobs-all-$RUN_ID.json the shared input handed to cut-release-tag so it reuses
the existing response instead of fetching jobs?filter=all again, or explicitly
document why both collections are independent and preserve distinct evidence
purposes. Update the surrounding instructions and filenames/references
consistently.

In `@test/release-e2e-evidence.test.ts`:
- Around line 78-83: Update the assertion for parallelExplicit.jobs in the
release evidence test to compare the selected job names order-independently,
such as by sorting both arrays or comparing sets. Preserve validation of the
same four job identifiers without requiring their declaration order.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a01a45b8-4d0c-4ca0-8e23-4ef2d3eeafb2

📥 Commits

Reviewing files that changed from the base of the PR and between 962b283 and 4de43ac.

📒 Files selected for processing (9)
  • .agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md
  • .agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts
  • .agents/skills/nemoclaw-maintainer-e2e/SKILL.md
  • .agents/skills/nemoclaw-maintainer-evening/SKILL.md
  • .agents/skills/nemoclaw-maintainer-policies/references/daily-flow.md
  • .agents/skills/nemoclaw-maintainer-policies/references/release-train.md
  • package.json
  • test/maintainer-skills-policy.test.ts
  • test/release-e2e-evidence.test.ts

Comment on lines +299 to +312
const defaultJobIds = inventory.workflowJobs.filter(
(jobId) => jobId !== "shared-e2e" && !explicitJobs.has(jobId),
);
const executions = [
...defaultJobIds.flatMap((jobId) =>
jobExecutions(jobId, record(jobs[jobId], `workflow.jobs.${jobId}`), "default", plan),
),
...jobExecutions("live", record(jobs.live, "workflow.jobs.live"), "default", plan),
...jobExecutions(
"shared-e2e",
record(jobs["shared-e2e"], "workflow.jobs.shared-e2e"),
"default",
plan,
),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🔴 Critical | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Locate the relevant symbols and files.
git ls-files | rg 'release-e2e-evidence\.mts$|free-standing|workflowJobs|readFreeStandingJobsInventory|buildE2eWorkflowPlan'

# Map the target file first.
ast-grep outline .agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts --view expanded || true

# Search for the inventory type/source and the job list construction.
rg -n "workflowJobs|readFreeStandingJobsInventory|FreeStandingJobsInventory|shared-e2e|live" .agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts

Repository: NVIDIA/NemoClaw

Length of output: 3292


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the inventory source and the preflight job construction.
ast-grep outline tools/e2e/workflow-boundary.mts --view expanded || true
rg -n "readFreeStandingJobsInventory|workflowJobs|shared-e2e|live" tools/e2e/workflow-boundary.mts .agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts

# Read the relevant sections with line numbers.
sed -n '1,260p' tools/e2e/workflow-boundary.mts
sed -n '266,320p' .agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts

Repository: NVIDIA/NemoClaw

Length of output: 50371


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Read the inventory derivation and the preflight assembly around the relevant lines.
sed -n '409,510p' tools/e2e/workflow-boundary.mts
printf '\n---\n'
sed -n '270,315p' .agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts

Repository: NVIDIA/NemoClaw

Length of output: 5646


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Find the live job definition and its free-standing markers.
rg -n -A20 -B8 '^\s*live:' .github/workflows/e2e.yaml
printf '\n---\n'
rg -n -A6 -B6 'E2E_JOB|E2E_TARGET_ID|E2E_DEFAULT_ENABLED' .github/workflows/e2e.yaml

Repository: NVIDIA/NemoClaw

Length of output: 50371


Exclude live from the generic default job list.
inventory.workflowJobs includes live (it carries E2E_JOB=1), so this filter and the later explicit jobExecutions("live", ...) both add it. That creates duplicate execution IDs and hits the uniqueness check, breaking default preflight generation. Add jobId !== "live" here, or derive the default list from jobs that are neither special-cased nor explicitly appended.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
@.agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts
around lines 299 - 312, Exclude the special-cased "live" job from defaultJobIds
alongside "shared-e2e" and explicit jobs. Update the filter in the executions
construction so jobExecutions("live", ...) remains the sole source of live
executions and execution IDs stay unique.

@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 7 warnings · 2 suggestions
  • Model comparison: normalized findings differ; normalized E2E selections differ; Nemotron reported the same number of blockers, 7 more warnings, 2 more suggestions.

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-onboard, credential-sanitization, security-posture

1 optional E2E recommendation
  • staging-brev-launchable

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md (2)

194-219: 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Include all required acceptance evidence in manifest.json.

The manifest only links run/job JSON and selector metadata, but full-mode acceptance also requires the trusted dispatch receipt, qualification identity, and cleanup result. Those artifacts cannot be proven from GitHub run/job JSON alone, so --manifest cannot reliably enforce the exact Brev evidence contract.

Proposed manifest extension
     {
       "runJson": "run-123.json",
       "jobsJson": "jobs-123.json",
       "defaultSuiteSelected": true,
-      "selectedJobs": []
+      "selectedJobs": [],
+      "dispatchReceiptJson": "dispatch-123.json",
+      "qualificationJson": "qualification-123.json",
+      "cleanupJson": "cleanup-123.json"
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md around lines 194
- 219, Extend the manifest schema and the instructions around the
release:e2e-evidence workflow to include trusted dispatch receipt, qualification
identity, and cleanup result artifacts alongside run/job and selector metadata.
Ensure the ledger builder validates and records these required
acceptance-evidence fields for full-mode runs, rather than relying solely on
GitHub run/job JSON.

219-232: 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Rerun every candidate-bound coverage group after SHA drift.

The helper binds all runs to the candidate SHA, so a changed origin/main invalidates prior default, explicit, conditional, and qualification evidence—not only full-mode evidence. Regenerating the plan and rebuilding the ledger is insufficient; explicitly rerun preflight and all required dispatch groups, then capture a new manifest before confirmation.

Proposed wording
-If it moved, regenerate the plan and rebuild the ledger for the new SHA;
+If it moved, discard all prior candidate-bound evidence, regenerate the plan,
+rerun preflight and every required dispatch group for the new SHA, capture a
+new manifest, and rebuild the ledger;
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md around lines 219
- 232, Update the SHA-drift handling in the release confirmation procedure so
that, after refreshing origin/main and detecting a changed SHA, it regenerates
the plan, reruns preflight and every required candidate-bound dispatch
group—including default, explicit, conditional, and exact Brev qualification
groups—and captures a new evidence manifest before rebuilding the ledger and
requesting confirmation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/release-e2e-evidence.test.ts`:
- Around line 78-85: Update the dispatch assertion in the release evidence test
to preserve order-independent comparison while also asserting that the actual
jobs list length equals the expected jobs list length. Keep the existing Set
comparison and expected job IDs unchanged, and derive the actual list from
plan.dispatches.parallelExplicit.jobs.

---

Outside diff comments:
In @.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md:
- Around line 194-219: Extend the manifest schema and the instructions around
the release:e2e-evidence workflow to include trusted dispatch receipt,
qualification identity, and cleanup result artifacts alongside run/job and
selector metadata. Ensure the ledger builder validates and records these
required acceptance-evidence fields for full-mode runs, rather than relying
solely on GitHub run/job JSON.
- Around line 219-232: Update the SHA-drift handling in the release confirmation
procedure so that, after refreshing origin/main and detecting a changed SHA, it
regenerates the plan, reruns preflight and every required candidate-bound
dispatch group—including default, explicit, conditional, and exact Brev
qualification groups—and captures a new evidence manifest before rebuilding the
ledger and requesting confirmation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 958d8ae0-8961-4a70-af01-730a64ceedeb

📥 Commits

Reviewing files that changed from the base of the PR and between 4de43ac and 1080449.

📒 Files selected for processing (6)
  • .agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md
  • .agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts
  • .agents/skills/nemoclaw-maintainer-e2e/SKILL.md
  • .agents/skills/nemoclaw-maintainer-policies/references/release-train.md
  • test/maintainer-skills-policy.test.ts
  • test/release-e2e-evidence.test.ts
🚧 Files skipped from review as they are similar to previous changes (4)
  • .agents/skills/nemoclaw-maintainer-policies/references/release-train.md
  • .agents/skills/nemoclaw-maintainer-e2e/SKILL.md
  • test/maintainer-skills-policy.test.ts
  • .agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts

Comment thread test/release-e2e-evidence.test.ts Outdated
@cv cv added the v0.0.97 label Jul 27, 2026
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv
cv merged commit fbfbe68 into main Jul 27, 2026
99 of 103 checks passed
@cv
cv deleted the codex/release-preflight-ledger branch July 27, 2026 21:46
@cjagwani cjagwani added v0.0.98 and removed v0.0.97 labels Jul 28, 2026
@cjagwani cjagwani mentioned this pull request Jul 28, 2026
23 tasks
cv pushed a commit that referenced this pull request Jul 28, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Add the canonical dated changelog entry for NemoClaw v0.0.97 before the
release plan captures `origin/main`.
The entry groups the user-visible and maintainer-facing changes since
v0.0.96 while preserving the Deferred dual-Station status, experimental
runtime-identity boundary, and pending physical IGX validation.

## Changes

- Add `docs/changelog/2026-07-28.mdx` with the parser-safe MDX SPDX
comment and exact `## v0.0.97` heading.
- Summarize the 43 merged PRs in the release range, omitting
internal-only changes from the public entry and linking each grouped
change to its most specific published documentation.
- Keep the experimental Okta reference explicitly opt-in and outside
normal onboarding, keep the two-Station path Deferred, and state that
physical IGX Orin validation remains pending.

### Source summary

- [#7440](#7440),
[#7443](#7443), and
[#7445](#7445) ->
`docs/changelog/2026-07-28.mdx`: Document read-only host readiness
reports and fail-closed platform qualification.
- [#7030](#7030) ->
`docs/changelog/2026-07-28.mdx`: Document the Deferred trusted
two-Station vLLM evaluation.
- [#7265](#7265) ->
`docs/changelog/2026-07-28.mdx`: Document the bounded experimental
direct-runner Okta runtime-identity reference.
- [#7711](#7711) and
[#7648](#7648) ->
`docs/changelog/2026-07-28.mdx`: Document compatible-endpoint reasoning
effort and retired NVIDIA Build model paths.
- [#7746](#7746),
[#7763](#7763), and
[#7681](#7681) ->
`docs/changelog/2026-07-28.mdx`: Document safe compatible-provider
creation, replacement refusal, and narrow OpenShell bridge URL handling.
- [#7641](#7641),
[#7690](#7690),
[#7631](#7631), and
[#7710](#7710) ->
`docs/changelog/2026-07-28.mdx`: Document paused-container recovery,
recreation journaling, pre-mutation uninstall checks, and
source-checkout OpenShell selection.
- [#7624](#7624) and
[#7762](#7762) ->
`docs/changelog/2026-07-28.mdx`: Document Jetson release diagnostics and
bounded render-device group propagation.
- [#7639](#7639),
[#7760](#7760),
[#7721](#7721), and
[#7761](#7761) ->
`docs/changelog/2026-07-28.mdx`: Document Telegram, MCP media-type,
Hermes image-mode, and locked-restart fixes.
- [#7653](#7653) and
[#7680](#7680) ->
`docs/changelog/2026-07-28.mdx`: Document Deep Agents policy tasks and
the bounded Claude Code OAuth path.
- [#7679](#7679) ->
`docs/changelog/2026-07-28.mdx`: Document the checksum-bound libssh2 and
Python HTMLParser backports.
- [#7655](#7655),
[#7651](#7651),
[#7664](#7664),
[#7666](#7666),
[#7670](#7670),
[#7719](#7719), and
[#7741](#7741) ->
`docs/changelog/2026-07-28.mdx`: Document exact candidate E2E evidence,
Launchable selection, diagnostic consolidation, and trusted WSL
validation.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [x] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [ ] Tests added or updated for changed behavior
- [x] Existing tests cover changed behavior — justification:
`test/changelog-docs.test.ts` validates the dated changelog contract,
MDX header, heading uniqueness, and release-entry structure.
- [ ] Tests not applicable — justification:
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Documentation Writer Review

- [x] Documentation writer subagent reviewed the completed changes
- Result: `docs-updated`
- Evidence: The committed `docs/changelog/2026-07-28.mdx` blob exactly
matches the reviewed file. Completeness, factual accuracy, link shape,
parser-safe MDX header, one-sentence-per-line style, `.docs-skip`
compliance, and bounded product claims passed.
- Agent: Codex Desktop documentation writer subagent
<!-- docs-review-head-sha: da6aa27 -->
<!-- docs-review-agents-blob-sha: be20a09 -->

## DGX Station Hardware Evidence

- [ ] Tested on DGX Station
- Tested commit: Not applicable; this PR changes only the dated
changelog.
- Station profile/scenario: Not applicable.
- Result: Not applicable.
- Supporting evidence: Not applicable.

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — `npx vitest run
test/changelog-docs.test.ts` passed 6/6.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — not applicable to this doc-only release
entry.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) —
completed with 0 errors and 2 pre-existing Fern warnings.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)
— native changelog entries use the required parser-safe MDX SPDX comment
and intentionally have no frontmatter.

---
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added improved host readiness reporting and Jetson onboarding
guidance.
* Added controls for reasoning effort with compatible endpoints and
enhanced managed MCP discovery.
  * Improved Deep Agents task publication and preset support.
* **Bug Fixes**
* Hardened provider switching, sandbox recovery, uninstall behavior, and
Telegram connectivity.
* Improved container image integrity checks, media-type handling, and
checksum validation.
  * Enhanced vLLM evaluation behavior and release diagnostics.
* **Documentation**
  * Added the NemoClaw v0.0.97 changelog.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
@wscurran wscurran added area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure feature PR adds or expands user-visible functionality labels Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure feature PR adds or expands user-visible functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants