Skip to content

fix(e2e): preserve protected main dispatch identity - #8570

Merged
jyaunches merged 1 commit into
mainfrom
codex/e2e-main-dispatch-sha
Aug 7, 2026
Merged

fix(e2e): preserve protected main dispatch identity#8570
jyaunches merged 1 commit into
mainfrom
codex/e2e-main-dispatch-sha

Conversation

@jyaunches

@jyaunches jyaunches commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Ordinary main E2E dispatches now provide candidate identity to protected managed-image and DGX Spark qualification without enabling manual-PR risk signals. Before this change, both managed-image architecture jobs failed before qualification because their candidate SHA was empty.

Changes

  • Add lane-specific candidate SHA variables that resolve to checkout_sha for PR runs and github.sha for ordinary main runs.
  • Keep NEMOCLAW_E2E_EXPECTED_SHA bound only to checkout_sha so main runs do not emit manual-PR risk evidence.
  • Validate both contracts in the workflow boundaries and cover the ordinary-main identity path in E2E-support tests.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: This repairs internal E2E candidate identity and does not change public commands, configuration, workflow inputs, defaults, or operator procedures.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Workflow boundary tests preserve the PR-only risk signal and require exact candidate identity for each protected lane.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: Repairs internal E2E candidate-SHA plumbing and its boundary tests; no public CLI, configuration, workflow input, default, or operator procedure changes.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: npm exec -- vitest run --project e2e-support test/e2e/support/managed-image-protected-runtime-workflow.test.ts test/e2e/support/managed-image-multiarch-startup-helpers.test.ts test/e2e/support/llama-cpp-dgx-spark-qualification-workflow.test.ts passed 39 tests.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Julie Yaunches jyaunches@nvidia.com

Summary by CodeRabbit

  • Bug Fixes

    • Improved workflow validation for protected managed-image startup and qualification runs.
    • Ensured jobs consistently use the selected checkout revision, falling back to the workflow commit when needed.
    • Prevented dispatch validation from relying on legacy revision settings.
  • Tests

    • Added end-to-end coverage for checkout revision binding and invalid workflow configurations.

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
@jyaunches jyaunches self-assigned this Aug 7, 2026
@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c0a7ffe8-6c5f-4abe-ba04-1430c6c0abc5

📥 Commits

Reviewing files that changed from the base of the PR and between aae23eb and e99c73b.

📒 Files selected for processing (8)
  • .github/workflows/e2e.yaml
  • test/e2e/live/llama-cpp-dgx-spark-qualification-helpers.ts
  • test/e2e/live/managed-image-multiarch-startup-helpers.ts
  • test/e2e/support/llama-cpp-dgx-spark-qualification-workflow.test.ts
  • test/e2e/support/managed-image-multiarch-startup-helpers.test.ts
  • test/e2e/support/managed-image-protected-runtime-workflow.test.ts
  • tools/e2e/llama-cpp-dgx-spark-qualification-workflow-boundary.mts
  • tools/e2e/managed-image-multiarch-workflow-boundary.mts

📝 Walkthrough

Walkthrough

The workflows now expose job-specific candidate head SHA variables. Runtime helpers consume these variables, and boundary tests verify checkout SHA fallback and explicit managed-image dispatch identity validation.

Changes

Candidate SHA propagation

Layer / File(s) Summary
Workflow SHA bindings and validation
.github/workflows/e2e.yaml, tools/e2e/*workflow-boundary.mts
The protected managed-image and llama.cpp qualification jobs use job-specific head SHA variables with `${{ inputs.checkout_sha
Runtime candidate SHA consumption
test/e2e/live/*helpers.ts, test/e2e/support/managed-image-multiarch-startup-helpers.test.ts
Runtime helpers read the job-specific variables. Managed-image tests use the protected candidate SHA and reject empty values without legacy fallback.
Candidate SHA contract tests
test/e2e/support/llama-cpp-dgx-spark-qualification-workflow.test.ts, test/e2e/support/managed-image-protected-runtime-workflow.test.ts
Boundary tests reject workflow bindings that omit the github.sha fallback.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

  • NVIDIA/NemoClaw#8477: Both changes update candidate SHA bindings and validation for the protected E2E workflows.
  • NVIDIA/NemoClaw#8518: Both changes modify protected E2E workflow SHA bindings and boundary tests.
  • NVIDIA/NemoClaw#8562: Both changes modify DGX Spark qualification workflow validation, but address different concerns.

Suggested labels: v0.0.105

Suggested reviewers: cv

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: preserving protected main dispatch identity in E2E workflows.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/e2e-main-dispatch-sha

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit e99c73b in the codex/e2e-main-dispa... branch remains at 96%, unchanged from commit aae23eb in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit e99c73b in the codex/e2e-main-dispa... branch remains at 81%, unchanged from commit aae23eb in the main branch.

Show a code coverage summary of the most impacted files.
File main aae23eb codex/e2e-main-dispa... e99c73b +/-
src/lib/onboard...flight-ports.ts 67% 33% -34%
src/lib/messagi...flow-planner.ts 91% 91% 0%
src/lib/core/immutable.ts 97% 97% 0%
src/lib/messagi...etup-applier.ts 68% 68% 0%
src/lib/messaging/utils.ts 76% 76% 0%
src/lib/onboard/temp-files.ts 100% 100% 0%
src/lib/sandbox...rce-identity.ts 83% 83% 0%

Updated August 07, 2026 16:09 UTC

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: Review the warnings below.
Findings: 0 blockers · 1 warning · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 1 warning · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Model comparison: normalized findings differ; normalized terminology decisions differ; normalized E2E selections differ; Nemotron reported the same number of blockers, 1 fewer warning, the same number of suggestions.
1 terminology difference from the second opinion

Advisory only. These are normalized differences from the primary terminology receipt.

  • protected candidate at test/e2e/support/managed-image-multiarch-startup-helpers.test.ts:60: selected only by the second-opinion lane as justified.
1 additional E2E selection from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

  • managed-image-protected-runtime: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

1 semantic terminology decision

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

  • justified — candidate identity at test/e2e/support/llama-cpp-dgx-spark-qualification-workflow.test.ts:76: Keep “candidate identity” for the protected evidence identity. Its modifier distinguishes it from the manual PR risk-signal binding.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite against this exact revision.

Recommended E2E: cloud-inference, cloud-onboard, security-posture

2 optional E2E recommendations
  • managed-image-multiarch-startup
  • llama-cpp-dgx-spark-qualification
1 warning · 0 suggestions

Warnings

Warnings do not block.

PRA-1 Warning — Cover the llama.cpp dedicated-head environment contract

  • Location: test/e2e/live/llama-cpp-dgx-spark-qualification-helpers.ts:38
  • Category: tests
  • Problem: The changed helper reads NEMOCLAW_LLAMA_CPP_QUALIFICATION_HEAD_SHA, but the checked-in focused tests do not prove that the old NEMOCLAW_E2E_EXPECTED_SHA variable cannot satisfy this required protected identity.
  • Impact: A future fallback to the PR-only risk-signal variable could make ordinary-main evidence identity regress without a focused test detecting it.
  • Recommendation: Add a focused support test for llamaCppDgxSparkQualificationEnvironment() that supplies valid dispatch values and NEMOCLAW_E2E_EXPECTED_SHA, omits NEMOCLAW_LLAMA_CPP_QUALIFICATION_HEAD_SHA, and expects the required-variable error. Assert separately that a valid dedicated value becomes identity.headSha.
  • Verification: Inspect the llama.cpp helper support tests for an environment-stubbed negative case that omits the dedicated head variable while setting NEMOCLAW_E2E_EXPECTED_SHA.
  • Test coverage: A helper test rejects a missing NEMOCLAW_LLAMA_CPP_QUALIFICATION_HEAD_SHA even when NEMOCLAW_E2E_EXPECTED_SHA is present, and accepts the dedicated SHA as identity.headSha.
  • Evidence: test/e2e/live/llama-cpp-dgx-spark-qualification-helpers.ts:37-46 reads NEMOCLAW_LLAMA_CPP_QUALIFICATION_HEAD_SHA into identity.headSha. test/e2e/support/managed-image-multiarch-startup-helpers.test.ts:60-67 covers the equivalent managed-image negative case. test/e2e/live/llama-cpp-dgx-spark-qualification.test.ts:27-58 consumes the helper in a live test but does not isolate absent-environment behavior.

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@jyaunches
jyaunches enabled auto-merge (squash) August 7, 2026 16:24
@jyaunches
jyaunches merged commit 272cc64 into main Aug 7, 2026
67 checks passed
@jyaunches
jyaunches deleted the codex/e2e-main-dispatch-sha branch August 7, 2026 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants