Skip to content

fix(e2e): attest protected candidate checkout - #8611

Merged
ericksoa merged 3 commits into
mainfrom
fix/protected-e2e-candidate-attestation
Aug 8, 2026
Merged

fix(e2e): attest protected candidate checkout#8611
ericksoa merged 3 commits into
mainfrom
fix/protected-e2e-candidate-attestation

Conversation

@ericksoa

@ericksoa ericksoa commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Protected managed-image GPU qualification runs trusted workflow code beside an exact candidate checkout, but the risk reporter resolved HEAD from the trusted workspace while expecting the candidate SHA. This binds the reporter to the explicit isolated candidate root so protected evidence attests the code actually under test.

Related Issue

Prerequisite for #8058.

Changes

  • Allow the risk reporter to resolve an explicitly provided absolute tested-root path while retaining the existing workspace fallback for other jobs.
  • Bind the protected managed-image GPU job to ${{ github.workspace }}/.candidate-runtime and the exact candidate SHA.
  • Add reporter and workflow-boundary tests proving the trusted workspace cannot be mistaken for the candidate checkout.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: This corrects internal protected-E2E evidence identity; it changes no user command, configuration, default, API, or supported runtime behavior.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Maintainer-approved prerequisite scope for feat(runtime): persist engine lifecycle and state mutation contract #8058. The change narrows evidence identity to the explicitly checked-out candidate, requires an absolute tested root, retains exact-SHA equality, and adds negative workflow-contract coverage; it does not change credentials, permissions, checkout provenance, or runtime code.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: This prerequisite and follow-up change only internal protected E2E commit attestation, relative-root rejection, and their workflow/source contracts. They add no user-facing command, configuration, default, API, or supported behavior.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit: Not applicable; scripts/prepare-dgx-station-host.sh is unchanged.
  • Station profile/scenario: Not applicable.
  • Result: Not applicable.
  • Supporting evidence: Not applicable.

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — test/e2e-risk-signal-reporter.test.ts and test/e2e/support/managed-image-protected-runtime-workflow.test.ts: 36/36 passed; npm run typecheck:cli, Biome, workflow/config validation, test-conditionals, source-shape, test-size, pre-commit, commit-msg, and pre-push gates passed.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: Not applicable; the five-file change is limited to one protected workflow identity binding and focused executable contract tests.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Aaron Erickson aerickson@nvidia.com

Summary by CodeRabbit

  • Bug Fixes

    • Improved end-to-end validation to consistently identify the correct tested checkout and commit.
    • Ensured protected runtime evidence is tied to the isolated candidate runtime.
    • Added safeguards for absolute checkout paths and expected commit values.
  • Tests

    • Added coverage for separated trusted and candidate runtimes.
    • Added boundary tests for runtime-root and checkout-commit configuration.
    • Verified commit selection works with both manual checkout values and the current commit.

@coderabbitai

coderabbitai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The E2E risk signal now resolves commits from an explicit candidate runtime root. The protected managed-image workflow exports the expected SHA and candidate root. Boundary tests validate both bindings.

Changes

Protected runtime checkout binding

Layer / File(s) Summary
Tested root resolution
tools/e2e/risk-signal.ts, test/e2e-risk-signal-reporter.test.ts
NEMOCLAW_E2E_TESTED_ROOT is preferred for Git HEAD resolution. Relative roots are rejected. Tests cover explicit candidate checkout handling.
Workflow binding validation
tools/e2e/managed-image-protected-runtime-workflow-boundary.mts, .github/workflows/e2e.yaml, test/e2e/support/managed-image-protected-runtime-workflow.test.ts
The workflow exports the expected candidate SHA and .candidate-runtime root. Boundary tests enforce both bindings.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested reviewers: cv

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: fixing E2E attestation for the protected candidate checkout.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/protected-e2e-candidate-attestation

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 9c18039 in the fix/protected-e2e-ca... branch remains at 96%, unchanged from commit 02398f3 in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 9c18039 in the fix/protected-e2e-ca... branch remains at 81%, unchanged from commit 02398f3 in the main branch.

Show a code coverage summary of the most impacted files.
File main 02398f3 fix/protected-e2e-ca... 9c18039 +/-
src/lib/onboard...shared-state.ts 90% 73% -17%
src/lib/policy/...ateway-state.ts 73% 56% -17%
src/lib/onboard.../docker-spec.ts 74% 65% -9%
src/lib/onboard...strap/docker.ts 72% 64% -8%
src/lib/onboard...ne-authority.ts 91% 84% -7%
src/lib/onboard...cker-journal.ts 75% 70% -5%
src/lib/onboard...trap/adapter.ts 70% 68% -2%
src/lib/onboard...mo-lifecycle.ts 86% 84% -2%
src/lib/policy/index.ts 62% 63% +1%
src/lib/onboard...an-lifecycle.ts 79% 95% +16%

Updated August 08, 2026 05:26 UTC

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/e2e.yaml:
- Line 2592: Use the same fallback expression for the expected candidate SHA
everywhere: update NEMOCLAW_E2E_EXPECTED_SHA in .github/workflows/e2e.yaml at
lines 2592-2592 and the corresponding expectation in
tools/e2e/managed-image-protected-runtime-workflow-boundary.mts at lines 117-117
to use inputs.checkout_sha || github.sha. Add a no-input regression test in
test/e2e/support/managed-image-protected-runtime-workflow.test.ts verifying
github.sha is used when checkout_sha is absent.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 727ff8d7-12d7-47b1-bf95-eebffb270199

📥 Commits

Reviewing files that changed from the base of the PR and between 34e8745 and 2ee0bea.

📒 Files selected for processing (5)
  • .github/workflows/e2e.yaml
  • test/e2e-risk-signal-reporter.test.ts
  • test/e2e/support/managed-image-protected-runtime-workflow.test.ts
  • tools/e2e/managed-image-protected-runtime-workflow-boundary.mts
  • tools/e2e/risk-signal.ts

Comment thread .github/workflows/e2e.yaml Outdated
@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Model comparison: normalized findings match; normalized terminology decisions differ; normalized E2E selections differ; severity counts match.
3 terminology differences from the second opinion

Advisory only. These are normalized differences from the primary terminology receipt.

  • isolated exact candidate at test/e2e/support/managed-image-protected-runtime-workflow.test.ts:53: selected only by the second-opinion lane as justified.
  • relative tested root at test/e2e-risk-signal-reporter.test.ts:130: selected only by the second-opinion lane as justified.
  • explicit tested root at test/e2e-risk-signal-reporter.test.ts:114: selected only by the second-opinion lane as justified.
1 additional E2E selection from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

  • managed-image-multiarch-startup: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

2 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

  • justified — tested root at tools/e2e/risk-signal.ts:79: Keep `tested root` for the directory used to resolve the attested commit.
  • established — candidate checkout at test/e2e-risk-signal-reporter.test.ts:114: Keep `candidate checkout` for the isolated candidate source tree.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite against this exact revision.

Recommended E2E: cloud-inference, cloud-onboard, security-posture

1 optional E2E recommendation
  • managed-image-protected-runtime

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
@ericksoa
ericksoa merged commit b834de6 into main Aug 8, 2026
80 of 83 checks passed
@ericksoa
ericksoa deleted the fix/protected-e2e-candidate-attestation branch August 8, 2026 05:30
@github-actions github-actions Bot added the v0.0.105 Release target label Aug 8, 2026
@ericksoa
ericksoa restored the fix/protected-e2e-candidate-attestation branch August 8, 2026 05:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

v0.0.105 Release target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant