fix(e2e): record risk-signal evidence only for manual PR runs

[prekshivyas]

Summary

On each push to main, the protected jobs no longer record risk-signal evidence for a manual PR run. The candidate SHA checks and job evidence validation remain unchanged.

This claims the root cause observed in E2E run 31140352904:

• Protected managed-image startup (linux/amd64)
• Protected managed-image startup (linux/arm64)

Both jobs passed their evidence validation before the reporter failed. The main fallback supplied NEMOCLAW_E2E_EXPECTED_SHA without the manual PR plan hash or correlation ID.

Changes

• Keep NEMOCLAW_E2E_EXPECTED_SHA bound to the checkout_sha input in both protected jobs.
• Update both workflow boundary validators to preserve the manual PR risk-signal contract.
• Add a mutation test for each validator that rejects the github.sha fallback.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Quality Gates

• Tests added or updated for changed behavior
• Existing tests cover changed behavior — justification:
• Tests not applicable — justification:
• Docs updated for user-facing behavior changes
• Docs not applicable — justification: this restores the documented manual PR run behavior and does not change a user-visible interface.
• Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
• Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: nine-category review passed for c1ad5d379 with no findings; see the current PR conversation.
• Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

• Documentation writer subagent reviewed the completed changes
• Result: blocked
• Evidence: Independent documentation writer review is unavailable under the current collaboration constraints. No repository documentation changed. WRITING.md and the controlled word list were applied to the PR text after maintainer feedback.
• Agent: Codex Desktop

DGX Station Hardware Evidence

• Tested on DGX Station
• Tested commit: Not applicable
• Station profile/scenario: Not applicable
• Result: Not applicable
• Supporting evidence: Not applicable

Verification

• PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
• Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable — npm run validate:pr passed on c1ad5d379 after merging current main.
• Targeted behavior tests pass for the current change set, or tests are marked not applicable above — the managed-image, DGX Spark workflow, and DGX Spark plan test files pass 35 tests.
• Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: not applicable because this focused change does not alter the runtime, test harness, hooks, formatters, generated checks, or shared validation behavior.
• Quality Gates section completed with required justifications or waivers
• No secrets, API keys, or credentials committed
• npm run docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new docs only)

---

Signed-off-by: Prekshi Vyas prekshiv@nvidia.com

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

📝 Walkthrough

Walkthrough

Protected managed-image and llama.cpp qualification workflows now bind NEMOCLAW_E2E_EXPECTED_SHA only to inputs.checkout_sha. Boundary validators and tests enforce this exact binding.

Changes

Expected SHA binding

Layer / File(s)	Summary
Update expected SHA bindings .github/workflows/e2e.yaml, tools/e2e/*workflow-boundary.mts	Protected workflow jobs and boundary validators no longer fall back to github.sha.
Add binding regression tests test/e2e/support/*workflow.test.ts	Validation tests reject bindings that include a github.sha fallback.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• NVIDIA/NemoClaw#8477: Modifies NEMOCLAW_E2E_EXPECTED_SHA and related managed-image and llama.cpp boundary validation.

Suggested reviewers: cv

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: restricting E2E risk-signal evidence to manual PR runs.

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch agent/fix-main-risk-signal-activation

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-code-quality]

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit c1ad5d3 in the agent/fix-main-risk-... branch remains at 96%, unchanged from commit b572cbe in the main branch.

---

_{Updated August 07, 2026 04:30 UTC}

[cv]

"exact-head" is not in our controlled docs and comms style. Prekshi's agent, why did you say that in the PR description? What else about our docs and comms did you overlook or violate?

[prekshivyas]

You are right. I wrote the first PR description before I applied WRITING.md and the controlled word list. In addition to exact-head, I used lane where the repository guidance uses job, and I used manual-PR instead of manual PR run. I replaced those terms with candidate SHA, job, and manual PR run. I also split the failure explanation so each sentence states one result. The description now preserves literal identifiers only where they name workflow inputs or environment variables. The code diff does not add explanatory text.

[github-actions]

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

• GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
• Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
• Model comparison: normalized findings match; normalized terminology decisions differ; normalized E2E selections differ; severity counts match.

2 additional E2E selections from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

• managed-image-multiarch-startup: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
• llama-cpp-dgx-spark-qualification: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

2 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

• define — manual PR risk signals at test/e2e/support/llama-cpp-dgx-spark-qualification-workflow.test.ts:40: Define the term in the test title or use a concrete phrase such as 'does not set the manual PR checkout SHA on main pushes'.
• define — manual PR risk signals at test/e2e/support/managed-image-protected-runtime-workflow.test.ts:53: Define the term in the test title or use a concrete phrase such as 'does not set the manual PR checkout SHA on main pushes'.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite against this exact revision.

Recommended E2E: cloud-inference, cloud-onboard, security-posture

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

[prekshivyas]

Security review verdict

PASS for e3fed1804. This change stops manual PR risk-signal recording on pushes to main. It does not change job selection, permissions, credentials, candidate SHA validation, cleanup, or artifact upload.

This verdict covers the security review only. Required checks and maintainer review still control the merge.

Findings

The review found no security findings.

Detailed analysis

1. Secrets and Credentials — PASS. The diff does not add or move a credential. It removes an unintended reporter configuration from main push jobs.
2. Input Validation and Data Sanitization — PASS. The workflow still validates the candidate SHA through each protected job's CHECKOUT_SHA guard. Both workflow boundary validators require the manual PR input for risk-signal metadata.
3. Authentication and Authorization — PASS. Repository, branch, event, runner, and workflow SHA checks are unchanged. Job permissions are unchanged.
4. Dependencies and Third-Party Libraries — PASS. The diff adds no dependency, action, image, download, or registry.
5. Error Handling and Logging — PASS. Main push evidence failures still fail their jobs. The change prevents missing manual PR metadata from replacing the job result with a reporter error.
6. Cryptography and Data Protection — PASS. The diff does not change cryptography, transport protection, hashing, or stored data.
7. Configuration and Security Headers — PASS. The change affects only the reporter opt-in value. Protected job selectors, candidate checkouts, permissions, and cleanup remain unchanged.
8. Security Testing — PASS. The affected workflow validators are part of the 21 passing focused tests. npm run validate:pr also passes on the current branch.
9. System Security — PASS. Pushes to main use the protected jobs' candidate SHA and evidence contracts. Manual PR runs still supply checkout_sha and keep the existing risk-signal path.

Files reviewed

• .github/workflows/e2e.yaml
• tools/e2e/managed-image-multiarch-workflow-boundary.mts
• tools/e2e/llama-cpp-dgx-spark-qualification-workflow-boundary.mts

[prekshivyas]

Security review verdict

PASS for 25e7fa8dd. This change stops manual PR risk-signal recording on pushes to main. It does not change job selection, permissions, credentials, candidate SHA validation, cleanup, or artifact upload.

This verdict covers the security review only. Required checks and maintainer review still control the merge.

Findings

The review found no security findings.

Detailed analysis

1. Secrets and Credentials — PASS. The diff does not add or move a credential. It removes an unintended reporter configuration from main push jobs.
2. Input Validation and Data Sanitization — PASS. The workflow still validates the candidate SHA through each protected job's CHECKOUT_SHA guard. Both workflow boundary validators require the manual PR input for risk-signal metadata.
3. Authentication and Authorization — PASS. Repository, branch, event, runner, and workflow SHA checks are unchanged. Job permissions are unchanged.
4. Dependencies and Third-Party Libraries — PASS. The diff adds no dependency, action, image, download, or registry.
5. Error Handling and Logging — PASS. Main push evidence failures still fail their jobs. The change prevents missing manual PR metadata from replacing the job result with a reporter error.
6. Cryptography and Data Protection — PASS. The diff does not change cryptography, transport protection, hashing, or stored data.
7. Configuration and Security Headers — PASS. The change affects only the reporter opt-in value. Protected job selectors, candidate checkouts, permissions, and cleanup remain unchanged.
8. Security Testing — PASS. Two mutation tests restore the old github.sha fallback and require both validators to reject it. All 35 focused tests and npm run validate:pr pass.
9. System Security — PASS. Pushes to main use the protected jobs' candidate SHA and evidence contracts. Manual PR runs still supply checkout_sha and keep the existing risk-signal path.

Files reviewed

• .github/workflows/e2e.yaml
• test/e2e/support/managed-image-protected-runtime-workflow.test.ts
• test/e2e/support/llama-cpp-dgx-spark-qualification-workflow.test.ts
• tools/e2e/managed-image-multiarch-workflow-boundary.mts
• tools/e2e/llama-cpp-dgx-spark-qualification-workflow-boundary.mts

[prekshivyas]

Addressed PRA-1 in 25e7fa8dd. Each affected workflow boundary test now restores ${{ inputs.checkout_sha || github.sha }} and requires its validator to reject that fallback. The focused run passes 35 tests, and npm run validate:pr passes on the same commit.

[prekshivyas]

Security review verdict

PASS for c1ad5d379. This change stops manual PR risk-signal recording on pushes to main. It does not change job selection, permissions, credentials, candidate SHA validation, cleanup, or artifact upload. The branch includes b572cbe52 from main; that merge does not change the effective PR diff.

This verdict covers the security review only. Required checks and maintainer review still control the merge.

Findings

The review found no security findings.

Detailed analysis

1. Secrets and Credentials — PASS. The diff does not add or move a credential. It removes an unintended reporter configuration from main push jobs.
2. Input Validation and Data Sanitization — PASS. The workflow still validates the candidate SHA through each protected job's CHECKOUT_SHA guard. Both workflow boundary validators require the manual PR input for risk-signal metadata.
3. Authentication and Authorization — PASS. Repository, branch, event, runner, and workflow SHA checks are unchanged. Job permissions are unchanged.
4. Dependencies and Third-Party Libraries — PASS. The diff adds no dependency, action, image, download, or registry.
5. Error Handling and Logging — PASS. Main push evidence failures still fail their jobs. The change prevents missing manual PR metadata from replacing the job result with a reporter error.
6. Cryptography and Data Protection — PASS. The diff does not change cryptography, transport protection, hashing, or stored data.
7. Configuration and Security Headers — PASS. The change affects only the reporter opt-in value. Protected job selectors, candidate checkouts, permissions, and cleanup remain unchanged.
8. Security Testing — PASS. Two mutation tests restore the old github.sha fallback and require both validators to reject it. All 35 focused tests and npm run validate:pr pass.
9. System Security — PASS. Pushes to main use the protected jobs' candidate SHA and evidence contracts. Manual PR runs still supply checkout_sha and keep the existing risk-signal path.

Files reviewed

• .github/workflows/e2e.yaml
• test/e2e/support/managed-image-protected-runtime-workflow.test.ts
• test/e2e/support/llama-cpp-dgx-spark-qualification-workflow.test.ts
• tools/e2e/managed-image-multiarch-workflow-boundary.mts
• tools/e2e/llama-cpp-dgx-spark-qualification-workflow-boundary.mts