ci(e2e): unify maintainer approval - #7832
Conversation
📝 WalkthroughWalkthroughThe PR E2E gate now uses a unified maintainer-only ChangesUnified PR E2E approval
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Maintainer
participant PRGateWorkflow
participant PrE2EGateController
participant CoordinationCheck
Maintainer->>PRGateWorkflow: Dispatch approve-e2e with PR and SHA inputs
PRGateWorkflow->>PrE2EGateController: Invoke controller in approve-e2e mode
PrE2EGateController->>CoordinationCheck: Validate approval and revision identity
PrE2EGateController->>PRGateWorkflow: Start authorized E2E evaluation
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tools/e2e/pr-e2e-gate.mts (1)
3108-3127: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winFork authorization instructions omit the "gate passes only if evidence verifies" clarification.
The control-plane authorization message (Line 3156) explicitly states: "This gate passes only if the dispatched evidence references both SHAs and verifies successfully." The fork authorization message doesn't carry the equivalent clarification, only stating that no job ran and no credential was exposed. Given the path instructions explicitly require that "Approval is authorization only: it must not make the gate pass by itself," maintainers approving fork PRs should get the same clarification as maintainers approving control-plane PRs to avoid assuming
approve-e2ealone completes the gate.As per path instructions (test/e2e/README.md): "Approval is authorization only: it must not make the gate pass by itself—coordination succeeds only when the selected E2E run succeeds and every selected job shard/typed target uploads complete passing evidence."
📝 Proposed fix to align fork instructions with control-plane instructions
[ `Review scope: PR #${pull.number}; head repository \`${command.headRepository}\`; head SHA \`${command.headSha}\`; base SHA \`${ciIdentity.baseSha}\`; ${selectionSummary}; deterministic plan \`${plan.planHash}\`.`, "No selected E2E job or target ran. No repository credential was exposed to fork code.", `A repository maintainer must review the exact fork code and risk plan in ${gateRunLink}, then launch a first-attempt \`approve-e2e\` operation from the [${WORKFLOW_NAME}](${workflowUrl}) workflow.`, `Use \`pr_number=${pull.number}\`, \`expected_head_sha=${command.headSha}\`, \`expected_base_sha=${ciIdentity.baseSha}\`, and a specific \`review_reason\`. The trusted controller verifies the maintainer role and exact reviewed inputs before it dispatches this plan.`, + "This gate passes only if the dispatched evidence references both SHAs and verifies successfully.", ].join("\n\n"),🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tools/e2e/pr-e2e-gate.mts` around lines 3108 - 3127, Update the fork authorization message assembled in the markCheckInProgress call to explicitly state that approval only authorizes execution and does not make the gate pass. Add the clarification that the gate passes only after the dispatched evidence references both SHAs and verifies successfully, while preserving the existing fork credential and review instructions.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/e2e/README.md`:
- Around line 772-774: Update the shared resolver description in the README to
replace “matching fork approval state” with unified coordination-state
terminology, such as matching pending coordination or maintainer approval state.
Do not imply a separate fork-specific authorization mechanism.
---
Outside diff comments:
In `@tools/e2e/pr-e2e-gate.mts`:
- Around line 3108-3127: Update the fork authorization message assembled in the
markCheckInProgress call to explicitly state that approval only authorizes
execution and does not make the gate pass. Add the clarification that the gate
passes only after the dispatched evidence references both SHAs and verifies
successfully, while preserving the existing fork credential and review
instructions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: e647d18b-fbaa-40e0-aca9-c43cf7091f54
📒 Files selected for processing (12)
.agents/skills/nemoclaw-maintainer-day/MERGE-GATE.md.github/workflows/pr-e2e-gate.yamltest/e2e/README.mdtest/maintainer-skills-policy.test.tstest/pr-e2e-gate-command.test.tstest/pr-e2e-gate-dispatch-recovery.test.tstest/pr-e2e-gate-fork-approval.test.tstest/pr-e2e-gate-lifecycle.test.tstest/pr-e2e-gate-retry-history.test.tstest/pr-e2e-gate-workflow.test.tstest/pr-e2e-required.test.tstools/e2e/pr-e2e-gate.mts
PR Review Advisor — No blocking findings reportedAdvisor assessment: No blocking advisor findings reported Model lanes
Nemotron output stays in workflow artifacts and does not change the assessment above. E2E guidanceAdvisory only. E2E / PR Gate selects and runs jobs independently. Recommended E2E: This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge. |
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
<!-- markdownlint-disable MD041 --> ## Summary Publish `E2E / PR Gate` directly from the trusted controller instead of mirroring the controller verdict through a six-hour polling job. The required check keeps the exact PR head, base, GitHub App, retry-history, and final-state validation already enforced by the controller. ## Related Issue Closes #7451 ## Changes - Remove the polling `required` workflow job and delete `tools/e2e/pr-e2e-required.mts` with its observer tests. - Rename the controller-owned custom check to the required `E2E / PR Gate` context and update the child-workflow authorization boundary. - Keep `E2E / PR Gate Coordination` as a rollout-only maintainer-inspection fallback when the current check is absent. - Update risk planning, watch triggers, merge-gate inspection, tests, and operator documentation for the direct lifecycle. - Preserve automatic internal dispatch and the unified exact-SHA `approve-e2e` route for selected fork work established by #7826 and #7832. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: [Nine-category security review](#7935 (comment)) passed all categories with no findings. Exact head/base identity, GitHub App ownership, retry lineage, final live-state checks, and child authorization remain fail closed. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `docs-updated` - Evidence: `.agents/skills/nemoclaw-maintainer-day/MERGE-GATE.md`, `test/e2e/README.md`, and `test/e2e/docs/README.md` accurately document the direct check, rollout fallback, automatic internal dispatch, and fork-only `approve-e2e` route. The exact-head documentation review corrected the seeded-check identity rule; a follow-up review confirmed the final test-only fixture change needs no additional prose. Focused documentation-policy validation passed 19 tests. - Agent: Codex Desktop <!-- docs-review-head-sha: 848d79aba921 --> <!-- docs-review-agents-blob-sha: 3dd7c24 --> ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: - Station profile/scenario: - Result: - Supporting evidence: ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — `npm exec -- vitest run --project integration test/pr-e2e-gate*.test.ts test/maintainer-skills-policy.test.ts test/pr-risk-plan.test.ts test/skills/check-gates*.test.ts test/vitest-watch-triggers.test.ts` passed 23 files and 508 tests; the final ordered-comparison fixture passed 26/26 controller tests; `npm exec -- vitest run --project e2e-support test/e2e/support/e2e-operations-workflow-boundary.test.ts` passed 45 tests; `npm run test:changed` passed with no source-backed tests selected. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: Not applicable. This is a focused controller/check-name migration; the targeted integration and workflow-boundary suites cover the changed behavior, and required CI supplies the broad result. `npm run check:diff` and `npm run typecheck:cli` passed against current `main`. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Charan Jagwani <cjagwani@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved E2E merge-gate verification using the trusted `E2E / PR Gate` check, with clearer retry semantics and fail-closed behavior. * Strengthened PR identity, stale-state, dispatch authorization, fork approval, and runner-loss recovery handling. * Improved verdict and evidence reporting, including compatibility with the former check name during rollout. * **Documentation** * Updated merge-gate and E2E CI documentation for revised lifecycle, authorization, retry, and rollout behavior. * **Tests** * Expanded coverage for lifecycle, retries, recovery, fork approval, rollout compatibility, and evidence handling. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
Summary
Use one trusted
approve-e2eoperation for internal and fork PR authorization. Any account with currentmaintainoradminpermission can authorize the exact PR SHA, base SHA, and risk plan. The controller still requires passing evidence before it completes the gate.Changes
maintainoradminrepository permission for everyapprove-e2erequest.Type of Change
Quality Gates
426b12b16with no findings. The controller checks currentmaintainoradminpermission and rejectswritepermission before any credential-bearing dispatch.Documentation Writer Review
docs-updated.agents/skills/nemoclaw-maintainer-day/MERGE-GATE.mdandtest/e2e/README.mddocument the maintainer prerequisite, exact-revision review, credential risk, failure behavior, and passing-evidence criterion. The review found no actionable findings and confirmed that nodocs/source page is needed.DGX Station Hardware Evidence
Verification
Signed-off-by:line and every commit appears asVerifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run check:diffpassed when hooks were skipped or unavailablenpm exec -- vitest run --project integration test/pr-e2e-gate*.test.ts test/pr-e2e-required.test.ts test/maintainer-skills-policy.test.tspassed 16 files and 276 tests on426b12b16.npm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes — command/result: Not applicable. This change does not modify the repository-wide test harness or validation configuration.npm run check:diffpassed, and required CI supplies the broad Linux result.npm run docsbuilds without warnings (doc changes only)npm run docspassed with zero errors and two existing Fern warnings.Signed-off-by: Julie Yaunches jyaunches@nvidia.com