Skip to content

ci(e2e): explain maintainer approval state - #7863

Merged
cv merged 5 commits into
mainfrom
codex/7861-e2e-approval-diagnostics
Aug 3, 2026
Merged

ci(e2e): explain maintainer approval state#7863
cv merged 5 commits into
mainfrom
codex/7861-e2e-approval-diagnostics

Conversation

@cjagwani

@cjagwani cjagwani commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

Replace the opaque maintainer-approval state error with bounded, actionable diagnostics. Approval still fails closed on every mismatch, but maintainers can now distinguish an early, wrong-route, already-running, terminal, or malformed coordination state without exposing the observed check output.

Related Issue

Fixes #7861

Changes

  • Classify only known coordination states and provide fixed remediation plus the fixed expected authorization title.
  • Keep the existing exact-revision, role, first-attempt, single-check, and pending-title authorization predicates unchanged.
  • Add regression coverage for early approval, wrong route, active execution, terminal state, malformed state, missing title, and suppression of untrusted output.
  • Document the fail-closed recovery guidance for maintainers.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification:
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Codex Desktop independently reviewed exact head d74c058fe80154db11a856850cae9b7627783d9f against base d5b64a72a5fcb8299e8cec1c2f22746b5c0a7f32 (tree 2fe73e677f0f71ec07d9b38af3709403f64fd174; stable patch 11fbee7d594e9ba80e110c87d56a2e1174bd816e). All nine security categories PASS; the exact authorization predicate is unchanged, all classified states fail closed before mutation or dispatch, and observed output is never echoed.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: docs-updated
  • Evidence: Updated test/e2e/README.md to document the exact pending authorization title and fixed fail-closed remediation for preparing, executing, terminal, and malformed coordination states. Reviewed every changed diagnostic, test title, commit message, and documentation sentence against the repository writing rules and exact controller behavior. Focused fork-approval tests passed 24/24; npm run docs passed with 0 errors and 2 existing warnings; npm run validate:pr passed.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — npm exec vitest -- run --project integration test/pr-e2e-gate-fork-approval.test.ts (24/24)
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: Not applicable to a bounded diagnostic-only controller change.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Additional exact-head validation: focused fork-approval tests passed 24/24; npm run validate:pr passed; npm run docs completed with 0 errors and 2 existing warnings.


Signed-off-by: Charan Jagwani cjagwani@nvidia.com

Summary by CodeRabbit

  • Bug Fixes
    • Improved pull request gate validation with clearer errors for invalid, completed, pending, mismatched, or already-running approval states.
    • Standardized E2E approval handling for fork-based pull requests.
    • Prevented E2E approval requests for same-repository pull requests.
    • Updated approval and execution messages to clarify verification of the reviewed commit and recorded base revision.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@cjagwani cjagwani self-assigned this Jul 30, 2026
@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The PR adds state-specific maintainer-approval diagnostics for the E2E gate, detects already-running execution, routes invalid approval states through targeted errors, and removes control-plane-specific authorization handling from the approval flow.

Changes

Maintainer approval diagnostics

Layer / File(s) Summary
Classify and report coordinator approval states
tools/e2e/pr-e2e-gate.mts
Adds running-E2E title detection, classifies terminal, preparing, mismatched, running, and malformed coordinator states, and uses the resulting error in startAuthorizedPrGate.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Maintainer
  participant startAuthorizedPrGate
  participant CoordinatorCheck
  participant maintainerApprovalStateError
  Maintainer->>startAuthorizedPrGate: request approve-e2e
  startAuthorizedPrGate->>CoordinatorCheck: inspect status and title
  startAuthorizedPrGate->>maintainerApprovalStateError: classify non-pending state
  maintainerApprovalStateError-->>Maintainer: return targeted error
Loading

Possibly related PRs

  • NVIDIA/NemoClaw#7867: Changes the same maintainer-approval and fork-versus-same-repository routing.
  • NVIDIA/NemoClaw#7868: Overlaps in control-plane gating removal and coordinator title/state handling.
  • NVIDIA/NemoClaw#7857: Alters the PR E2E gate’s maintainer authorization stop and approval state behavior.

Suggested labels: bug-fix, area: security

Suggested reviewers: jyaunches, apurvvkumaria, cv

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The diagnostics work aligns with the issue, but the required focused regression tests and documentation changes are not shown here. Add regression tests for early, wrong-route, running, terminal, and malformed states, and document when approval becomes valid and how to recover.
✅ Passed checks (4 passed)
Check name Status Explanation
Out of Scope Changes check ✅ Passed The changes stay within E2E approval gating and diagnostics; no unrelated features or broad additions are evident.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the E2E maintainer approval diagnostic change, which is the main purpose of the pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/7861-e2e-approval-diagnostics

Comment @coderabbitai help to get the list of available commands.

@cjagwani

cjagwani commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator Author

Sensitive-path security review

Verdict

PASS. The change is safe from a security-boundary perspective. It changes only the error selected after the existing authorization predicate has rejected a state; exact PR/head/base matching, current maintainer role, first workflow attempt, single matching check, live fork detection, pending status, and exact expected title remain unchanged. No dispatch or check mutation is reached by any rejected test case.

Findings

No security findings.

Category review

  1. Secrets and credentials — PASS. No secrets, tokens, credentials, or fixtures were added.
  2. Input validation and sanitization — PASS. Remote check titles are compared only with fixed allowlisted titles/patterns and are never interpolated into diagnostics. The only printed title is the route-derived fixed expected title.
  3. Authentication and authorization — PASS. The original fail-closed predicate is unchanged; the helper executes only after it rejects the pending status or exact title. Maintainer role, first-attempt, exact revision, and single-check validation remain before this point.
  4. Dependencies — PASS. No dependency or build configuration changes.
  5. Error handling and logging — PASS. Diagnostics expose a bounded state class and safe action. The malformed-title regression proves untrusted check output is suppressed.
  6. Cryptography and data protection — PASS. No cryptographic or stored-data changes.
  7. Configuration and security headers — PASS. No runtime configuration, permissions, environments, ports, or headers change.
  8. Security testing — PASS. Tests cover early approval, wrong route, existing execution, terminal state, malformed state, missing title, no mutation/dispatch, and untrusted-title suppression.
  9. System security — PASS. No authorization, concurrency, retry, TOCTOU, dispatch, or check-lifecycle behavior changes.

Files reviewed

  • tools/e2e/pr-e2e-gate.mts
  • test/pr-e2e-gate-fork-approval.test.ts
  • test/e2e/README.md

Reviewed exact head: 5ca5f7812be5e19648c728ee42ef2d9de3b71a6d. The second commit only removes a conditional test assertion while preserving unconditional expected-title and untrusted-title suppression checks; the security verdict is unchanged.

@github-code-quality

github-code-quality Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit d74c058 in the codex/7861-e2e-appro... branch remains at 96%, unchanged from commit d5b64a7 in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit d74c058 in the codex/7861-e2e-appro... branch remains at 81%, unchanged from commit d5b64a7 in the main branch.

Show a code coverage summary of the most impacted files.
File main d5b64a7 codex/7861-e2e-appro... d74c058 +/-
src/lib/credentials/store.ts 56% 55% -1%
src/lib/sandbox...rce-identity.ts 88% 88% 0%
src/lib/state/m...-acquisition.ts 75% 75% 0%
src/lib/onboard...ndbox-create.ts 83% 91% +8%
src/lib/domain/.../connect-env.ts 89% 97% +8%
src/lib/onboard...-create-plan.ts 75% 88% +13%
src/lib/onboard...ndbox-create.ts 33% 83% +50%

Updated August 02, 2026 15:44 UTC

@cjagwani cjagwani added area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure area: observability Logging, metrics, tracing, diagnostics, or debug output labels Jul 30, 2026
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — Informational

Advisor assessment: Informational / low confidence
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions
Status: PR review advisor failed: PR review advisor SDK execution failed: session: terminology-review-analysis omitted required analysis; turn: terminology-review-analysis: terminology-review-analysis omitted required analysis

Model lanes

  • GPT-5.6 Terra (primary): Failed
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions

Second-opinion terminology and E2E selections are advisory. They do not change the primary assessment or E2E / PR Gate.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-inference, cloud-onboard, security-posture

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@copy-pr-bot

copy-pr-bot Bot commented Jul 30, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
tools/e2e/pr-e2e-gate.mts (2)

3314-3314: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Do not publish the untrusted review reason in check output.

reason is dispatch input and is interpolated into a public check-run message. Backtick replacement does not prevent secret disclosure or Markdown/control-text injection. Keep this message fixed or use a tightly bounded non-sensitive identifier, and add a regression assertion that arbitrary review reasons are absent.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/e2e/pr-e2e-gate.mts` at line 3314, Update the check-run message in the
reviewed fork PR flow to avoid interpolating the untrusted reason value; use
fixed text or a tightly bounded non-sensitive identifier instead. Add a
regression assertion confirming arbitrary review reasons do not appear in check
output, while preserving the existing message behavior otherwise.

3247-3249: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Fail closed when the PR head repository is unavailable.

This condition rejects same-repository PRs, but pull.head.repo?.full_name may be absent for a deleted or otherwise unavailable fork. In that case the condition is false and approve-e2e can continue without proving that the PR is fork-based. Require a non-empty head repository and ensure it differs from repository.

Proposed fix
-    if (pull.head.repo?.full_name === repository) {
+    const headRepository = pull.head.repo?.full_name;
+    if (!headRepository || headRepository === repository) {
       throw new Error("approve-e2e is only supported for fork pull requests");
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/e2e/pr-e2e-gate.mts` around lines 3247 - 3249, The fork validation
around pull.head.repo?.full_name must fail closed when the head repository is
missing. Update the condition in the approve-e2e validation to require a
non-empty head repository name and reject it when it equals repository,
preserving the existing error behavior for invalid or same-repository pull
requests.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@tools/e2e/pr-e2e-gate.mts`:
- Line 3314: Update the check-run message in the reviewed fork PR flow to avoid
interpolating the untrusted reason value; use fixed text or a tightly bounded
non-sensitive identifier instead. Add a regression assertion confirming
arbitrary review reasons do not appear in check output, while preserving the
existing message behavior otherwise.
- Around line 3247-3249: The fork validation around pull.head.repo?.full_name
must fail closed when the head repository is missing. Update the condition in
the approve-e2e validation to require a non-empty head repository name and
reject it when it equals repository, preserving the existing error behavior for
invalid or same-repository pull requests.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 37d23e3e-78e8-4c59-8b85-202179b08afc

📥 Commits

Reviewing files that changed from the base of the PR and between 5ca5f78 and 7014e33.

📒 Files selected for processing (1)
  • tools/e2e/pr-e2e-gate.mts

cv added 2 commits August 2, 2026 07:46
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>

@apurvvkumaria apurvvkumaria left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve — reviewed exact head d74c058. The change preserves the existing maintainer-approval authorization predicate and replaces only its opaque rejection with fixed, state-specific fail-closed diagnostics. Tests cover preparing, queued, running, terminal, malformed, missing, null, and non-string states and prove no mutation, dispatch, or echo of observed untrusted output. Exact-head CI and E2E gate pass; no blocking defect found.

@cv
cv merged commit 7f7583a into main Aug 3, 2026
81 of 85 checks passed
@cv
cv deleted the codex/7861-e2e-approval-diagnostics branch August 3, 2026 23:39
@github-actions github-actions Bot added the v0.0.102 Release target label Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure area: observability Logging, metrics, tracing, diagnostics, or debug output v0.0.102 Release target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci(e2e): explain why maintainer approval is not ready

3 participants