Skip to content

test(e2e): bridge retired controller selectors - #7647

Merged
cv merged 3 commits into
mainfrom
codex/bridge-retired-e2e-selectors
Jul 27, 2026
Merged

test(e2e): bridge retired controller selectors#7647
cv merged 3 commits into
mainfrom
codex/bridge-retired-e2e-selectors

Conversation

@cv

@cv cv commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

PR #7625 correctly retired six local-only E2E scenarios, but the trusted controller on main can still dispatch those retired selector IDs while a candidate workflow is taking over. This adds a fail-closed transition bridge: the candidate runs the ordinary replacement tests once and emits controller-bound evidence for each retired selector while the remaining live jobs continue.

Related Issue

Follow-up to #7625 and #7616
Parent: #7614

Changes

  • Add a compatibility job for the current requirement that trusted PR E2E controllers can select IDs retired by the candidate they dispatch. The consumer is the SHA-bound e2e.yaml workflow invoked by the controller on main.
  • Keep the raw trusted selector list for evidence, filter only the six known retired IDs from candidate planning, and fail closed when no active controller-selected job remains.
  • Run each replacement Vitest project once, require the retired live files to remain absent, and emit SHA-, plan-, correlation-, and shard-bound risk signals for all selected retired IDs.
  • Extend the prepare/upload workflow boundary contracts and add planner and compatibility regression tests. A direct controller change is insufficient because the controller must remain on trusted main; restoring the retired live jobs would reintroduce the cost this migration removes.
  • Keep the retired selector list authoritative in the executable workflow boundary, reject workflow-gate drift, and pass the caller-supplied environment to every replacement command.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: This is an internal controller-transition path. It adds no supported CLI behavior, configuration, operator action, or public documentation contract.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: The bridge remains SHA-, plan-, correlation-, and shard-bound, rejects unsafe selectors, preserves raw trusted evidence, and fails closed without another active controller job. The exact nine-selector replay emitted six valid bound signals.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: No documentation paths changed or require updates. The reviewer checked docs/, README.md, CONTRIBUTING.md, and test/e2e/README.md; existing guidance already covers controller planning, SHA-bound checkout, protected approval, and evidence validation without promising transient selector IDs.
  • Agent: Codex Desktop documentation-writer subagent (/root/documentation_writer_review)

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — exact nine-selector replay: 47 integration, 3 installer-integration, and 4 package-contract assertions passed with six bound risk signals; focused workflow contracts: 75/75; full E2E-support: 163 files passed, 3 skipped, 1,737 tests passed, 17 skipped; source-shape cases remain zero; semantic phase coverage: 117 tests across 76 files; Vitest membership: exact 1,861 files across 8 projects; npm run build:cli and npm run check:diff passed.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: npm test was attempted on the original PR head and reproduced three unchanged DGX Station host-detection fixture failures plus one unchanged root-topology fixture failure, then stopped making progress and was interrupted. GitHub CI will provide hosted aggregate evidence for the updated head.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

  • New Features
    • Added retired-selector-compatibility checks that run targeted replacement E2E tests and generate compatibility evidence artifacts (including risk-signal and summary output).
    • Expanded CI and PR reporting to include a dedicated retired-selector-compatibility job when relevant.
    • Updated workflow planning and boundary validation to map controller-selected jobs to compatible sets and enforce retired-only selection rules.
  • Bug Fixes
    • Prevented planning/runs when controller-selected jobs become fully retired without required targets.
  • Tests
    • Added E2E coverage for selector filtering, artifact outputs, error/guard behaviors, and workflow selector gate drift detection.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Adds retired controller selector compatibility execution, planner filtering, evidence artifacts, workflow integration, and validation tests for successful and failing selection paths.

Changes

Retired selector compatibility

Layer / File(s) Summary
Selector contracts and replacement mapping
tools/e2e/retired-selector-compatibility.mts, tools/e2e/workflow-boundary.mts
Defines retired selector IDs, replacement projects, selection filtering, and Vitest command construction.
Compatibility execution and evidence
tools/e2e/retired-selector-compatibility.mts, test/e2e/support/retired-selector-compatibility.test.ts
Validates replacement boundaries, runs replacement commands, writes risk-signal and summary artifacts, and tests success and failure paths.
Workflow planning integration
tools/e2e/workflow-plan.mts, test/e2e/support/workflow-plan.test.ts
Filters retired selectors from planned jobs and rejects plans containing only retired selectors without targets.
Workflow job and boundary validation
.github/workflows/e2e.yaml, tools/e2e/*workflow-boundary.mts, test/e2e/support/e2e-workflow.test.ts
Adds the compatibility job, PR reporting dependency, upload/prepare contracts, and selector-gate drift validation.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Suggested labels: area: ci, area: e2e

Sequence Diagram(s)

sequenceDiagram
  participant Workflow as GitHub Actions workflow
  participant Planner as workflow-plan
  participant Runner as retired-selector-compatibility
  participant Vitest
  participant Artifacts as E2E artifact upload
  Workflow->>Planner: plan selected E2E jobs
  Planner->>Planner: filter retired selectors
  Workflow->>Runner: run selected compatibility jobs
  Runner->>Vitest: execute replacement projects
  Vitest-->>Runner: return test status
  Runner->>Artifacts: write and upload evidence
  Workflow->>Workflow: report compatibility result to PR
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: bridging retired controller selectors in E2E.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/bridge-retired-e2e-selectors

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 8980c62 in the codex/bridge-retired... branch remains at 96%, unchanged from commit d0a95dc in the main branch.


Updated July 27, 2026 18:01 UTC

@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 2 warnings · 0 suggestions
  • Model comparison: normalized findings differ; normalized E2E selections match; Nemotron reported the same number of blockers, 2 more warnings, the same number of suggestions.

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-onboard, credential-sanitization, security-posture

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/e2e.yaml:
- Line 349: Update the selector gate in the workflow condition to derive from
the compatibility contract represented by RETIRED_CONTROLLER_SELECTOR_IDS and
the planner, rather than duplicating the six job IDs. Expose planner output for
compatibility selection or add deterministic validation that fails when this
condition diverges from the source contract, ensuring retired selectors always
trigger the job.

In `@tools/e2e/retired-selector-compatibility.mts`:
- Around line 138-152: Update the default runCommand helper and its invocation
within runRetiredSelectorCompatibility to accept and forward the provided
environment, using it as the spawnSync env instead of process.env. Thread
environment through the options.runCommand override call while preserving the
existing command execution and error handling behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6c475694-617e-41a4-9294-56a43d347007

📥 Commits

Reviewing files that changed from the base of the PR and between a8748ab and 594ead3.

📒 Files selected for processing (7)
  • .github/workflows/e2e.yaml
  • test/e2e/support/retired-selector-compatibility.test.ts
  • test/e2e/support/workflow-plan.test.ts
  • tools/e2e/prepare-e2e-workflow-boundary.mts
  • tools/e2e/retired-selector-compatibility.mts
  • tools/e2e/upload-e2e-artifacts-workflow-boundary.mts
  • tools/e2e/workflow-plan.mts

Comment thread .github/workflows/e2e.yaml
Comment thread tools/e2e/retired-selector-compatibility.mts Outdated
cv added 2 commits July 27, 2026 10:14
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv
cv merged commit 5e6a821 into main Jul 27, 2026
89 of 98 checks passed
@cv
cv deleted the codex/bridge-retired-e2e-selectors branch July 27, 2026 18:07
@wscurran wscurran added area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure chore Build, CI, dependency, or tooling maintenance labels Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure chore Build, CI, dependency, or tooling maintenance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants