test(e2e): retire priority 1 dedicated jobs

[prekshivyas]

Summary

Retire the dedicated diagnostics, credential-migration, and credential-sanitization live E2E jobs after relocating their stable assertions to normal coverage projects. The distinct real-sandbox checks now run inside the existing cloud-onboard and cloud-inference journeys, reducing scheduled runner work without dropping the security boundaries.

Related Issue

Fixes #7617

Changes

• Add compiled CLI coverage for diagnostics archive creation, redaction, sandbox scope, and error cleanup, and keep one scoped archive assertion in cloud-onboard.
• Strengthen fast legacy credential migration coverage through the production registration and finalization flow with mocked OpenShell responses, including failed-registration retention, successful cleanup, and the allowlist. Preserve the live seam in cloud-onboard with an isolated legacy file and real gateway registration.
• Move the sandbox agent-state credential scan into cloud-inference and retain local credential-filter and blueprint assertions in fast tests.
• Remove the three dedicated workflow jobs, live files, Brev selectors, mock-parity entries, and obsolete workflow-boundary support.
• Point credential-security and E2E control-plane risk plans at the retained cloud-inference job.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Quality Gates

• Tests added or updated for changed behavior
• Existing tests cover changed behavior — justification:
• Tests not applicable — justification:
• Docs updated for user-facing behavior changes
• Docs not applicable — justification: This reorganizes test coverage and internal CI selectors without changing supported user behavior. The internal test/e2e/README.md inventory was updated.
• Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
• Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Focused diff review confirmed that the existing sandbox secret scan moved intact to cloud-inference, migration rejects non-allowlisted environment injection, diagnostics redact exact and shaped credentials, and no production credential logic changed.
• Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

• Documentation writer subagent reviewed the completed changes
• Result: no-docs-needed
• Evidence: test/e2e/README.md updates the internal E2E control-plane job inventory. .github/workflows/e2e-branch-validation.yaml and .github/workflows/brev-nightly-e2e.yaml update internal suite comments and options. Follow-ups preserve the credential-migration live seam in cloud-onboard, harden retained test probes, and preserve controller compatibility for retired selectors through ordinary tests. The completed change reorganizes test coverage and CI selectors without changing supported user behavior; no docs/ page or release entry is needed.
• Agent: Codex Desktop documentation writer subagent

DGX Station Hardware Evidence

• Tested on DGX Station
• Tested commit:
• Station profile/scenario:
• Result:
• Supporting evidence:

Verification

• PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
• Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
• Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 1/1 compiled diagnostics contract; 2/2 focused migration lifecycle scenarios; 174/174 risk/controller tests; 74/74 focused E2E workflow/support tests; CLI and root type-checks, Vitest project membership, and E2E semantic phase checks passed.
• Applicable broad gate passed — npm test was attempted but stopped after this macOS host produced unrelated timeout and platform/plugin failures across untouched Linux-service, rollback, installer, and package-contract tests. The affected targeted projects and tests pass.
• Quality Gates section completed with required justifications or waivers
• No secrets, API keys, or credentials committed
• npm run docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

---

Signed-off-by: Prekshi Vyas prekshiv@nvidia.com

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

This change retires dedicated diagnostics and credential E2E jobs, narrows supported suites, relocates diagnostics and credential checks into existing tests, and updates deterministic E2E risk planning from credential-sanitization to cloud-inference.

Changes

E2E workflow retirement and suite updates

Layer / File(s)	Summary
Retire jobs and narrow suite selectors .github/workflows/*, tools/e2e/*, test/brev-*, test/e2e/support/*	Dedicated diagnostics and credential jobs are removed, supported suites are narrowed, and related workflow-boundary, selector, parity, and invocation tests are updated.
Relocate diagnostics and credential coverage test/e2e/live/*, test/package-contract/cli/*, test/onboard-inference-reconciliation.test.ts, test/credential-migration-reconciliation.test.ts, src/lib/security/*	Cloud inference scans sandbox state for credentials, cloud onboarding creates a diagnostics archive, and focused tests cover credential filtering, migration allowlisting, and CLI redaction.
Update deterministic risk selection tools/advisors/risk-plan.mts, test/e2e-recommendations.test.ts, test/pr-*, test/e2e/README.md	Risk-plan floors and related dispatch expectations replace credential-sanitization with cloud-inference and advance the plan version.

Estimated code review effort: 4 (Complex) | ~45 minutes

Suggested labels: area: ci, area: e2e

Suggested reviewers: cv, brandonpelfrey, ericksoa, aasthajh

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	Most retirements are covered, but the required removal of the network-policy zero-presets matrix row is not reflected in the changed files.	Remove the network-policy zero-presets row and ensure its assertion lives in the fast policy tests or existing live-probes journey.
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Out of Scope Changes check	✅ Passed	The edits stay focused on retiring live E2E jobs and migrating their assertions into existing tests and workflow support.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: retiring dedicated Priority 1 E2E jobs.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/7617-retire-dedicated-e2e-jobs

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-code-quality]

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 3c56a35 in the codex/7617-retire-de... branch remains at 96%, unchanged from commit fbfbe68 in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 3c56a35 in the codex/7617-retire-de... branch remains at 80%, unchanged from commit fbfbe68 in the main branch.

Show a code coverage summary of the most impacted files.

File	main fbfbe68	codex/7617-retire-de... 3c56a35	+/-
src/lib/onboard...eway-service.ts	81%	82%	+1%
src/lib/actions...dbox/destroy.ts	93%	95%	+2%
src/lib/credentials/store.ts	59%	62%	+3%
src/lib/actions...e-validation.ts	81%	84%	+3%
src/lib/onboard...shboard-port.ts	90%	93%	+3%
src/lib/actions...x/mcp-bridge.ts	35%	41%	+6%
src/lib/actions...lution-probe.ts	88%	95%	+7%
src/lib/actions...-add-restart.ts	10%	19%	+9%
src/lib/actions...on-readiness.ts	91%	100%	+9%
src/lib/actions...time-command.ts	82%	100%	+18%

---

_{Updated July 27, 2026 22:04 UTC}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/live/cloud-inference.test.ts`:
- Around line 234-275: The sandbox secret scan in the secretProbe command
currently emits matching secret content and suppresses scan failures. Update it
to report only matching file paths or metadata, preserve the exclusion filters,
and ensure grep/find errors propagate while treating only the normal “no
matches” result as clean; remove the unconditional `|| true`. Apply the same
error-preservation requirement to the authProbe command by avoiding discarded
errors.

In `@test/package-contract/cli/debug-cli-command.test.ts`:
- Around line 80-100: Remove the shell if statements from the openshell and
docker fixtures in the debug CLI test, replacing their command-specific behavior
with linear fixture output or case-based dispatch. Preserve the existing outputs
and exit statuses for sandbox list, sandbox ssh-config, docker format checks,
and diagnostic commands while avoiding added shell conditionals.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: cffbb8bf-f133-4b87-8d23-dce0866d9059

📥 Commits

Reviewing files that changed from the base of the PR and between d0a95dc and 204daa4.

📒 Files selected for processing (31)

• .github/workflows/brev-nightly-e2e.yaml
• .github/workflows/e2e-branch-validation.yaml
• .github/workflows/e2e.yaml
• ci/source-shape-test-budget.json
• src/lib/security/credential-filter-secret-patterns.test.ts
• test/brev-nightly-workflow.test.ts
• test/brev-remote-vitest.test.ts
• test/e2e-recommendations.test.ts
• test/e2e/README.md
• test/e2e/brev-e2e.test.ts
• test/e2e/live/cloud-inference.test.ts
• test/e2e/live/cloud-onboard.test.ts
• test/e2e/live/credential-migration.test.ts
• test/e2e/live/credential-sanitization.test.ts
• test/e2e/live/diagnostics.test.ts
• test/e2e/mock-parity.json
• test/e2e/support/dockerhub-auth-workflow-boundary.test.ts
• test/e2e/support/e2e-operations-workflow-boundary.test.ts
• test/e2e/support/e2e-workflow.test.ts
• test/e2e/support/live-vitest-invocation.test.ts
• test/e2e/support/upload-e2e-artifacts-workflow-boundary.test.ts
• test/onboard-inference-reconciliation.test.ts
• test/package-contract/cli/debug-cli-command.test.ts
• test/pr-e2e-gate-dispatch-recovery.test.ts
• test/pr-e2e-gate-fork-approval.test.ts
• test/pr-e2e-gate-workflow.test.ts
• test/pr-e2e-gate.test.ts
• test/pr-risk-plan.test.ts
• tools/advisors/risk-plan.mts
• tools/e2e/brev-remote-vitest.mts
• tools/e2e/workflow-boundary.mts

💤 Files with no reviewable changes (9)

• test/e2e/live/diagnostics.test.ts
• test/e2e/live/credential-sanitization.test.ts
• tools/e2e/brev-remote-vitest.mts
• test/e2e/live/credential-migration.test.ts
• ci/source-shape-test-budget.json
• .github/workflows/e2e.yaml
• tools/e2e/workflow-boundary.mts
• test/e2e/support/e2e-workflow.test.ts
• test/e2e/mock-parity.json

[github-actions]

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

• GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
• Nemotron 3 Ultra (second opinion): Failed after a partial review · low confidence · 5 blockers · 2 warnings · 1 suggestion

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-onboard, credential-sanitization, security-posture, cloud-inference, credential-migration, diagnostics, inference-routing, network-policy

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

[coderabbitai]

♻️ Duplicate comments (1)

test/e2e/live/cloud-inference.test.ts (1)

252-293: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Per-file error checks are dead code — pipe-into-while runs in a subshell.

printf '%s\n' "$filtered" | while IFS= read -r file; do ... done places the loop body on the right side of a pipe, so in POSIX sh it executes in a subshell. The exit "$match_status"/exit "$unstripped_status" calls inside only terminate that subshell — the outer script (and the for dir loop) continues unaffected, and the pipeline's own exit status is never captured. Any real grep failure while scanning an individual matched file (permission error, unreadable file, etc.) is therefore silently discarded, and secretProbe.exitCode will still report 0 with empty stdout — a false "clean" result for a security-sensitive credential-boundary check. This is a continuation of the previously flagged "swallows scan failures" concern; the secret-content emission part has been fixed (only filenames are now printed), but the failure-propagation guarantee is not actually achieved due to this shell semantics gotcha.

Route the per-file loop through a temp file (or another non-pipe input) so it runs in the current shell and exit actually propagates:

🔒 Proposed fix using a temp file instead of a pipe

        '       case "$filter_status" in',
-    `         0) printf '%s\\n' "$filtered" | while IFS= read -r file; do`,
+    `         0) tmp_filtered=$(mktemp)`,
+    `            printf '%s\\n' "$filtered" > "$tmp_filtered"`,
+    `            while IFS= read -r file; do`,
     `              matching_lines=$(grep -IE 'nvapi-|ghp_|npm_' "$file")`,
     "              match_status=$?",
     '              case "$match_status" in',
     `                0) printf '%s' "$matching_lines" | grep -qv 'STRIPPED'`,
     "                   unstripped_status=$?",
     '                   case "$unstripped_status" in',
     `                     0) printf '%s\\n' "$file" ;;`,
     "                     1) ;;",
-    '                     *) exit "$unstripped_status" ;;',
+    '                     *) rm -f "$tmp_filtered"; exit "$unstripped_status" ;;',
     "                   esac",
     "                   ;;",
     "                1) ;;",
-    '                *) exit "$match_status" ;;',
+    '                *) rm -f "$tmp_filtered"; exit "$match_status" ;;',
     "              esac",
-    "            done",
+    `            done < "$tmp_filtered"`,
+    `            rm -f "$tmp_filtered"`,
     "            ;;",

Based on a previous review comment on this function noting the scan probes "discard errors" and can let "an unreadable state directory... pass as clean," which is only partially resolved by the current implementation.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/live/cloud-inference.test.ts` around lines 252 - 293, Update the
secretScanCommand loop around filtered to avoid piping printf into while, since
that runs the loop in a subshell and discards per-file failures. Feed the
filtered filenames through a temporary file or another non-pipeline input so the
loop executes in the current shell and exit "$match_status" and exit
"$unstripped_status" propagate to secretProbe.exitCode; preserve the existing
filename filtering and STRIPPED handling.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@test/e2e/live/cloud-inference.test.ts`:
- Around line 252-293: Update the secretScanCommand loop around filtered to
avoid piping printf into while, since that runs the loop in a subshell and
discards per-file failures. Feed the filtered filenames through a temporary file
or another non-pipeline input so the loop executes in the current shell and exit
"$match_status" and exit "$unstripped_status" propagate to secretProbe.exitCode;
preserve the existing filename filtering and STRIPPED handling.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3b22fc1d-36ae-449f-a83b-642ed71e340f

📥 Commits

Reviewing files that changed from the base of the PR and between 82cf50c and 81c4207.

📒 Files selected for processing (2)

• test/e2e/live/cloud-inference.test.ts
• test/package-contract/cli/debug-cli-command.test.ts

🚧 Files skipped from review as they are similar to previous changes (1)

• test/package-contract/cli/debug-cli-command.test.ts

[prekshivyas]

Addressed the latest CodeRabbit shell-semantics follow-up in 77a3bbf. The filtered path list now feeds the per-file loop through a private temporary file, so the loop runs in the current shell and non-no-match grep statuses propagate to the probe exit code. Direct shell validation covered syntax, exclusions, STRIPPED handling, path-only output, and a simulated per-file grep status 2.

[prekshivyas]

Resolved the CLI shard regression in 0ffe7cc. The latest main branch added controller compatibility for retired selectors; this PR now registers credential-migration, credential-sanitization, and diagnostics with precise ordinary-test replacements and updates the candidate workflow gate. Focused planner/compatibility/workflow tests pass 74/74, and each P1 replacement command passes (migration 1/1, credential filter 13/13, diagnostics 1/1).

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/credential-migration-reconciliation.test.ts`:
- Around line 73-78: Update the reconciliation test around setupInference() to
stop manually calling removeLegacyCredentialsFile(). Exercise the actual mocked
gateway registration flow for both outcomes: verify the legacy credentials file
remains after registration failure and is removed after successful registration,
using assertions that validate production cleanup behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 895e1edd-c1bc-4811-a332-ad5b220683c9

📥 Commits

Reviewing files that changed from the base of the PR and between 77a3bbf and 0ffe7cc.

📒 Files selected for processing (7)

• .github/workflows/e2e.yaml
• test/credential-migration-reconciliation.test.ts
• test/e2e/support/retired-selector-compatibility.test.ts
• test/e2e/support/workflow-plan.test.ts
• test/onboard-inference-reconciliation.test.ts
• tools/e2e/retired-selector-compatibility.mts
• tools/e2e/workflow-boundary.mts

💤 Files with no reviewable changes (1)

• test/onboard-inference-reconciliation.test.ts

🚧 Files skipped from review as they are similar to previous changes (1)

• tools/e2e/workflow-boundary.mts