Skip to content

fix(e2e): preserve catalog selector stdout - #10794

Merged
prekshivyas merged 5 commits into
mainfrom
codex/fix-pr-image-selector-output
Sep 1, 2026
Merged

fix(e2e): preserve catalog selector stdout#10794
prekshivyas merged 5 commits into
mainfrom
codex/fix-pr-image-selector-output

Conversation

@prekshivyas

@prekshivyas prekshivyas commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Outcome

Manual pull-request E2E can consume an exact candidate managed-image catalog again. Artifact retry evidence remains visible in GitHub Actions, while the selector stdout stays a single machine-readable base-cohort or candidate-catalog token.

Reason

The exact-candidate resolver introduced by #10745 downloads three managed-image contracts through the bounded artifact reader. That reader emitted one audit record per download to stdout, so the workflow command substitution received four lines instead of the single selector token and failed with PR managed-image selection is invalid before candidate code ran.

The failure reproduced in the focused Hermes Discord dispatch for #10682: run 33535511885.

Related issues

Refs #10682
Relates to #10745

Changes

  • Send default artifact-attempt evidence to stderr so machine-readable stdout remains unambiguous.
  • Preserve the existing injected logger contract for callers and tests that collect structured retry evidence.
  • Add regression coverage proving a successful default download writes no stdout and retains its attempt record on stderr.
  • Stabilize the existing readiness deadline-budget test with its supported injected clock after CI exposed a 1 ms wall-clock race. Production readiness behavior is unchanged.

Verification

  • Exact selector replay against PR fix(policy): restrict Hermes Discord binary access #10682 head 1e6b8c07d78f74106cd5968797df6175b7815b6e and Images run 33532697986 — stdout was exactly candidate-catalog; all three attempt records were on stderr; the 1,754-byte catalog validated.
  • npx vitest run --project e2e-support test/e2e/support/exact-artifact-download.test.ts test/e2e/support/pr-managed-image-publication.test.ts test/e2e/support/pr-managed-image-workflow-boundary.test.ts — 54 tests passed on final head.
  • npx vitest run --project cli src/lib/onboard/sandbox-readiness-tracing.test.ts — 41 tests passed with 1 intentional skip on final head.
  • Readiness deadline test isolated replay — passed 5 consecutive runs after injecting the deterministic clock.
  • npm run test:changed — affected growth, readiness, and E2E-support coverage passed.
  • npm run validate:pr — pre-commit, commit-message, pre-push, repository, source-shape, growth, secret-scan, and CLI type-check gates passed after merging upstream main.
  • Upstream main commit c67be3984f2d629ed767393ddefd32c403efb2f9 is an ancestor of exact head 9b8e851b508591bd9f7cff5e7237f006f17e69e7.
  • git diff --check origin/main...HEAD passed.
  • Diff review — no secrets, API keys, credentials, or documentation changes.

Review notes

The first CI run exposed a pre-existing one-millisecond race in sandbox-readiness-tracing.test.ts: CLI shard 6 observed 5999 instead of 6000 because the test used wall time even though the production waiter already supports an injected clock. The test-only clock injection removes that nondeterminism without changing production behavior.

An Advisor code-reduction note about parseArtifactReadEvidence was checked against the exact base and is not in this PR's diff; the helper already existed on main, so no unrelated churn was added.

GitHub Actions captures stderr, so the bounded attempt evidence remains available without contaminating selector stdout. After this fix reaches main, the exact-head focused Hermes Discord run for #10682 should be dispatched fresh rather than rerunning the pre-execution failure.


Signed-off-by: Prekshi Vyas prekshiv@nvidia.com

Summary by CodeRabbit

  • Bug Fixes
    • Download retry and status messages now appear in standard error output instead of standard output.
    • Successful artifact downloads continue to work as expected.
    • Improved cleanup after download operations to prevent lingering console behavior.
    • Enhanced reliability of sandbox readiness checks for more consistent startup behavior.

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
@github-code-quality

github-code-quality Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall line coverage in commit 9b8e851 in the codex/fix-pr-image-s... branch remains at 96%, unchanged from commit c67be39 in the main branch.

TypeScript / code-coverage/cli

The overall line coverage in commit 9b8e851 in the codex/fix-pr-image-s... branch remains at 84%, unchanged from commit 2b7ae7f in the main branch.

Show a line coverage summary of the most impacted files.
File main 2b7ae7f codex/fix-pr-image-s... 9b8e851 +/-
src/lib/onboard...press-resume.ts 78% 76% -2%
src/lib/onboard...tup-nim-flow.ts 93% 92% -1%
src/lib/onboard...tup-nim-vllm.ts 90% 90% 0%
src/lib/sandbox...rce-identity.ts 82% 82% 0%
src/lib/state/s...le-key-merge.ts 100% 100% 0%
src/lib/onboard...uild-context.ts 74% 75% +1%
src/lib/inferen.../vllm-models.ts 93% 95% +2%
src/lib/inferen...verification.ts 0% 97% +97%

Updated September 01, 2026 19:26 UTC

@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2f3c083e-54ac-4b2a-a789-001a8fb67a75

📥 Commits

Reviewing files that changed from the base of the PR and between 5dc1765 and 9b8e851.

📒 Files selected for processing (1)
  • src/lib/onboard/sandbox-readiness-tracing.test.ts

Included review availability: Your plan provides up to 12 included reviews per hour; 8 remain after this review.


📝 Walkthrough

Walkthrough

The default artifact download logger now writes retry and download evidence to stderr instead of stdout. End-to-end coverage verifies successful downloading and output separation. A readiness waiter test now uses a fixed clock.

Changes

Artifact download and test determinism

Layer / File(s) Summary
Default logging and validation
tools/e2e/exact-artifact-download.mts, test/e2e/support/exact-artifact-download.test.ts
The default log callback uses console.error. The end-to-end test verifies stderr output, stdout silence, one request, successful cleanup, and unchanged explicit callbacks.
Deterministic readiness test clock
src/lib/onboard/sandbox-readiness-tracing.test.ts
The Docker-driver readiness waiter test supplies a fixed clock returning 0.
Estimated code review effort: 2 (Simple) ~10 minutes

Merge Risk: ⚪ Minimal · up to 9b8e8

The change keeps the selector output machine-readable while preserving retry diagnostics and makes a readiness test deterministic; no actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: rsliter

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 3 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the E2E fix and the primary behavior change: preserving machine-readable catalog selector output on stdout.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-pr-image-selector-output

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor finished for commit 9b8e851. Include the Advisor findings in the complete PR feedback collection. Verify and group valid findings before repair.

All previous runs

@rsliter rsliter left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 9b8e851 against c67be39. The default artifact logger now preserves the selector’s single-token stdout contract while retaining bounded attempt evidence on stderr, and the focused regression verifies both channels without exposing the credential. The unrelated readiness test change only injects the supported clock. Merged #10745 establishes the workflow contract, no competing open fix exists, automated findings are clear, every required check passes, and the nine-category security review found no blocking weakness.

@prekshivyas
prekshivyas merged commit 4518958 into main Sep 1, 2026
116 checks passed
@prekshivyas
prekshivyas deleted the codex/fix-pr-image-selector-output branch September 1, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants