Skip to content

fix(ci): authenticate exact-base qualification evidence - #10842

Closed
prekshivyas wants to merge 16 commits into
mainfrom
codex/fix-exact-base-qualification-receipts
Closed

fix(ci): authenticate exact-base qualification evidence#10842
prekshivyas wants to merge 16 commits into
mainfrom
codex/fix-exact-base-qualification-receipts

Conversation

@prekshivyas

@prekshivyas prekshivyas commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Outcome

PR qualification now consumes authenticated, exact-base evidence instead of relying on stale run metadata or unbound artifacts. OpenShell SDK selection and managed runtime activation both bind candidate, base, controller, run attempt, artifact digests, and cleanup evidence before they can affect the result.

Reason

Qualification of PR #10790 exposed two infrastructure gaps: the SDK consumer could reject a valid package because GitHub retained stale pull-request base metadata, and managed-image activation had no authenticated exact-base comparison for separating candidate regressions from base or infrastructure failures.

Related issues

Relates to #10790

Changes

  • Add producer and consumer receipts for OpenShell SDK packages, with current-PR API checks, exact run-name base identity, artifact digest verification, bounded archive downloads, a one-time resolver controlled by the exact PR base, and trusted receipt resolution after the feature reaches the base branch.
  • Keep GitHub Packages access in exact-base jobs and pass only the verified OpenShell SDK archive artifact into candidate execution.
  • Add candidate and exact-base managed-runtime activation receipts plus a manual base-qualification workflow that replays the same OpenClaw scenario on a fresh runner and classifies pass, candidate failure, base failure, or infrastructure failure.
  • Bind every managed-runtime controller checkout and receipt to the exact PR base commit, even when GitHub reports a different workflow source commit.
  • Refresh managed-image publication and exact-base qualification when a PR changes base, while ignoring unrelated edited events.
  • Emit a validated linux/amd64 runtime catalog from the selected managed-image cohort and bind image identity, OpenShell version, workflow source, evidence files, and cleanup proof.
  • Protect the mechanisms with adversarial E2E-support and security workflow-contract tests, and document the exact-base comparison contract and Pi candidate workflow ownership in the E2E guide.

Verification

  • Contributor validation: npm run validate:pr passed after the final upstream refresh, including pre-commit, commit-msg, and pre-push checks.
  • Tests: 34 PR workflow contract tests, 10 OpenShell SDK receipt tests, 17 managed-runtime comparison tests, and 34 managed-image workflow contract tests passed; repository checks and the source-shape policy check passed.
  • Broad gate: npm run validate:pr passed on commit 2e280afd43fa500a12800cd332223b2a17adb063 against upstream main commit 16c21afdc858f5ca549b73d5e1fa6624bfc23c8b.
  • Documentation: Updated the E2E guide to name the workflow and job that publish Pi candidate images and to document paired replay selectors; the shared writing contract was applied.
  • Secrets review: The diff contains no secrets, API keys, or credentials.

Signed-off-by: Prekshi Vyas prekshiv@nvidia.com

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 2, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall line coverage in commit 2e280af in the codex/fix-exact-base... branch remains at 96%, unchanged from commit 16c21af in the main branch.


Updated September 02, 2026 07:52 UTC

Comment thread tools/e2e/openshell-sdk-package-receipt.mts Fixed
@prekshivyas prekshivyas closed this Sep 2, 2026
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>

# Conflicts:
#	.github/workflows/managed-images.yaml
#	.github/workflows/managed-runtime-base-qualification.yaml
#	test/e2e/README.md
#	test/e2e/support/managed-runtime-comparison.test.ts
#	test/e2e/support/openshell-sdk-package-receipt.test.ts
#	test/inference/managed/managed-image-publication-workflow.test.ts
#	tools/e2e/managed-runtime-comparison.mts
#	tools/e2e/openshell-sdk-package-receipt.mts
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
@prekshivyas prekshivyas reopened this Sep 2, 2026
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Comment thread tools/e2e/openshell-sdk-package-receipt.mts Fixed
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Comment thread tools/e2e/openshell-sdk-package-receipt.mts Fixed
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor finished for commit 2e280af. Include the Advisor findings in the complete PR feedback collection. Verify and group valid findings before repair.

All previous runs

@prekshivyas

Copy link
Copy Markdown
Collaborator Author

Superseded by #10829; the fix and Advisor follow-up now live there.

@prekshivyas prekshivyas closed this Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants