ci(advisor): preserve E2E lane disagreements - #8017
Merged
Merged
Conversation
Contributor
📝 WalkthroughWalkthroughThe PR adds trusted E2E recommendations to advisor lane reports, publishes second-opinion-only selectors as advisory disagreements, and derives managed-startup E2E jobs from changed files. It updates risk-plan versioning, documentation, and regression tests. ChangesE2E advisor and risk-plan updates
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested labels: Suggested reviewers: Sequence Diagram(s)sequenceDiagram
participant PrimaryLane
participant SecondOpinionLane
participant CommentRenderer
PrimaryLane->>CommentRenderer: provide normalized E2E recommendations
SecondOpinionLane->>CommentRenderer: provide normalized E2E recommendations
CommentRenderer->>CommentRenderer: filter, deduplicate, and limit second-opinion-only selectors
CommentRenderer-->>PrimaryLane: publish primary guidance and advisory disagreement
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Contributor
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tools/pr-review-advisor/comment.mts`:
- Around line 488-499: Update renderSecondOpinionE2eRecommendations to return no
recommendations when either completed lane has partial: true, while preserving
the existing status and E2E presence checks. Add a regression case covering a
completed partial second-opinion lane with a valid full-e2e selection and verify
that no disagreement is published.
- Around line 730-772: Update trustedLaneE2eRecommendations and
trustedLaneE2eTier so coverage and target arrays filter out malformed entries,
including null and non-record values, before trustedCoverageIds or
trustedTargetIds reads id, workflow, selectorType, or other properties. Preserve
valid entries and existing recommendation behavior, and add fixtures covering
malformed coverage and target entries to verify invalid second-opinion artifacts
are ignored.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 8410615d-98b7-46c6-a17f-ea4f0fd6fecf
📒 Files selected for processing (5)
test/pr-review-advisor-comment-cli.test.tstest/pr-risk-plan.test.tstools/advisors/risk-plan.mtstools/pr-review-advisor/README.mdtools/pr-review-advisor/comment.mts
Contributor
PR Review Advisor — No blocking findings reportedAdvisor assessment: No blocking advisor findings reported Model lanes
Nemotron output stays in workflow artifacts and does not change the assessment above. E2E guidanceAdvisory only. E2E / PR Gate selects and runs jobs independently. Recommended E2E: This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge. |
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
cv
approved these changes
Jul 31, 2026
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tools/pr-review-advisor/comment.mts`:
- Around line 598-609: Guard the collection input in both trustedCoverageIds and
trustedTargetIds with an array check before calling flatMap, returning an empty
result for non-array values such as objects or strings. Preserve filtering of
malformed entries and valid allowed IDs, and add focused tests covering
non-array collections, malformed entries, valid arrays, and false-positive
behavior.
- Around line 622-630: Update trustedTargetIds and its trustedTuple validation
to require item.selectorType to be included in inventory.selectorTypes before
accepting a matching ID, rather than relying only on the hard-coded
selector-type cases. Preserve the existing workflow, required, and tuple
behavior, and add coverage for both an allowed and an excluded selector type.
Derive the inventory from the canonical source where applicable.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 9faa56a9-cad6-4151-a33f-aef0bb9cb7e8
📒 Files selected for processing (2)
test/pr-review-advisor-comment-cli.test.tstools/pr-review-advisor/comment.mts
🚧 Files skipped from review as they are similar to previous changes (1)
- test/pr-review-advisor-comment-cli.test.ts
Comment on lines
+598
to
609
| items: unknown[] | undefined, | ||
| inventory: TrustedE2eRecommendationInventory, | ||
| ): string[] { | ||
| const allowedIds = new Set([...inventory.allowedJobIds, ...inventory.liveSupportedTargetIds]); | ||
| const seen = new Set<string>(); | ||
| return (items ?? []).flatMap((item) => { | ||
| if (!isRecord(item)) return []; | ||
| const id = item.id; | ||
| if (!id || !allowedIds.has(id) || seen.has(id)) return []; | ||
| if (typeof id !== "string" || !allowedIds.has(id) || seen.has(id)) return []; | ||
| seen.add(id); | ||
| return [id]; | ||
| }); |
Contributor
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Reject malformed E2E collections before calling .flatMap.
items ?? [] handles only null and undefined. A malformed artifact such as requiredTests: {} or targets.required: "full-e2e" reaches .flatMap and throws. This can abort comment rendering instead of ignoring the invalid artifact.
Guard the collection itself in both helpers. Add tests for non-array collections, malformed entries, and valid arrays.
As per path instructions, review advisors as product code and require focused tests for detection and false-positive behavior.
Suggested guard
- return (items ?? []).flatMap((item) => {
+ const entries = Array.isArray(items) ? items : [];
+ return entries.flatMap((item) => {Apply this in both trustedCoverageIds and trustedTargetIds.
Also applies to: 621-630
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tools/pr-review-advisor/comment.mts` around lines 598 - 609, Guard the
collection input in both trustedCoverageIds and trustedTargetIds with an array
check before calling flatMap, returning an empty result for non-array values
such as objects or strings. Preserve filtering of malformed entries and valid
allowed IDs, and add focused tests covering non-array collections, malformed
entries, valid arrays, and false-positive behavior.
Source: Path instructions
Comment on lines
+622
to
+630
| if (!isRecord(item)) return []; | ||
| const id = item.id; | ||
| const selectorType = item.selectorType; | ||
| if (!id || item.workflow !== inventory.workflow || item.required !== required) return []; | ||
| if ( | ||
| typeof id !== "string" || | ||
| item.workflow !== inventory.workflow || | ||
| item.required !== required | ||
| ) | ||
| return []; |
Contributor
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Use the canonical selector-type allowlist.
TrustedE2eRecommendationInventory exposes selectorTypes, but trustedTargetIds does not check it. The trustedTuple logic hard-codes "all", "job", and "target". If the inventory excludes one type, a matching ID is still accepted and published as trusted.
Validate selectorType against inventory.selectorTypes before evaluating the tuple. Add tests for both an allowed and an excluded selector type.
As per path instructions, derive inventories from a canonical source where possible.
Suggested validation
const selectorType = item.selectorType;
if (
typeof id !== "string" ||
+ typeof selectorType !== "string" ||
+ !inventory.selectorTypes.some((allowedType) => allowedType === selectorType) ||
item.workflow !== inventory.workflow ||
item.required !== required
) {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tools/pr-review-advisor/comment.mts` around lines 622 - 630, Update
trustedTargetIds and its trustedTuple validation to require item.selectorType to
be included in inventory.selectorTypes before accepting a matching ID, rather
than relying only on the hard-coded selector-type cases. Preserve the existing
workflow, required, and tuple behavior, and add coverage for both an allowed and
an excluded selector type. Derive the inventory from the canonical source where
applicable.
Source: Path instructions
23 tasks
cv
pushed a commit
that referenced
this pull request
Jul 31, 2026
<!-- markdownlint-disable MD041 --> ## Summary Harden the PR Review Advisor publisher against malformed E2E collections and selector types outside the canonical inventory. This is the post-merge follow-up to the final CodeRabbit findings on #8017. ## Related Issue Follow-up to #8017 and #8016. ## Changes - Treat coverage and target collections as untrusted input and require arrays before iteration. - Require target selector types to exist in the canonical E2E inventory before tuple validation. - Add focused positive and negative tests for malformed collections and allowed or excluded selector types. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates <!-- Check one tests line and one docs line. Check other lines when applicable. Add every requested justification or approval reference. --> - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: This validation hardening preserves the trusted selector allowlisting contract documented by #8017 and does not change valid output or UI text. - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review <!-- Required for code and documentation changes after the changes and applicable validation are complete. Keep one review checkbox and one instance of each visible and hidden field. For Evidence, list changed documentation paths. For documentation-only changes, also state that the writing rules and documentation style were reviewed. For other results, explain why no documentation change is needed or why the review is blocked. For Agent, use a consistent product and surface name, such as Codex Desktop, Codex CLI, Claude Code, or Cursor. After committing all review changes, put git rev-parse --short HEAD and git rev-parse --short HEAD:AGENTS.md in the hidden metadata below. Rerun the review and refresh that metadata after any new commit. This receipt is advisory during the data-collection pilot. --> - [x] Documentation writer subagent reviewed the completed changes - Result: `docs-not-needed` - Evidence: Existing `tools/pr-review-advisor/README.md` documents publisher-side coverage-ID and selector-tuple allowlisting; focused integration tests passed 78/78; CLI type-check, Biome, and `git diff --check` passed. - Agent: Codex Desktop <!-- docs-review-head-sha: 9a8fd47 --> <!-- docs-review-agents-blob-sha: c052d60 --> ## DGX Station Hardware Evidence <!-- Required only when scripts/prepare-dgx-station-host.sh changes. Maintainers must review the linked evidence before approving or merging. This is human-reviewed evidence, not authenticated hardware provenance. Exceptional bypasses use existing repository governance and must be documented on the PR. --> - [ ] Tested on DGX Station - Tested commit: Not applicable. - Station profile/scenario: Not applicable. - Result: Not applicable; this change does not modify `scripts/prepare-dgx-station-host.sh`. - Supporting evidence: Not applicable. ## Verification <!-- Check each applicable item only when supported by the requested evidence. Run targeted tests once per relevant change set and rerun after later edits or hook autofixes that can affect the tested behavior. Do not rerun hook-covered checks. --> - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: `npx vitest run --project integration test/pr-review-advisor-comment-cli.test.ts test/pr-risk-plan.test.ts` passed 78 tests. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: Not applicable; focused advisor tests cover the validation boundary. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the style guide (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- <!-- DCO sign-off is required in this PR description, and every commit must appear as Verified in GitHub. --> Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved handling of malformed coverage data to prevent invalid entries from appearing in review comments. * Restricted displayed job selectors to trusted, supported values and excluded workflow selectors. * Added validation for selector types and input collections before processing. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
23 tasks
23 tasks
senthilr-nv
added a commit
that referenced
this pull request
Aug 1, 2026
<!-- markdownlint-disable MD041 --> ## Summary Adds the canonical dated changelog entry for `v0.0.100` so the maintainer release plan can verify the pre-tag documentation prerequisite. The entry summarizes the user-facing changes merged since `v0.0.99` and links to the relevant guides. ## Changes - Add `docs/changelog/2026-07-31.mdx` with the exact `## v0.0.100` heading. - Cover restored OpenClaw pairing, transactional replacement, Deep Agents Code, onboarding recovery, lifecycle cleanup, Hermes builds, host provenance, documentation, and trusted E2E evidence. - Distinguish active Docker and Kubernetes runtime-bundle enforcement from the still-inactive managed shared-state transaction foundation. ## Source Coverage The release entry maps the doc-impacting merged PRs in the `v0.0.99..main` release range to `docs/changelog/2026-07-31.mdx`: #8021, #8024, #7973, #8028, #7947, #7788, #7884, #8023, #7969, #8020, #7989, #8000, #7907, #7942, #7567, #8013, #7955, #8017, #8014, #8015, #7629, #7644, #7821, #7971, and #7991. PR #7974 was reviewed after the final rebase and excluded because it changes internal maintainer-skill attribution policy and tests only; it does not change a user-facing product or documentation surface. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [x] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [ ] Tests added or updated for changed behavior - [x] Existing tests cover changed behavior — justification: the changelog contract test validates the dated entry, version heading, SPDX form, and route constraints. - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `docs-updated` - Evidence: `docs/changelog/2026-07-31.mdx`; exact-head review passed for `6093f44f`; writing rules and documentation style reviewed; `npx vitest run test/changelog-docs.test.ts` passed 6/6; `npm run docs` passed with zero Fern errors and two generic Fern upgrade notices. - Agent: Codex Desktop <!-- docs-review-head-sha: 6093f44 --> <!-- docs-review-agents-blob-sha: 3dd7c24 --> ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: Not applicable; no DGX Station host script changed. - Station profile/scenario: Not applicable. - Result: Not applicable. - Supporting evidence: Not applicable. ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: `npx vitest run test/changelog-docs.test.ts` passed 6/6 at `6093f44f`. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: Not applicable to a dated prose-only release entry. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — validation passed with zero errors; Fern emitted two generic upgrade notices. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) — the changelog entry has the required parser-safe MDX SPDX header; dated changelog entries intentionally do not use page frontmatter. --- Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added release notes for v0.0.100. * Documented improvements to restore pairing, sandbox replacement, onboarding recovery, lifecycle cleanup, runtime handling, build support, host readiness, and end-to-end validation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The PR Review Advisor now shows trusted E2E selections that only the completed second-opinion lane selected.
The primary assessment and E2E guidance remain authoritative, while managed-startup delivery changes receive the focused startup and authentication E2E floor identified after v0.0.99.
Related Issue
Fixes #8016
Changes
device-auth-health,issue-4462-scope-upgrade-approval, andopenclaw-inference-switchfor managed-startup delivery paths.tools/pr-review-advisor/README.md.Type of Change
Quality Gates
Documentation Writer Review
docs-updatedtools/pr-review-advisor/README.md; focused integration tests passed 77/77;npm run docspassed with 0 errors and 2 Fern warnings;git diff --checkpassed.DGX Station Hardware Evidence
scripts/prepare-dgx-station-host.sh.Verification
Signed-off-by:line and every commit appears asVerifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run validate:prpassed after refreshingorigin/mainwhen hooks were skipped or unavailablenpx vitest run --project integration test/pr-review-advisor-comment-cli.test.ts test/pr-risk-plan.test.tspassed 77 tests.npm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes — command/result: Not applicable; focused advisor and risk-plan tests cover the changed behavior.npm run docsbuilds without warnings (doc changes only)Signed-off-by: Prekshi Vyas prekshiv@nvidia.com
Summary by CodeRabbit