Skip to content

fix(shields): preserve managed MCP policies - #8141

Merged
cv merged 1 commit into
codex/shields-deadline-safety-7952from
codex/shields-mcp-policy-reconciliation-7952
Aug 3, 2026
Merged

fix(shields): preserve managed MCP policies#8141
cv merged 1 commit into
codex/shields-deadline-safety-7952from
codex/shields-mcp-policy-reconciliation-7952

Conversation

@jyaunches

@jyaunches jyaunches commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary

nemoclaw shields down replaced the complete live OpenShell policy and dropped generated policy entries for registered Model Context Protocol (MCP) servers. This change reconciles only exact NemoClaw-managed MCP entries during Shields transitions, so a surviving server remains reachable while removed servers stay removed.

Stacked on prerequisite #8130, which makes Shields deadline recovery serialize with lifecycle mutations without signaling the lock owner, this focused fix supersedes the MCP portion of #7980.

Related Issue

Fixes #7952

Changes

  • Prove managed MCP policy ownership from exact agreement between the sandbox registry, committed generated-policy record, and live gateway policy.
  • Save the owned MCP key manifest with the Shields snapshot, remove snapshot-time managed entries during restoration, and overlay only current exact entries.
  • Fail closed on ambiguous, stale, incomplete, malformed, or legacy ownership during manual transitions. At an expired deadline, omit unproven managed MCP entries and audit the omission instead of extending the Shields-down window.
  • Preserve current managed MCP entries when building the permissive runtime policy, while rejecting an unreadable or ambiguous live policy.
  • Clean staged runtime policy files across early failure paths.
  • Restore the Hermes live regression assertions at the actual failure boundary and around the unrelated server lifecycle.
  • Document MCP policy reconciliation for manual and automatic restoration.

Failure Timing and Hermes Upgrade Context

The original journey had a hidden Shields lifecycle between the first successful call to server A and the later lifecycle for server B:

  1. Run shields up.
  2. Restart the Hermes gateway.
  3. Run shields down.
  4. Exercise the configuration rollback path.
  5. Add and remove B.
  6. Call A.

Boundary instrumentation recorded in #7952 showed that A remained healthy through Shields up and the gateway restart. It became unusable immediately after Shields down, which dropped A's generated MCP policy. The later failure after B was removed was only where the test noticed the already-broken route; B removal was a misleading correlation.

This surfaced during the Hermes upgrade work because new coverage and upgrade repairs landed nearly back-to-back:

The corrected regression order is:

  1. Run shields up.
  2. Restart the Hermes gateway.
  3. Run shields down.
  4. Call A immediately.
  5. Exercise the configuration rollback path.
  6. Add B, prove the DNS-rebinding connection is denied, remove B, and verify that A's managed policy is unchanged while B's policy is gone.
  7. Call A before the later explicit restart.
  8. Capture the authenticated rediscovery offset.
  9. Run mcp restart A without resupplying the secret.
  10. Call A and verify authenticated rediscovery.

Whole-policy Shields replacement and the filesystem-only runtime merge predate the Hermes upgrade. This is a latent NemoClaw Shields policy-composition defect detected by expanded Hermes regression coverage, not a Hermes upgrade regression.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification:
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Independent exact-head Codex security review passed all nine categories at 18039569796d6ac7604de032edb7abf84f2c73c4; no findings.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit: Not applicable; scripts/prepare-dgx-station-host.sh is unchanged.
  • Station profile/scenario: Not applicable.
  • Result: Not applicable.
  • Supporting evidence: Not applicable.

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: Focused CLI 123/123, integration 11/11, E2E support 13/13, npm run typecheck:cli, npm run checks:repository, test-size guardrail, E2E semantic phase plans, and serial npm run test:changed 674/674 passed.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: Standard PR CI run 30824992396 passed. One inherited 50 ms lifecycle-lock assertion timing flake passed on the failed-job rerun without a code change.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

npm run docs passed with 0 errors and 2 existing Fern warnings, so the warning-free checkbox remains unchecked. No new documentation pages were added.

Trusted E2E run 30826792180 passed all 10 selected checks: cloud inference, cloud onboard, security posture, inference routing, MCP bridge, MCP bridge dev, network policy, onboard repair, onboard resume, and OpenShell credential-generation window. The primary review advisor reported no findings. Nemotron completed after retrying a protocol-only failure; its one test warning requested the exact transition/state ownership-mismatch deadline regression already present in src/lib/shields/policy-transition.test.ts, which passed.


Signed-off-by: Julie Yaunches jyaunches@nvidia.com

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e93d4269-9e9b-46dc-a253-651418880da4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@jyaunches jyaunches added bug-fix PR fixes a bug or regression area: docs Documentation, examples, guides, or docs build area: policy Network policy, egress rules, presets, or sandbox policy area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery area: security Security controls, permissions, secrets, or hardening integration: hermes Hermes integration behavior v0.0.101 labels Aug 3, 2026
@jyaunches jyaunches self-assigned this Aug 3, 2026
@github-code-quality

github-code-quality Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 1803956 in the codex/shields-mcp-po... branch remains at 96%, unchanged from commit d6ac402 in the codex/shields-deadli... branch.

TypeScript / code-coverage/cli

The overall coverage in commit 1803956 in the codex/shields-mcp-po... branch remains at 81%, unchanged from commit d6ac402 in the codex/shields-deadli... branch.

Show a code coverage summary of the most impacted files.
File codex/shields-deadli... d6ac402 codex/shields-mcp-po... 1803956 +/-
src/lib/shields...sive-runtime.ts 96% 78% -18%
src/lib/onboard...eway-service.ts 86% 82% -4%
src/lib/shields...nsition-lock.ts 88% 85% -3%
src/lib/onboard.ts 31% 31% 0%
src/lib/actions...ridge-policy.ts 66% 69% +3%
src/lib/state/m...-acquisition.ts 75% 82% +7%
src/lib/shields...imer-control.ts 80% 90% +10%
src/lib/onboard...eway-cutover.ts 70% 90% +20%
src/lib/shields...y-transition.ts 0% 94% +94%
src/lib/state/m...er-authority.ts 0% 100% +100%

Updated August 03, 2026 15:16 UTC

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 1 warning · 0 suggestions
  • Model comparison: normalized findings differ; normalized terminology decisions differ; normalized E2E selections match; Nemotron reported the same number of blockers, 1 more warning, the same number of suggestions.
8 terminology differences from the second opinion

Advisory only. These are normalized differences from the primary terminology receipt.

  • managed MCP policy at docs/manage-sandboxes/runtime-controls.mdx:118: primary classified it as established; the second opinion classified it as justified.
  • ownership manifest at src/lib/shields/index.ts:2796: selected only by the second-opinion lane as established.
  • unproven managed MCP at docs/manage-sandboxes/runtime-controls.mdx:121: selected only by the second-opinion lane as justified.
  • snapshot-time managed MCP at docs/manage-sandboxes/runtime-controls.mdx:119: selected only by the second-opinion lane as justified.
  • canonical managed MCP policy at src/lib/actions/sandbox/mcp-bridge-policy.ts:202: selected only by the second-opinion lane as justified.
  • Shields transition ownership at src/lib/shields/index.ts:2786: selected only by the second-opinion lane as justified.
  • deadline composition at src/lib/shields/mcp-policy-transition.ts:94: selected only by the second-opinion lane as justified.
  • exact managed MCP policy at src/lib/shields/flow.test.ts:422: selected only by the second-opinion lane as justified.

Second-opinion terminology and E2E selections are advisory. They do not change the primary assessment or E2E / PR Gate.

3 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

  • established — managed MCP policy at docs/manage-sandboxes/runtime-controls.mdx:118: Use managed MCP policy for these NemoClaw-owned policy entries.
  • justified — exact agreement at docs/reference/commands.mdx:1175: Retain exact agreement and name the three required records where it first appears.
  • justified — snapshot-time managed MCP entries at docs/manage-sandboxes/runtime-controls.mdx:119: Retain snapshot-time managed MCP entries where restoration timing distinguishes saved and current entries.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-inference, cloud-onboard, security-posture, inference-routing, mcp-bridge, mcp-bridge-dev, network-policy, onboard-repair, onboard-resume

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@jyaunches
jyaunches requested a review from cv August 3, 2026 18:40
@github-actions github-actions Bot added v0.0.102 and removed v0.0.101 labels Aug 3, 2026

@apurvvkumaria apurvvkumaria left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 1803956 against its current stacked base. Exact policy ownership proof, canonical generated-policy validation, reserved-key stripping, fail-closed drift handling, and deadline restoration are sound and well covered. I found no blocking defect; exact-head CI and E2E are green. Re-review affected areas if the stacked base changes materially.

@cv
cv merged commit 0f6d801 into codex/shields-deadline-safety-7952 Aug 3, 2026
117 of 121 checks passed
@cv
cv deleted the codex/shields-mcp-policy-reconciliation-7952 branch August 3, 2026 23:37
jyaunches added a commit that referenced this pull request Aug 5, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Shields down previously replaced the complete live OpenShell policy,
which removed NemoClaw-generated MCP entries and made a surviving Hermes
MCP server unreachable. This change reconciles only exact, independently
proven NemoClaw-managed MCP entries so server A stays reachable while a
removed server B stays removed.

This is the focused MCP-policy change built on the lifecycle and
deadline prerequisite merged in #8130. It replaces the focused behavior
from the closed historical work in #7980 and #8141.

## Related Issue

Fixes #7952

## Changes

- Prove managed MCP ownership from the registry, generated-policy
record, and live gateway policy before preserving an entry.
- Save the managed-key manifest with the Shields snapshot, remove
snapshot-time managed entries during restoration, and overlay only the
current exact entries.
- Fail closed for ambiguous, malformed, or manually edited ownership;
deadline restoration omits unproven entries and records the omission
count.
- Preserve exact current managed entries in the permissive Shields
policy without copying unrelated live egress.
- Sanitize untrusted registry and policy identifiers before including
them in operator diagnostics.
- Clean staged policy files across success and failure paths.
- Reuse the validated unchanged snapshot when both saved and current
managed MCP sets are empty, so deadline restoration does not depend on
temporary staging.
- Update the Hermes MCP regression so it calls A immediately after
Shields down and again after B removal, before the later explicit `mcp
restart A` coverage.
- Document managed MCP policy behavior during Shields transitions.

### Why this appeared during the Hermes upgrade

The original live journey contained a hidden lifecycle between the first
successful call to A and the later B lifecycle:

1. Raise Shields.
2. Restart the Hermes gateway.
3. Lower Shields.
4. Exercise config rollback.
5. Add and remove B.
6. Call A.

A remained healthy through Shields up and the gateway restart. It became
unusable immediately after Shields down, which dropped A's generated MCP
policy. The later failure after B removal was only where the test
detected the damage; B removal was a misleading correlation.

This surfaced alongside the Hermes upgrade because coverage and upgrade
fixes landed close together:

- #7761 added the helper containing Shields up, gateway restart, Shields
down, and rollback, but did not run the complete live E2E.
- #7771 upgraded Hermes, while its selected E2E did not include the MCP
bridge target.
- #7849 fixed Hermes 0.19 migrations and the `mcp__fake__*` tool naming,
allowing the journey to progress far enough to expose the later failure.
- #7866 moved an explicit restart of A before the post-removal call,
which reapplied A's policy and masked the defect.

The whole-policy Shields replacement predates those changes. This is a
latent NemoClaw Shields policy-composition bug exposed by expanded
Hermes upgrade regression coverage, not evidence of a Hermes regression.

### Corrected live regression order

1. Raise Shields and restart the Hermes gateway.
2. Lower Shields and call A immediately.
3. Exercise config rollback.
4. Add B, prove DNS-rebinding access is denied, remove B, and verify A's
policy is unchanged while B is gone.
5. Call A before the later explicit restart.
6. Capture authenticated rediscovery state, restart A without
resupplying its secret, and call A again.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [x] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [x] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification: Independent maintainer
security review of commit `5d46d4a2ee924edf743ac36d04807948eec01c96`
passed all nine categories. The review covered managed-policy ownership,
diagnostic sanitization, deadline restoration, empty-MCP staging
failure, and current-main integration.
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Documentation Writer Review

- [x] Documentation writer subagent reviewed the completed changes
- Result: `docs-updated`
- Evidence: Reviewed the complete PR change at commit
`5d46d4a2ee924edf743ac36d04807948eec01c96` against current main,
including generated OpenClaw and Hermes variants, operator-facing
assertions, test titles, and the empty-MCP deadline-restore regression.
The current-main merge was mechanical, contributor attribution remains
intact, and the existing documentation remains accurate. Normal hooks
passed; GitHub CI is the current validation authority.
- Agent: Codex Desktop
<!-- docs-review-head-sha: 5d46d4a -->
<!-- docs-review-agents-blob-sha: 3dd7c24 -->

## DGX Station Hardware Evidence

- [ ] Tested on DGX Station
- Tested commit: Not applicable; `scripts/prepare-dgx-station-host.sh`
is unchanged.
- Station profile/scenario: Not applicable.
- Result: Not applicable.
- Supporting evidence: Not applicable.

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run validate:pr` passed after refreshing `origin/main` when hooks
were skipped or unavailable
- [ ] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification:
GitHub CI is running for commit
`5d46d4a2ee924edf743ac36d04807948eec01c96`.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result: GitHub CI is running for
commit `5d46d4a2ee924edf743ac36d04807948eec01c96`.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) — GitHub
CI is running for commit `5d46d4a2ee924edf743ac36d04807948eec01c96`.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [x] New doc pages include SPDX header and frontmatter (new pages only)
— no new pages

---
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Shields transitions now securely track and reconcile managed MCP
policies.
* Existing MCP servers retain verified endpoints and address pins during
unlock and restoration.
* Removed MCP servers remain removed instead of being unintentionally
restored.
* Automatic restoration omits policies that cannot be independently
verified and records clear warnings.

* **Bug Fixes**
* Improved fail-closed behavior for malformed, mismatched, incomplete,
or unavailable policy data.

* **Documentation**
* Expanded guidance on MCP policy handling during manual and automatic
Shields transitions.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Co-authored-by: Carlos Villela <cvillela@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: docs Documentation, examples, guides, or docs build area: policy Network policy, egress rules, presets, or sandbox policy area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery area: security Security controls, permissions, secrets, or hardening bug-fix PR fixes a bug or regression integration: hermes Hermes integration behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants