Skip to content

fix(relay): preserve upstream HTTP error statuses - #1316

Open
afourniernv wants to merge 1 commit into
mainfrom
codex/fix-relay-upstream-status-20261113
Open

afourniernv wants to merge 1 commit into
mainfrom
codex/fix-relay-upstream-status-20261113

Conversation

@afourniernv

@afourniernv afourniernv commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

What

Keep the upstream HTTP status when a managed Switchyard route fails through Relay. Use Relay's existing HTTP-error helper and keep the generic error message so provider error bodies stay private.

The 0.10.1 patch bump is needed for this fix: it adds the SDK's http_error_message(...) helper and the host-side handling for those errors. The minimum host version is raised because 0.10.0 would still return generic errors. Both versions use native ABI 7; there is no ABI change.

Why

Fixes SWITCH-1951.

Upstream 401, 429 and 503 responses currently become HTTP 400 through the plugin. Callers lose the distinction between authentication errors, rate limits and temporary failures.

The earlier fix in #1027 landed on release/0.3, but not on main.

Notes for reviewers

The code change is in SwitchyardRuntime::execute. It uses the status already available in the safe error summary. Errors without an HTTP status keep the existing generic message. No new public APIs or changes to libsy or the codecs.

This fixes buffered response statuses. Once Relay has sent HTTP 200 for a stream, later failures cannot change that status.

Validation:

  • Regression test covers 401, 429 and 503 through the runtime and HTTP client, including provider-body privacy. All 44 plugin tests pass.
  • Rust workspace tests, strict workspace Clippy, formatting and the strict docs build pass.
  • Plugin bundle packaging tests pass.
  • All 214 non-integration Python tests, Ruff and mypy pass.
  • Tested the built plugin in released Relay 0.11.1 with a local HTTP backend. All nine buffered cases preserved their status across Chat Completions, Responses and Anthropic Messages. Plugin-off controls also preserved their statuses. No provider API calls were needed.

Summary by CodeRabbit

  • New Features
    • Managed routes now preserve known upstream HTTP error statuses in buffered responses, while keeping provider error details private. Failures without a known status retain existing handling.
  • Compatibility
    • The plugin now requires NeMo Relay 0.10.1 or later, before 1.0.0. Streaming responses retain existing error behavior if a failure occurs after the response begins.

Signed-off-by: Alex Fournier <afournier@nvidia.com>
@afourniernv
afourniernv requested a review from a team as a code owner November 13, 2026 19:07
@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The plugin dependency and minimum Relay version move to 0.10.1. Route execution errors use Relay’s HTTP-error formatter when an upstream status is available, with a generic message as fallback. Tests cover selected statuses and provider-body privacy.

Changes

NeMo Relay compatibility

Layer / File(s) Summary
Relay version requirement
Cargo.toml, crates/switchyard-nemo-relay-plugin/relay-plugin.toml, crates/switchyard-nemo-relay-plugin/README.md, docs/integrations/nemo_relay.md, tests/relay_plugin/test_package_bundle.py
The plugin dependency and minimum Relay version change to 0.10.1. The upper version bound and native plugin API label remain unchanged. Documentation describes the HTTP-error status handling and the compatibility limits for older Relay versions. The bundle test expects the updated minimum.
Upstream HTTP error handling
crates/switchyard-nemo-relay-plugin/src/runtime.rs
Route execution errors use Relay’s HTTP-error formatter when a summarized upstream status is available. The generic failure message remains the fallback. Tests cover statuses 401, 429, and 503, check provider-body privacy, and set max_retries to 0 in the target-runtime test configuration.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: Minimal · up to 3f8a6

This change preserves upstream HTTP error statuses for buffered Relay routes and raises the minimum supported Relay version to 0.10.1. No actionable merge-blocking risk was found.

Security architecture risk: Minimal · up to 3f8a6

The inspected change preserves upstream error statuses while retaining generic error text and existing request-routing controls. No introduced or worsened risk to security, failure containment, rollback, or ownership was established.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The introduced response behavior is bounded to failures on requests already handled by configured Switchyard routes. The unchanged intercept guards continue to delegate unsupported protocols and unmanaged models to Relay; the inspected change does not expand request reachability or authorization authority.

Trust Boundaries and Controls

  • observed — The host-facing boundary remains an Err(String) return. Existing sanitization processes errors and events before they leave runtime execution. The supplied boundary evidence identifies Relay host interpretation of that string as outside the repository source, so the final HTTP response mapping was not independently verified.
Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title "fix(relay): preserve upstream HTTP error statuses" matches the main change. The raw summary and objectives show that SwitchyardRuntime::execute now keeps upstream HTTP error statuses (401, …
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Full details: Title check

Explanation

The title "fix(relay): preserve upstream HTTP error statuses" matches the main change. The raw summary and objectives show that SwitchyardRuntime::execute now keeps upstream HTTP error statuses (401, 429, 503) through Relay's HTTP-error helper. The other changes (dependency bump to nemo-relay-plugin 0.10.1, manifest and docs updates, and the test) support this fix. The title is short, specific, and follows a conventional commit style.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

A rabbit checks the status line,
“Four-oh-one, four-twenty-nine, five-oh-three!”
The private body stays tucked away,
A clear fallback marks the cloudy day,
Relay hops to version ten-oh-one!

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 287365f and 3f8a65b.

Files ignored due to path filters: 1

  • Cargo.lock is excluded by !**/*.lock, !Cargo.lock
Files selected for processing: 6

  • Cargo.toml
  • crates/switchyard-nemo-relay-plugin/README.md
  • crates/switchyard-nemo-relay-plugin/relay-plugin.toml
  • crates/switchyard-nemo-relay-plugin/src/runtime.rs
  • docs/integrations/nemo_relay.md
  • tests/relay_plugin/test_package_bundle.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.2

🚀 View preview at
https://NVIDIA-NeMo.github.io/Switchyard/pr-preview/pr-1316/

Built to branch gh-pages at 2026-11-13 19:10 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant