Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,42 @@ target.

## [Unreleased]

### Added

- **Bedrock providers can authenticate with a Bedrock API key.** Pick
*Bedrock API Key* as the authentication mode and paste the key AWS
generated. It is kept like any other provider's API key, as an
encrypted `Authorization: Bearer` header, and replaced in the
provider's Edit dialog. A Bedrock provider that sends an
`Authorization` header is not SigV4-signed; one without it is signed
as before, with access keys or the instance role. Use a long-term key:
a short-term one expires within 12 hours.
- **Test Connection for Bedrock providers with an API key.** It lists the
region's foundation models with the key, so a wrong key or a missing
permission shows up before any traffic does. Those models are not
offered for import: most current models are only served through an
inference profile, which that list does not name. Providers signed
with access keys or the instance role still have no connection test.

### Fixed

- **`error_type` in a failed request's gateway log could be the error's
debug text** rather than its tag: for an upstream HTTP error it was
`ProviderHttpError { status: …, message: "…" }`, with the upstream's
reply inside, and for an upstream rate limit a cut-off
`UpstreamRateLimited { retry_after_secs: Some`. Both now log their
tag, `ProviderHttpError` and `UpstreamRateLimited`, as the metric
labels and streamed requests always did.
- **Bedrock providers could not be created or edited in the console.**
The region was checked as a URL, so saving failed with
`400 Invalid URL`. A Bedrock provider's `base_url` is now checked as an
AWS region such as `us-east-1`, and anything else is refused, since the
host is built from it. A provider saved with a URL there never reached
Bedrock; set its region in the Edit dialog.
- **Typing into a provider's API key field and clearing it again wiped
the saved key on save.** The field sent `Bearer ` with nothing after
it. A cleared field now keeps the saved key, as a blank one always did.

## [2.0.0] — 2026-09-24

Callers now get errors in their own API's format, and an upstream that
Expand Down
1 change: 1 addition & 0 deletions crates/common/src/dto/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,7 @@ pub struct CreateProviderRequest {
pub name: String,
pub display_name: String,
pub provider_type: String,
/// The upstream's URL, or for Bedrock its AWS region (`us-east-1`).
pub base_url: String,
/// Unified request headers (auth + custom + identity templates).
/// Stored in config_json.headers as `[{key, value}]`.
Expand Down
55 changes: 55 additions & 0 deletions crates/common/src/validation.rs
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,31 @@ pub fn validate_url(url_str: &str) -> Result<(), AppError> {
Ok(())
}

/// Checks an AWS region code such as `us-east-1` or `us-gov-west-1`.
///
/// Bedrock providers keep a region where other providers keep a URL, and
/// the host is built from it: `bedrock-runtime.{region}.amazonaws.com`.
/// So this is a host check as much as a format one — anything beyond
/// dash-joined lowercase words and a trailing number could carry the
/// request, and the credential riding on it, to another host.
pub fn validate_aws_region(region: &str) -> Result<(), AppError> {
let valid = region.rsplit_once('-').is_some_and(|(name, number)| {
!number.is_empty()
&& number.bytes().all(|b| b.is_ascii_digit())
&& name.split('-').count() >= 2
&& name
.split('-')
.all(|word| !word.is_empty() && word.bytes().all(|b| b.is_ascii_lowercase()))
});
if valid {
Ok(())
} else {
Err(AppError::BadRequest(
"Invalid AWS region: expected one like us-east-1".into(),
))
}
}

/// Aggregate "do not connect to" check covering loopback, unspecified
/// (0.0.0.0 / ::), private ranges, link-local, ULA, IPv4-mapped IPv6,
/// 6to4, and IPv6 multicast.
Expand Down Expand Up @@ -418,6 +443,36 @@ mod tests {
assert!(validate_url("http://").is_err());
}

#[test]
fn validate_aws_region_accepts_region_codes() {
for region in [
"us-east-1",
"eu-central-2",
"ap-southeast-7",
"us-gov-west-1",
] {
assert!(validate_aws_region(region).is_ok(), "{region}");
}
}

#[test]
fn validate_aws_region_rejects_anything_that_could_change_the_host() {
for bad in [
"",
"us-east",
"useast1",
"US-EAST-1",
"us--east-1",
"-us-east-1",
"us-east-1.evil.example",
"evil.example#",
"evil.example/x?-1",
"https://bedrock-runtime.us-east-1.amazonaws.com",
] {
assert!(validate_aws_region(bad).is_err(), "{bad:?}");
}
}

#[test]
fn validate_custom_headers_accepts_valid() {
let mut headers = HashMap::new();
Expand Down
8 changes: 4 additions & 4 deletions crates/gateway/src/bedrock/sigv4.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
//! AWS SigV4 signing.
//!
//! Bedrock is the one upstream that does not take a bearer token: every
//! request is signed over its method, URL, time and the hash of its body.
//! So **signing has to happen after the body is final** — change one byte
//! and the signature no longer matches.
//! A Bedrock provider without an API key has no bearer token to send:
//! every request is signed over its method, URL, time and the hash of its
//! body. So **signing has to happen after the body is final** — change one
//! byte and the signature no longer matches.

use std::time::SystemTime;

Expand Down
65 changes: 50 additions & 15 deletions crates/gateway/src/proxy/log_ctx.rs
Original file line number Diff line number Diff line change
Expand Up @@ -112,21 +112,7 @@ pub(super) fn emit_gateway_error_log(
err: &GatewayError,
bodies: BodyCapture,
) {
let status = gateway_error_status(err);
let detail = serde_json::json!({
"model_id": model_id,
"provider": provider,
"input_tokens": 0i64,
"output_tokens": 0i64,
// Decimal-as-string in the audit JSON so the CH flush reader
// can reconstruct exact precision — the JSON `number` path
// would collapse through f64 in between.
"cost_usd": Decimal::ZERO.to_string(),
"latency_ms": latency_ms,
"status_code": status,
"error_type": format!("{err:?}").split('(').next().unwrap_or("Error"),
"error_message": err.to_string(),
});
let detail = error_detail(model_id, provider, latency_ms, err);
// Same `chat.completion` action as the success path — flush_gateway
// drops the action when it writes ChGatewayRow, so the trace
// endpoint distinguishes errors via `status_code` (>= 400) instead.
Expand All @@ -146,6 +132,31 @@ pub(super) fn emit_gateway_error_log(
audit.log(bodies.apply(entry));
}

/// The detail of a failed request's `gateway_logs` row.
fn error_detail(
model_id: &str,
provider: Option<&str>,
latency_ms: i64,
err: &GatewayError,
) -> serde_json::Value {
serde_json::json!({
"model_id": model_id,
"provider": provider,
"input_tokens": 0i64,
"output_tokens": 0i64,
// Decimal-as-string in the audit JSON so the CH flush reader
// can reconstruct exact precision — the JSON `number` path
// would collapse through f64 in between.
"cost_usd": Decimal::ZERO.to_string(),
"latency_ms": latency_ms,
"status_code": gateway_error_status(err),
// The tag, never the debug text: that carries every field, and
// an upstream error's fields hold the upstream's own reply.
"error_type": err.error_tag(),
"error_message": err.to_string(),
})
}

/// Same as `emit_gateway_log` but with an optional `extra` JSON object
/// whose fields are merged into the audit detail. Used by the
/// streaming on_done path to attach `error_type` / `error_message` /
Expand Down Expand Up @@ -248,3 +259,27 @@ pub(super) fn emit_gateway_log(
}));
audit.log(bodies.apply(entry));
}

#[cfg(test)]
mod tests {
use super::*;

#[test]
fn a_failed_requests_error_type_is_the_errors_tag() {
// A struct variant's debug text is its fields too, and an
// upstream's reply can name the account behind the credential
for err in [
GatewayError::ProviderHttpError {
status: 500,
message: "bedrock: arn:aws:iam::123456789012:user/gateway".into(),
},
GatewayError::UpstreamRateLimited {
retry_after_secs: Some(12),
},
GatewayError::ProviderError("boom".into()),
] {
let detail = error_detail("m", Some("p"), 1, &err);
assert_eq!(detail["error_type"], err.error_tag());
}
}
}
126 changes: 120 additions & 6 deletions crates/gateway/src/proxy/transport.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,8 @@ pub enum Shape {
/// Azure addresses a model by deployment name in the URL; there is no
/// model field it reads from the body.
Azure { api_version: String },
/// `https://bedrock-runtime.{region}.amazonaws.com` + the path, signed.
/// `https://bedrock-runtime.{region}.amazonaws.com` + the path,
/// SigV4-signed unless the provider sends a Bedrock API key.
///
/// The provider row keeps the region in `base_url`. The dialect
/// already wrote `/model/{id}/converse[-stream]` into the path.
Expand Down Expand Up @@ -91,8 +92,6 @@ impl Upstream {
}

/// Send `body` to `path`, and turn a non-2xx answer into an error.
///
/// Signing happens last, over the exact bytes being sent.
pub async fn send(
&self,
body: Vec<u8>,
Expand All @@ -102,6 +101,27 @@ impl Upstream {
extra: &[(String, String)],
ctx: &CallCtx,
) -> Result<reqwest::Response, GatewayError> {
let resp = self
.request(body, path, query, dialect, extra, ctx)
.await?
.send()
.await
.map_err(transport_error)?;
check_status(resp, &self.label).await
}

/// The request [`send`](Self::send) sends, built but not sent.
///
/// Signing happens last, over the exact bytes being sent.
async fn request(
&self,
body: Vec<u8>,
path: &str,
query: Option<&str>,
dialect: tw_dialect::ir::Dialect,
extra: &[(String, String)],
ctx: &CallCtx,
) -> Result<reqwest::RequestBuilder, GatewayError> {
let url = self.url(&body, path, query);

let mut req = self
Expand All @@ -128,7 +148,7 @@ impl Upstream {
if let Some(trace) = &ctx.trace_id {
req = req.header("x-trace-id", trace.as_str());
}
if let Shape::Bedrock { signer } = &self.shape {
if let Some(signer) = self.signer() {
let signed = signer
.sign(&self.client, &url, &body)
.await
Expand All @@ -138,8 +158,28 @@ impl Upstream {
}
}

let resp = req.body(body).send().await.map_err(transport_error)?;
check_status(resp, &self.label).await
Ok(req.body(body))
}

/// Who signs this upstream's requests, if anyone does.
///
/// Bedrock takes two kinds of credential. A Bedrock API key is a
/// bearer token the provider row sends in its own `Authorization`
/// header: it is the whole credential, there is nothing to sign, and
/// signing anyway would add a second `authorization` header, which
/// AWS rejects. Without one, every request is SigV4-signed.
fn signer(&self) -> Option<&Signer> {
match &self.shape {
Shape::Bedrock { signer }
if !self
.headers
.iter()
.any(|(k, _)| k.eq_ignore_ascii_case("authorization")) =>
{
Some(signer)
}
_ => None,
}
}

fn url(&self, body: &[u8], path: &str, query: Option<&str>) -> String {
Expand Down Expand Up @@ -311,4 +351,78 @@ mod tests {
"https://bedrock-runtime.us-east-1.amazonaws.com/model/anthropic.claude-v2/converse"
);
}

const AK: &str = "AKIAIOSFODNN7EXAMPLE";
const SK: &str = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY";

fn bedrock(headers: &[(&str, &str)], keys: Option<(&str, &str)>) -> Upstream {
Upstream::new(
"us-east-1",
headers
.iter()
.map(|(k, v)| (k.to_string(), v.to_string()))
.collect(),
Shape::Bedrock {
signer: Arc::new(Signer {
region: "us-east-1".into(),
access_key_id: keys.map(|(ak, _)| ak.into()),
secret_access_key: keys.map(|(_, sk)| sk.into()),
}),
},
"test",
)
}

/// The `authorization` headers the request goes out with, and
/// whether SigV4 touched it.
async fn auth_headers(u: &Upstream) -> (Vec<String>, bool) {
let req = u
.request(
b"{}".to_vec(),
"/model/m/converse",
None,
tw_dialect::ir::Dialect::Bedrock,
&[],
&CallCtx::default(),
)
.await
.unwrap()
.build()
.unwrap();
let auth = req
.headers()
.get_all("authorization")
.iter()
.map(|v| v.to_str().unwrap().to_string())
.collect();
(auth, req.headers().contains_key("x-amz-date"))
}

#[tokio::test]
async fn a_bedrock_api_key_goes_out_as_it_is_and_nothing_is_signed() {
// No access keys: signing would have gone to IMDS for credentials
let u = bedrock(&[("Authorization", "Bearer ABSK-test")], None);
let (auth, signed) = auth_headers(&u).await;
assert_eq!(auth, ["Bearer ABSK-test"]);
assert!(!signed);
}

#[tokio::test]
async fn an_api_key_wins_over_access_keys() {
// Signing on top would add a second `authorization`, which AWS
// rejects. The header name matches in any case.
let u = bedrock(&[("authorization", "Bearer ABSK-test")], Some((AK, SK)));
let (auth, signed) = auth_headers(&u).await;
assert_eq!(auth, ["Bearer ABSK-test"]);
assert!(!signed);
}

#[tokio::test]
async fn bedrock_without_an_api_key_is_signed() {
let u = bedrock(&[("x-custom", "1")], Some((AK, SK)));
let (auth, signed) = auth_headers(&u).await;
assert_eq!(auth.len(), 1, "{auth:?}");
assert!(auth[0].starts_with("AWS4-HMAC-SHA256 "), "{auth:?}");
assert!(signed);
}
}
Loading
Loading