From c7f30c8b8f723b74f693fe7b2224acc103764d0b Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Fri, 5 Jun 2026 08:11:07 +0000 Subject: [PATCH 1/5] fix(sdk): reject Cloudflare-native model IDs, bridge multi-part auth env vars Closes #392. ## Problem Users configuring Cloudflare Workers AI on the Pi runtime hit three compounding failures: 1. @cf/... model IDs (Cloudflare's native format) pass isPiModelSelector() validation because they contain a slash. They parse as provider="@cf", which the Pi registry does not know, throwing a generic "Pi model not found" error that classifies as rather than . The result is the generic all-chunks-failed message: "This usually indicates an authentication problem." 2. WARDEN_CLOUDFLARE_ACCOUNT_ID is silently ignored by bridgeWardenProviderApiKeyEnv() because it does not end in _API_KEY. Cloudflare Workers AI requires CLOUDFLARE_ACCOUNT_ID in addition to an API key. Users following the WARDEN_ alias convention have no way to supply it. 3. The all_hunks_failed error message discards the per-chunk failure message and always implies the root cause is authentication, even when it is a config/model problem. ## Fix ### Model selector validation (model-selectors.ts) - isPiModelSelector() now validates the provider segment against /^[a-z0-9][a-z0-9-]*$/, rejecting @cf/... and other provider-native namespace prefixes that cannot be Pi provider names. - invalidPiModelSelectorMessage() emits targeted guidance for @cf/... model IDs, including the correct selector format and required env vars. - New piModelSelectorTip() returns context-specific repair tips used in the CLI reporter. - New findMissingCloudflareEnv() / missingCloudflareEnvMessage() detect missing CLOUDFLARE_ACCOUNT_ID / CLOUDFLARE_GATEWAY_ID for Cloudflare providers before analysis starts. ### Runtime defense (pi.ts) - resolvePiModel() now calls throwModelNotFound() on registry miss, which throws InvalidPiModelSelectorError instead of a generic Error. This classifies model-not-found as , opening the circuit breaker immediately and giving a specific error message rather than falling through to all_hunks_failed. ### Env bridging (utils/index.ts) - bridgeWardenProviderApiKeyEnv() now also bridges a WARDEN_PROVIDER_ENV_BRIDGE list of non-API-key env vars for providers that require compound auth: WARDEN_CLOUDFLARE_ACCOUNT_ID -> CLOUDFLARE_ACCOUNT_ID WARDEN_CLOUDFLARE_GATEWAY_ID -> CLOUDFLARE_GATEWAY_ID Native env vars are never overwritten. ### Error message quality (analyze.ts, errors.ts) - all_hunks_failed now includes the first failure message (sanitized, capped at 500 chars) so config errors are visible without per-hunk log inspection. - Message wording changed from "usually authentication" to "runtime, model, or provider configuration" to avoid mis-diagnosing config errors. - PI_AUTH_GUIDANCE updated to mention Cloudflare multi-part auth requirements. ### CLI/action preflight (main.ts, executor.ts, schedule.ts) - All three run paths now call findMissingCloudflareEnv() after the model selector check and fail fast with a specific message before any analysis starts. ### Docs (config/models.mdx, config-schema.md) - Pi model selectors table includes Cloudflare Workers AI and AI Gateway. - Explains provider-specific model ID format and multi-part credential requirements. - Env vars table updated with WARDEN_CLOUDFLARE_* aliases. Refs #392 Co-authored-by: immutable dcramer --- .../docs/src/content/docs/config/models.mdx | 10 +- .../warden/src/action/triggers/executor.ts | 7 +- .../warden/src/action/workflow/schedule.ts | 7 +- packages/warden/src/cli/main.ts | 17 +- packages/warden/src/sdk/analyze.ts | 12 +- packages/warden/src/sdk/errors.ts | 8 +- .../src/sdk/runtimes/model-selectors.test.ts | 185 +++++++++++++++++- .../src/sdk/runtimes/model-selectors.ts | 126 +++++++++++- packages/warden/src/sdk/runtimes/pi.ts | 21 +- packages/warden/src/utils/index.test.ts | 46 +++++ packages/warden/src/utils/index.ts | 35 +++- skills/warden/references/config-schema.md | 3 + 12 files changed, 461 insertions(+), 16 deletions(-) diff --git a/packages/docs/src/content/docs/config/models.mdx b/packages/docs/src/content/docs/config/models.mdx index 60b6e2a7..f237d820 100644 --- a/packages/docs/src/content/docs/config/models.mdx +++ b/packages/docs/src/content/docs/config/models.mdx @@ -44,16 +44,24 @@ Examples: | `groq/llama-3.3-70b-versatile` | `WARDEN_GROQ_API_KEY` | | `openrouter/meta-llama/llama-3.3-70b-instruct` | `WARDEN_OPENROUTER_API_KEY` | | `together/meta-llama/Llama-3.3-70B-Instruct-Turbo` | `WARDEN_TOGETHER_API_KEY` | +| `cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6` | `WARDEN_CLOUDFLARE_API_KEY` + `CLOUDFLARE_ACCOUNT_ID` | +| `cloudflare-ai-gateway/...` | `WARDEN_CLOUDFLARE_API_KEY` + `CLOUDFLARE_ACCOUNT_ID` + `CLOUDFLARE_GATEWAY_ID` | Warden splits Pi selectors at the first `/`. The provider comes before it and the provider-specific model ID comes after it, so model IDs may contain additional -slashes. +slashes. For example, `cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6` sets +provider `cloudflare-workers-ai` and model ID `@cf/moonshotai/kimi-k2.6`. **Credential convention:** Set `WARDEN_{PROVIDER}_API_KEY` to authenticate with a provider. Warden mirrors these to the native `{PROVIDER}_API_KEY` expected by each SDK at runtime. If you already have a native provider key set (e.g. `OPENAI_API_KEY`), Warden will use it directly and the `WARDEN_`-prefixed form is not required. +**Multi-part credentials:** Some providers require additional env vars beyond an API key. +Cloudflare providers require `CLOUDFLARE_ACCOUNT_ID` (or `WARDEN_CLOUDFLARE_ACCOUNT_ID`). +Cloudflare AI Gateway additionally requires `CLOUDFLARE_GATEWAY_ID` (or `WARDEN_CLOUDFLARE_GATEWAY_ID`). +Warden mirrors these automatically when the `WARDEN_`-prefixed form is set. + ## Claude Runtime Models When `runtime = "claude"`, use the model IDs accepted by Claude Code: diff --git a/packages/warden/src/action/triggers/executor.ts b/packages/warden/src/action/triggers/executor.ts index 2a85158d..be153ff4 100644 --- a/packages/warden/src/action/triggers/executor.ts +++ b/packages/warden/src/action/triggers/executor.ts @@ -28,7 +28,7 @@ import { SkillRunnerError } from '../../sdk/errors.js'; import type { Semaphore } from '../../utils/index.js'; import { Verbosity } from '../../cli/output/verbosity.js'; import type { ProviderFailureCircuitBreaker } from '../../sdk/circuit-breaker.js'; -import { assertValidPiModelSelectors } from '../../sdk/runtimes/model-selectors.js'; +import { assertValidPiModelSelectors, findMissingCloudflareEnv, missingCloudflareEnvMessage } from '../../sdk/runtimes/model-selectors.js'; import { captureActionTriggerError } from '../error-reporting.js'; /** Log-mode output for CI: no TTY, no color. */ @@ -135,6 +135,11 @@ export async function executeTrigger( try { assertValidPiModelSelectors([trigger]); + const missingCloudflare = findMissingCloudflareEnv([trigger]); + if (missingCloudflare) { + throw new Error(missingCloudflareEnvMessage(missingCloudflare)); + } + const taskOptions: SkillTaskOptions = { name: trigger.name, displayName: trigger.skill, diff --git a/packages/warden/src/action/workflow/schedule.ts b/packages/warden/src/action/workflow/schedule.ts index 21abd6b1..0890f845 100644 --- a/packages/warden/src/action/workflow/schedule.ts +++ b/packages/warden/src/action/workflow/schedule.ts @@ -15,7 +15,7 @@ import type { LayeredSkillRootsByName, ResolvedTrigger } from '../../config/load import type { ScheduleConfig } from '../../config/schema.js'; import { buildScheduleEventContext } from '../../event/schedule-context.js'; import { runSkill } from '../../sdk/runner.js'; -import { assertValidPiModelSelectors } from '../../sdk/runtimes/model-selectors.js'; +import { assertValidPiModelSelectors, findMissingCloudflareEnv, missingCloudflareEnvMessage } from '../../sdk/runtimes/model-selectors.js'; import { createOrUpdateIssue } from '../../output/github-issues.js'; import { shouldFail, countFindingsAtOrAbove, countSeverity } from '../../triggers/matcher.js'; import { resolveSkillAsync } from '../../skills/loader.js'; @@ -179,6 +179,11 @@ async function runScheduleWorkflowInner( try { assertValidPiModelSelectors([resolved]); + const missingCloudflare = findMissingCloudflareEnv([resolved]); + if (missingCloudflare) { + throw new Error(missingCloudflareEnvMessage(missingCloudflare)); + } + // Build context from paths filter const patterns = resolved.filters?.paths ?? ['**/*']; const ignorePatterns = resolved.filters?.ignorePaths; diff --git a/packages/warden/src/cli/main.ts b/packages/warden/src/cli/main.ts index f1bc5bb4..c16710b8 100644 --- a/packages/warden/src/cli/main.ts +++ b/packages/warden/src/cli/main.ts @@ -7,8 +7,12 @@ import type { SkillDefinition, WardenConfig, Effort } from '../config/schema.js' import { verifyAuth, type WardenAuthenticationError, type SkillRunnerOptions, type ChunkAnalysisResult } from '../sdk/runner.js'; import { findInvalidPiModelSelector as findInvalidPiModelSelectorTarget, + findMissingCloudflareEnv, invalidPiModelSelectorMessage, + missingCloudflareEnvMessage, + piModelSelectorTip, type InvalidPiModelSelector, + type MissingCloudflareEnv, } from '../sdk/runtimes/model-selectors.js'; import { mapExtractionErrorCode } from '../sdk/errors.js'; import { aggregateAuxiliaryUsageAttribution, mergeAuxiliaryUsage } from '../sdk/usage.js'; @@ -732,7 +736,13 @@ export function findInvalidPiModelSelector( function reportInvalidPiModelSelector(reporter: Reporter, invalid: InvalidPiModelSelector): void { reporter.error(invalidPiModelSelectorMessage(invalid)); - reporter.tip('Set a Pi model selector such as anthropic/claude-sonnet-4-6.'); + reporter.tip(piModelSelectorTip(invalid.model)); +} + +function reportMissingCloudflareEnv(reporter: Reporter, missing: MissingCloudflareEnv): void { + reporter.error(missingCloudflareEnvMessage(missing)); + const tip = missing.missing.map((v) => `export WARDEN_${v}=...`).join(' '); + reporter.tip(tip); } function emitInvalidPiModelSelectorRunLog( @@ -1518,6 +1528,11 @@ async function runConfigMode(options: CLIOptions, reporter: Reporter): Promise ({ ...s.runnerOptions }))); + if (missingCloudflare) { + reportMissingCloudflareEnv(reporter, missingCloudflare); + return 1; + } let tasks: SkillTaskOptions[]; const concurrency = options.parallel ?? config.runner?.concurrency ?? DEFAULT_CONCURRENCY; try { diff --git a/packages/warden/src/sdk/analyze.ts b/packages/warden/src/sdk/analyze.ts index e2398ad3..4f6a12b6 100644 --- a/packages/warden/src/sdk/analyze.ts +++ b/packages/warden/src/sdk/analyze.ts @@ -1144,9 +1144,17 @@ async function runSkillAnalysis( { code: 'provider_unavailable' }, ); } + // Include the first failure message so users can diagnose config problems + // without inspecting per-hunk logs. Messages are already sanitized by the + // time they reach failureMessage. + const firstFailureMsg = analysisFailures[0]?.message; + const failureDetail = firstFailureMsg + ? ` First failure: ${firstFailureMsg.slice(0, 500)}.` + : ''; throw new SkillRunnerError( - `All ${totalHunks} chunk${totalHunks === 1 ? '' : 's'} failed to analyze. ` + - `This usually indicates an authentication problem. ${allHunksFailedGuidance(options.runtime)}`, + `All ${totalHunks} chunk${totalHunks === 1 ? '' : 's'} failed to analyze.${failureDetail} ` + + `This usually indicates a runtime, model, or provider configuration problem. ` + + `${allHunksFailedGuidance(options.runtime)}`, { code: 'all_hunks_failed' }, ); } diff --git a/packages/warden/src/sdk/errors.ts b/packages/warden/src/sdk/errors.ts index babc0bb6..48cb61f8 100644 --- a/packages/warden/src/sdk/errors.ts +++ b/packages/warden/src/sdk/errors.ts @@ -67,8 +67,12 @@ https://console.anthropic.com/ for API keys`; /** User-friendly error message for authentication failures (Pi runtime) */ const PI_AUTH_GUIDANCE = ` - export WARDEN_MODEL=provider/model-id # e.g. openai/gpt-5.5 - export WARDEN_{PROVIDER}_API_KEY=... # WARDEN-prefixed key for that provider + export WARDEN_MODEL=provider/model-id # e.g. openai/gpt-5.5, cloudflare-workers-ai/@cf/model + export WARDEN_{PROVIDER}_API_KEY=... # WARDEN-prefixed API key for that provider + +Note: Some providers require additional env vars beyond an API key. +Cloudflare providers also require CLOUDFLARE_ACCOUNT_ID (or WARDEN_CLOUDFLARE_ACCOUNT_ID). +Cloudflare AI Gateway additionally requires CLOUDFLARE_GATEWAY_ID (or WARDEN_CLOUDFLARE_GATEWAY_ID). See https://warden.sentry.dev/config/models for provider selectors and credential names.`; diff --git a/packages/warden/src/sdk/runtimes/model-selectors.test.ts b/packages/warden/src/sdk/runtimes/model-selectors.test.ts index 8bcd43c8..843ebe30 100644 --- a/packages/warden/src/sdk/runtimes/model-selectors.test.ts +++ b/packages/warden/src/sdk/runtimes/model-selectors.test.ts @@ -1,9 +1,27 @@ import { describe, expect, it } from 'vitest'; -import { assertValidPiModelSelectors, isPiModelSelector } from './model-selectors.js'; +import { + assertValidPiModelSelectors, + findMissingCloudflareEnv, + invalidPiModelSelectorMessage, + isPiModelSelector, + piModelSelectorTip, +} from './model-selectors.js'; describe('isPiModelSelector', () => { - it('accepts provider-prefixed model IDs', () => { + it('accepts standard provider/model selectors', () => { expect(isPiModelSelector('openai/gpt-5.5')).toBe(true); + expect(isPiModelSelector('anthropic/claude-sonnet-4-6')).toBe(true); + expect(isPiModelSelector('groq/llama-3.3-70b-versatile')).toBe(true); + expect(isPiModelSelector('openrouter/meta-llama/llama-3.3-70b-instruct')).toBe(true); + }); + + it('accepts Pi provider names with hyphens', () => { + expect(isPiModelSelector('cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6')).toBe(true); + expect(isPiModelSelector('cloudflare-ai-gateway/@cf/moonshotai/kimi-k2.6')).toBe(true); + expect(isPiModelSelector('amazon-bedrock/us.anthropic.claude-sonnet-4')).toBe(true); + }); + + it('accepts provider-specific model IDs with internal slashes', () => { expect(isPiModelSelector('fireworks/accounts/fireworks/models/kimi-k2p6')).toBe(true); }); @@ -12,6 +30,64 @@ describe('isPiModelSelector', () => { expect(isPiModelSelector('/gpt-5.5')).toBe(false); expect(isPiModelSelector('fireworks/')).toBe(false); }); + + it('rejects Cloudflare native model IDs used without a Pi provider prefix', () => { + expect(isPiModelSelector('@cf/moonshotai/kimi-k2.6')).toBe(false); + expect(isPiModelSelector('@cf/meta/llama-3.3-70b')).toBe(false); + }); + + it('rejects provider-native namespace prefixes generally', () => { + // Any segment starting with @ is not a valid Pi provider name + expect(isPiModelSelector('@vendor/some-model')).toBe(false); + }); +}); + +describe('invalidPiModelSelectorMessage', () => { + it('emits targeted guidance for Cloudflare Workers AI native model IDs', () => { + const msg = invalidPiModelSelectorMessage({ option: 'model', model: '@cf/moonshotai/kimi-k2.6' }); + expect(msg).toContain('cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6'); + expect(msg).toContain('CLOUDFLARE_API_KEY'); + expect(msg).toContain('CLOUDFLARE_ACCOUNT_ID'); + }); + + it('emits generic namespace guidance for non-Cloudflare @ prefixes', () => { + const msg = invalidPiModelSelectorMessage({ option: 'model', model: '@vendor/some-model' }); + expect(msg).toContain('@vendor/'); + expect(msg).toContain('provider-native'); + }); + + it('emits standard format guidance for plain model IDs without a provider', () => { + const msg = invalidPiModelSelectorMessage({ option: 'model', model: 'gpt-5.5' }); + expect(msg).toContain('provider/model format'); + expect(msg).toContain('gpt-5.5'); + }); + + it('includes the spec name when provided', () => { + const msg = invalidPiModelSelectorMessage({ + specName: 'security-review', + option: 'auxiliaryModel', + model: '@cf/meta/llama-3.3', + }); + expect(msg).toContain('security-review'); + }); +}); + +describe('piModelSelectorTip', () => { + it('gives Cloudflare-specific repair tip for @cf/ models', () => { + const tip = piModelSelectorTip('@cf/moonshotai/kimi-k2.6'); + expect(tip).toContain('cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6'); + expect(tip).toContain('CLOUDFLARE_ACCOUNT_ID'); + }); + + it('gives generic namespace tip for non-Cloudflare @ models', () => { + const tip = piModelSelectorTip('@vendor/model'); + expect(tip).toContain('provider-name/@vendor/model'); + }); + + it('gives standard tip for plain model IDs', () => { + const tip = piModelSelectorTip('gpt-5.5'); + expect(tip).toContain('anthropic/claude-sonnet-4-6'); + }); }); describe('assertValidPiModelSelectors', () => { @@ -25,4 +101,109 @@ describe('assertValidPiModelSelectors', () => { }, ])).not.toThrow(); }); + + it('allows cloudflare-workers-ai selectors with native model IDs', () => { + expect(() => assertValidPiModelSelectors([ + { + runtime: 'pi', + model: 'cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6', + }, + ])).not.toThrow(); + }); + + it('throws for Cloudflare native model IDs used without a Pi provider prefix', () => { + expect(() => assertValidPiModelSelectors([ + { + runtime: 'pi', + model: '@cf/moonshotai/kimi-k2.6', + }, + ])).toThrow(/cloudflare-workers-ai\/@cf\/moonshotai\/kimi-k2\.6/); + }); +}); + +describe('findMissingCloudflareEnv', () => { + it('returns undefined when no Cloudflare provider is configured', () => { + const result = findMissingCloudflareEnv( + [{ runtime: 'pi', model: 'openai/gpt-5.5' }], + { OPENAI_API_KEY: 'key' }, + ); + expect(result).toBeUndefined(); + }); + + it('returns undefined when cloudflare-workers-ai has all required env vars', () => { + const result = findMissingCloudflareEnv( + [{ runtime: 'pi', model: 'cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6' }], + { CLOUDFLARE_API_KEY: 'key', CLOUDFLARE_ACCOUNT_ID: 'acct' }, + ); + expect(result).toBeUndefined(); + }); + + it('accepts WARDEN_CLOUDFLARE_ACCOUNT_ID as an alternative', () => { + const result = findMissingCloudflareEnv( + [{ runtime: 'pi', model: 'cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6' }], + { CLOUDFLARE_API_KEY: 'key', WARDEN_CLOUDFLARE_ACCOUNT_ID: 'acct' }, + ); + expect(result).toBeUndefined(); + }); + + it('reports missing CLOUDFLARE_ACCOUNT_ID for cloudflare-workers-ai', () => { + const result = findMissingCloudflareEnv( + [{ runtime: 'pi', model: 'cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6' }], + { CLOUDFLARE_API_KEY: 'key' }, + ); + expect(result).toMatchObject({ + provider: 'cloudflare-workers-ai', + missing: ['CLOUDFLARE_ACCOUNT_ID'], + }); + }); + + it('reports both missing vars for cloudflare-ai-gateway', () => { + const result = findMissingCloudflareEnv( + [{ runtime: 'pi', model: 'cloudflare-ai-gateway/workers-ai/@cf/moonshotai/kimi-k2.6' }], + { CLOUDFLARE_API_KEY: 'key' }, + ); + expect(result).toMatchObject({ + provider: 'cloudflare-ai-gateway', + missing: expect.arrayContaining(['CLOUDFLARE_ACCOUNT_ID', 'CLOUDFLARE_GATEWAY_ID']), + }); + }); + + it('reports only CLOUDFLARE_GATEWAY_ID when account ID is present', () => { + const result = findMissingCloudflareEnv( + [{ runtime: 'pi', model: 'cloudflare-ai-gateway/workers-ai/@cf/moonshotai/kimi-k2.6' }], + { CLOUDFLARE_API_KEY: 'key', CLOUDFLARE_ACCOUNT_ID: 'acct' }, + ); + expect(result).toMatchObject({ + provider: 'cloudflare-ai-gateway', + missing: ['CLOUDFLARE_GATEWAY_ID'], + }); + }); + + it('accepts WARDEN_CLOUDFLARE_GATEWAY_ID as an alternative', () => { + const result = findMissingCloudflareEnv( + [{ runtime: 'pi', model: 'cloudflare-ai-gateway/workers-ai/@cf/moonshotai/kimi-k2.6' }], + { + CLOUDFLARE_API_KEY: 'key', + CLOUDFLARE_ACCOUNT_ID: 'acct', + WARDEN_CLOUDFLARE_GATEWAY_ID: 'gw', + }, + ); + expect(result).toBeUndefined(); + }); + + it('skips non-Pi runtimes', () => { + const result = findMissingCloudflareEnv( + [{ runtime: 'claude', model: 'cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6' }], + {}, + ); + expect(result).toBeUndefined(); + }); + + it('skips targets with invalid selectors (already caught by other checks)', () => { + const result = findMissingCloudflareEnv( + [{ runtime: 'pi', model: '@cf/moonshotai/kimi-k2.6' }], + {}, + ); + expect(result).toBeUndefined(); + }); }); diff --git a/packages/warden/src/sdk/runtimes/model-selectors.ts b/packages/warden/src/sdk/runtimes/model-selectors.ts index 8706ca84..92446e69 100644 --- a/packages/warden/src/sdk/runtimes/model-selectors.ts +++ b/packages/warden/src/sdk/runtimes/model-selectors.ts @@ -1,9 +1,23 @@ /** * Return true when a Pi model selector uses provider/model-id syntax. + * + * Valid Pi selectors split at the first slash: the segment before it is the Pi + * provider name and the segment after is the provider-specific model ID. + * + * Pi provider names are lowercase alphanumeric strings with hyphens, e.g. + * `openai`, `anthropic`, `cloudflare-workers-ai`. Provider-native namespaces + * such as Cloudflare's `@cf/...` model IDs must be prefixed with the Pi + * provider name to form a valid Warden selector. */ export function isPiModelSelector(model: string): boolean { const slashIndex = model.indexOf('/'); - return slashIndex > 0 && slashIndex < model.length - 1; + if (slashIndex <= 0 || slashIndex >= model.length - 1) return false; + + // Pi provider names are lowercase letters, digits, and hyphens. + // Reject provider-native namespace prefixes (e.g. @cf/...) and other + // non-conforming provider segments that would silently fail at model lookup. + const provider = model.slice(0, slashIndex); + return /^[a-z0-9][a-z0-9-]*$/.test(provider); } export type PiModelSelectorOption = 'model' | 'auxiliaryModel' | 'synthesisModel'; @@ -24,10 +38,52 @@ export interface InvalidPiModelSelector { /** * Format the user-facing error for an invalid Pi model selector. + * + * Emits targeted guidance for known misuse patterns such as Cloudflare + * Workers AI native model IDs being passed without a Pi provider prefix. */ export function invalidPiModelSelectorMessage(invalid: InvalidPiModelSelector): string { const target = invalid.specName ? ` for ${invalid.specName}` : ''; - return `Pi runtime ${invalid.option}${target} must use provider/model format: ${invalid.model}`; + const { model } = invalid; + + // Cloudflare Workers AI native model IDs start with @cf/. Users sometimes + // set these directly instead of using the Warden Pi provider prefix. + if (model.startsWith('@cf/')) { + return ( + `Pi runtime ${invalid.option}${target} received a Cloudflare Workers AI native model ID: ${model}. ` + + `Use cloudflare-workers-ai/${model} as the Warden Pi selector instead. ` + + `Set CLOUDFLARE_API_KEY (or WARDEN_CLOUDFLARE_API_KEY) and CLOUDFLARE_ACCOUNT_ID ` + + `(or WARDEN_CLOUDFLARE_ACCOUNT_ID) for this provider.` + ); + } + + // Generic provider-native namespace prefix (@vendor/...) without a Pi provider. + const slashIndex = model.indexOf('/'); + if (model.startsWith('@') && slashIndex > 0) { + const namespace = model.slice(0, slashIndex + 1); + return ( + `Pi runtime ${invalid.option}${target} received a provider-native model ID: ${model}. ` + + `"${namespace}..." is a provider-native namespace, not a Pi provider name. ` + + `Prefix with the Pi provider name, e.g. provider-name/${model}. ` + + `See https://warden.sentry.dev/config/models for supported providers.` + ); + } + + return `Pi runtime ${invalid.option}${target} must use provider/model format: ${model}`; +} + +/** + * Return a contextual repair tip for an invalid Pi model selector. + * Used alongside invalidPiModelSelectorMessage to give actionable next steps. + */ +export function piModelSelectorTip(model: string): string { + if (model.startsWith('@cf/')) { + return `Use cloudflare-workers-ai/${model} and set CLOUDFLARE_API_KEY + CLOUDFLARE_ACCOUNT_ID.`; + } + if (model.startsWith('@')) { + return 'Prefix the model with its Pi provider name, e.g. provider-name/@vendor/model-id.'; + } + return 'Set a Pi model selector such as anthropic/claude-sonnet-4-6.'; } /** @@ -43,6 +99,72 @@ export class InvalidPiModelSelectorError extends Error { } } +export interface MissingCloudflareEnv { + provider: string; + /** The env var names that are missing (native form, e.g. CLOUDFLARE_ACCOUNT_ID). */ + missing: string[]; +} + +/** + * Find required Cloudflare provider env vars that are not set. + * + * Cloudflare Workers AI requires CLOUDFLARE_ACCOUNT_ID in addition to an API + * key. Cloudflare AI Gateway additionally requires CLOUDFLARE_GATEWAY_ID. + * Neither can be inferred from model selectors alone and both must be set as + * environment variables (Pi does not read them from its auth.json). + * + * Both the native form (CLOUDFLARE_ACCOUNT_ID) and the Warden alias + * (WARDEN_CLOUDFLARE_ACCOUNT_ID) are accepted. + * + * Returns the first target with missing required vars, or undefined. + */ +export function findMissingCloudflareEnv( + targets: PiModelSelectorTarget[], + env: NodeJS.ProcessEnv = process.env, +): MissingCloudflareEnv | undefined { + for (const target of targets) { + if ((target.runtime ?? 'pi') !== 'pi') continue; + + // Check the model fields in precedence order; use the first one set. + const model = target.model ?? target.auxiliaryModel ?? target.synthesisModel; + if (!model || !isPiModelSelector(model)) continue; + + const slashIndex = model.indexOf('/'); + const provider = model.slice(0, slashIndex); + + if (provider !== 'cloudflare-workers-ai' && provider !== 'cloudflare-ai-gateway') continue; + + const missing: string[] = []; + + if (!env['CLOUDFLARE_ACCOUNT_ID'] && !env['WARDEN_CLOUDFLARE_ACCOUNT_ID']) { + missing.push('CLOUDFLARE_ACCOUNT_ID'); + } + + if (provider === 'cloudflare-ai-gateway') { + if (!env['CLOUDFLARE_GATEWAY_ID'] && !env['WARDEN_CLOUDFLARE_GATEWAY_ID']) { + missing.push('CLOUDFLARE_GATEWAY_ID'); + } + } + + if (missing.length > 0) { + return { provider, missing }; + } + } + + return undefined; +} + +/** + * Format the user-facing error for missing required Cloudflare env vars. + */ +export function missingCloudflareEnvMessage(missing: MissingCloudflareEnv): string { + const vars = missing.missing.map((v) => `${v} (or WARDEN_${v})`).join(', '); + return ( + `Pi provider ${missing.provider} requires additional environment variables: ${vars}. ` + + `Set these alongside CLOUDFLARE_API_KEY before running Warden.` + ); +} + /** * Find the first Pi runner option using a model ID that is not provider/model. */ diff --git a/packages/warden/src/sdk/runtimes/pi.ts b/packages/warden/src/sdk/runtimes/pi.ts index 9827fc0f..42de2375 100644 --- a/packages/warden/src/sdk/runtimes/pi.ts +++ b/packages/warden/src/sdk/runtimes/pi.ts @@ -131,6 +131,23 @@ function parseModelSelector(model: string): PiModelSelector { }; } +/** + * Throw a classified invalid-selector error when Pi's registry cannot resolve + * a provider/model pair. + * + * Converts the generic "Pi model not found" error into an + * InvalidPiModelSelectorError so it is classified as `invalid_model_selector` + * rather than `unknown`. This (a) opens the circuit breaker immediately and + * (b) gives the user an actionable message instead of "This usually indicates + * an authentication problem." + */ +function throwModelNotFound(model: string): never { + // Cloudflare-specific: @cf/ is a provider-native model ID, not a Pi selector. + // isPiModelSelector now rejects these at preflight, but guard here too in + // case the runtime is invoked directly without the CLI preflight. + throw new InvalidPiModelSelectorError({ option: 'model', model }); +} + function legacyApiKeyProvider(model: string | undefined): string | undefined { if (!model) { return 'anthropic'; @@ -160,9 +177,7 @@ function resolvePiModel( const { provider, modelId } = parseModelSelector(model); const resolved = registry.find(provider, modelId); if (!resolved) { - throw new Error( - `Pi model not found: ${model}. Use provider/model, for example openai/gpt-5.5.` - ); + throwModelNotFound(model); } return resolved; } diff --git a/packages/warden/src/utils/index.test.ts b/packages/warden/src/utils/index.test.ts index 90b9695a..b73d5b97 100644 --- a/packages/warden/src/utils/index.test.ts +++ b/packages/warden/src/utils/index.test.ts @@ -118,4 +118,50 @@ describe('bridgeWardenProviderApiKeyEnv', () => { expect(env['MODEL']).toBeUndefined(); expect(env['SENTRY_DSN']).toBeUndefined(); }); + + it('bridges WARDEN_CLOUDFLARE_ACCOUNT_ID to CLOUDFLARE_ACCOUNT_ID', () => { + const env: NodeJS.ProcessEnv = { + WARDEN_CLOUDFLARE_API_KEY: 'cf-key', + WARDEN_CLOUDFLARE_ACCOUNT_ID: 'cf-account', + }; + + bridgeWardenProviderApiKeyEnv(env); + + expect(env['CLOUDFLARE_API_KEY']).toBe('cf-key'); + expect(env['CLOUDFLARE_ACCOUNT_ID']).toBe('cf-account'); + }); + + it('bridges WARDEN_CLOUDFLARE_GATEWAY_ID to CLOUDFLARE_GATEWAY_ID', () => { + const env: NodeJS.ProcessEnv = { + WARDEN_CLOUDFLARE_API_KEY: 'cf-key', + WARDEN_CLOUDFLARE_ACCOUNT_ID: 'cf-account', + WARDEN_CLOUDFLARE_GATEWAY_ID: 'cf-gateway', + }; + + bridgeWardenProviderApiKeyEnv(env); + + expect(env['CLOUDFLARE_ACCOUNT_ID']).toBe('cf-account'); + expect(env['CLOUDFLARE_GATEWAY_ID']).toBe('cf-gateway'); + }); + + it('does not overwrite native CLOUDFLARE_ACCOUNT_ID when Warden alias is also set', () => { + const env: NodeJS.ProcessEnv = { + WARDEN_CLOUDFLARE_ACCOUNT_ID: 'warden-account', + CLOUDFLARE_ACCOUNT_ID: 'native-account', + }; + + bridgeWardenProviderApiKeyEnv(env); + + expect(env['CLOUDFLARE_ACCOUNT_ID']).toBe('native-account'); + }); + + it('does not bridge Cloudflare account ID when the Warden alias is empty', () => { + const env: NodeJS.ProcessEnv = { + WARDEN_CLOUDFLARE_ACCOUNT_ID: '', + }; + + bridgeWardenProviderApiKeyEnv(env); + + expect(env['CLOUDFLARE_ACCOUNT_ID']).toBeUndefined(); + }); }); diff --git a/packages/warden/src/utils/index.ts b/packages/warden/src/utils/index.ts index 4b57e832..5991e8dd 100644 --- a/packages/warden/src/utils/index.ts +++ b/packages/warden/src/utils/index.ts @@ -60,9 +60,34 @@ export function getAnthropicApiKey(): string | undefined { } /** - * Mirrors WARDEN-prefixed provider API keys to the env names expected by SDKs. + * Additional WARDEN-prefixed env vars to bridge for providers that require + * non-API-key credentials alongside their API key. + * + * These cannot be inferred from the WARDEN_X_API_KEY → X_API_KEY pattern + * because they do not follow the _API_KEY suffix convention. + * + * Each entry is [warden-alias, native-env-var]. + */ +const WARDEN_PROVIDER_ENV_BRIDGE: ReadonlyArray = [ + // Cloudflare Workers AI: requires account ID in addition to API key + ['WARDEN_CLOUDFLARE_ACCOUNT_ID', 'CLOUDFLARE_ACCOUNT_ID'], + // Cloudflare AI Gateway: additionally requires a gateway ID + ['WARDEN_CLOUDFLARE_GATEWAY_ID', 'CLOUDFLARE_GATEWAY_ID'], +]; + +/** + * Mirrors WARDEN-prefixed provider credentials to the env names expected by SDKs. + * + * Handles two classes of bridging: + * + * 1. Generic API keys: WARDEN_X_API_KEY → X_API_KEY for any provider. + * 2. Provider-specific non-key vars: explicit list for credentials that + * providers require beyond their API key (e.g. Cloudflare account ID). + * + * Existing native env vars are never overwritten. */ export function bridgeWardenProviderApiKeyEnv(env: NodeJS.ProcessEnv = process.env): void { + // Bridge WARDEN_X_API_KEY → X_API_KEY for all providers for (const [key, value] of Object.entries(env)) { if (!value || !key.startsWith('WARDEN_') || !key.endsWith('_API_KEY')) { continue; @@ -73,4 +98,12 @@ export function bridgeWardenProviderApiKeyEnv(env: NodeJS.ProcessEnv = process.e env[providerKey] = value; } } + + // Bridge provider-specific non-key credentials + for (const [wardenKey, nativeKey] of WARDEN_PROVIDER_ENV_BRIDGE) { + const value = env[wardenKey]; + if (value && !env[nativeKey]) { + env[nativeKey] = value; + } + } } diff --git a/skills/warden/references/config-schema.md b/skills/warden/references/config-schema.md index 5747a670..2331605a 100644 --- a/skills/warden/references/config-schema.md +++ b/skills/warden/references/config-schema.md @@ -128,6 +128,9 @@ Always skipped (cannot be overridden): | `WARDEN_MODEL` | Default model (lowest priority) | | `WARDEN_OPENAI_API_KEY` | OpenAI API key for OpenAI Pi models | | `WARDEN_ANTHROPIC_API_KEY` | Anthropic API key for Anthropic Pi models or Claude runtime | +| `WARDEN_CLOUDFLARE_API_KEY` | Cloudflare API key (bridged to `CLOUDFLARE_API_KEY`) | +| `WARDEN_CLOUDFLARE_ACCOUNT_ID` | Cloudflare account ID (required for Cloudflare providers; bridged to `CLOUDFLARE_ACCOUNT_ID`) | +| `WARDEN_CLOUDFLARE_GATEWAY_ID` | Cloudflare AI Gateway ID (required for `cloudflare-ai-gateway`; bridged to `CLOUDFLARE_GATEWAY_ID`) | | `WARDEN_STATE_DIR` | Override cache location (default: `~/.local/warden`) | | `WARDEN_SKILL_CACHE_TTL` | Cache TTL in seconds for unpinned remotes (default: 86400) | From 6f5bf47f357690876d4be1fad0189d5d3303c3eb Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Fri, 5 Jun 2026 08:23:50 +0000 Subject: [PATCH 2/5] fix(sdk): fix lint, add provider guidance for Google/Vercel/Cloudflare-gateway MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Fix oxlint: ReadonlyArray → as const array (fixes CI build failure) - Bridge WARDEN_VERCEL_AI_GATEWAY_API_KEY → AI_GATEWAY_API_KEY; Pi's env var for the vercel-ai-gateway provider is AI_GATEWAY_API_KEY, not the provider-name-derived form, so accept both aliases - Add targeted error hint for gemini/... model IDs: Pi's provider name for Google Gemini is 'google', not 'gemini'; guide users to google/ and WARDEN_GEMINI_API_KEY - Improve generic invalid-selector message for valid-shape but unknown-provider selectors: say 'unknown Pi provider: X' rather than 'must use provider/model format' when the format is already correct - Update piModelSelectorTip to emit Google-specific repair tip - Docs (config/models.mdx): add google/, vercel-ai-gateway/ rows; expand Cloudflare AI Gateway to show workers-ai/@cf/... and native passthrough model ID formats; add provider-specific notes on naming gotchas - Docs (config-schema.md): add WARDEN_GEMINI_API_KEY, WARDEN_OPENROUTER_API_KEY, WARDEN_AI_GATEWAY_API_KEY, WARDEN_VERCEL_AI_GATEWAY_API_KEY to env vars table - Tests: add gemini/... hint, unknown-provider message, Vercel bridge coverage Co-authored-by: immutable dcramer --- .../docs/src/content/docs/config/models.mdx | 29 +++++++++++-- .../src/sdk/runtimes/model-selectors.test.ts | 20 +++++++++ .../src/sdk/runtimes/model-selectors.ts | 41 +++++++++++++++++-- packages/warden/src/utils/index.test.ts | 34 +++++++++++++++ packages/warden/src/utils/index.ts | 8 +++- skills/warden/references/config-schema.md | 14 ++++--- 6 files changed, 131 insertions(+), 15 deletions(-) diff --git a/packages/docs/src/content/docs/config/models.mdx b/packages/docs/src/content/docs/config/models.mdx index f237d820..5be8125e 100644 --- a/packages/docs/src/content/docs/config/models.mdx +++ b/packages/docs/src/content/docs/config/models.mdx @@ -40,17 +40,38 @@ Examples: | --- | --- | | `openai/gpt-5.5` | `WARDEN_OPENAI_API_KEY` | | `anthropic/claude-sonnet-4-6` | `WARDEN_ANTHROPIC_API_KEY` | +| `google/` | `WARDEN_GEMINI_API_KEY` | | `fireworks/accounts/fireworks/models/kimi-k2p6` | `WARDEN_FIREWORKS_API_KEY` | | `groq/llama-3.3-70b-versatile` | `WARDEN_GROQ_API_KEY` | | `openrouter/meta-llama/llama-3.3-70b-instruct` | `WARDEN_OPENROUTER_API_KEY` | +| `vercel-ai-gateway/` | `WARDEN_AI_GATEWAY_API_KEY` or `WARDEN_VERCEL_AI_GATEWAY_API_KEY` | | `together/meta-llama/Llama-3.3-70B-Instruct-Turbo` | `WARDEN_TOGETHER_API_KEY` | -| `cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6` | `WARDEN_CLOUDFLARE_API_KEY` + `CLOUDFLARE_ACCOUNT_ID` | -| `cloudflare-ai-gateway/...` | `WARDEN_CLOUDFLARE_API_KEY` + `CLOUDFLARE_ACCOUNT_ID` + `CLOUDFLARE_GATEWAY_ID` | +| `cloudflare-workers-ai/@cf/` | `WARDEN_CLOUDFLARE_API_KEY` + `WARDEN_CLOUDFLARE_ACCOUNT_ID` | +| `cloudflare-ai-gateway/workers-ai/@cf/` | `WARDEN_CLOUDFLARE_API_KEY` + `WARDEN_CLOUDFLARE_ACCOUNT_ID` + `WARDEN_CLOUDFLARE_GATEWAY_ID` | +| `cloudflare-ai-gateway/` | `WARDEN_CLOUDFLARE_API_KEY` + `WARDEN_CLOUDFLARE_ACCOUNT_ID` + `WARDEN_CLOUDFLARE_GATEWAY_ID` | +| `cloudflare-ai-gateway/` | `WARDEN_CLOUDFLARE_API_KEY` + `WARDEN_CLOUDFLARE_ACCOUNT_ID` + `WARDEN_CLOUDFLARE_GATEWAY_ID` | Warden splits Pi selectors at the first `/`. The provider comes before it and the provider-specific model ID comes after it, so model IDs may contain additional -slashes. For example, `cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6` sets -provider `cloudflare-workers-ai` and model ID `@cf/moonshotai/kimi-k2.6`. +slashes. + +**Notes on specific providers:** + +- **Google Gemini:** Pi's provider name is `google`, not `gemini`. The credential env var is + `GEMINI_API_KEY` (or `WARDEN_GEMINI_API_KEY`). Use `google/`, not `gemini/`. + +- **Vercel AI Gateway:** Pi's env var is `AI_GATEWAY_API_KEY`. The canonical Warden alias is + `WARDEN_AI_GATEWAY_API_KEY`. `WARDEN_VERCEL_AI_GATEWAY_API_KEY` is also accepted as a + convenience alias and will be bridged automatically. + +- **Cloudflare Workers AI:** Requires both `CLOUDFLARE_API_KEY` and `CLOUDFLARE_ACCOUNT_ID`. + Use the `@cf/provider/model-id` model format: e.g. `cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6`. + +- **Cloudflare AI Gateway:** Routes OpenAI/Anthropic models using their native IDs + (e.g. `cloudflare-ai-gateway/gpt-5.1`, `cloudflare-ai-gateway/claude-sonnet-4-5`). + Workers AI models use the `workers-ai/@cf/...` prefix. Upstream credentials are managed in + Cloudflare's AI Gateway dashboard (unified billing, stored BYOK) — Warden's provider API keys + are not forwarded through the gateway. **Credential convention:** Set `WARDEN_{PROVIDER}_API_KEY` to authenticate with a provider. Warden mirrors these to the native `{PROVIDER}_API_KEY` expected by each SDK at runtime. diff --git a/packages/warden/src/sdk/runtimes/model-selectors.test.ts b/packages/warden/src/sdk/runtimes/model-selectors.test.ts index 843ebe30..1ddd244a 100644 --- a/packages/warden/src/sdk/runtimes/model-selectors.test.ts +++ b/packages/warden/src/sdk/runtimes/model-selectors.test.ts @@ -56,6 +56,20 @@ describe('invalidPiModelSelectorMessage', () => { expect(msg).toContain('provider-native'); }); + it('emits targeted guidance for gemini/... (wrong provider name for Google)', () => { + const msg = invalidPiModelSelectorMessage({ option: 'model', model: 'gemini/gemini-2.5-flash' }); + expect(msg).toContain('google/gemini-2.5-flash'); + expect(msg).toContain('WARDEN_GEMINI_API_KEY'); + expect(msg).not.toContain('must use provider/model format'); + }); + + it('emits unknown-provider guidance for valid-shape selectors with unrecognized providers', () => { + const msg = invalidPiModelSelectorMessage({ option: 'model', model: 'unknown-provider/gpt-5.5' }); + expect(msg).toContain('unknown Pi provider'); + expect(msg).toContain('unknown-provider'); + expect(msg).not.toContain('must use provider/model format'); + }); + it('emits standard format guidance for plain model IDs without a provider', () => { const msg = invalidPiModelSelectorMessage({ option: 'model', model: 'gpt-5.5' }); expect(msg).toContain('provider/model format'); @@ -84,6 +98,12 @@ describe('piModelSelectorTip', () => { expect(tip).toContain('provider-name/@vendor/model'); }); + it('gives targeted Google tip for gemini/... models', () => { + const tip = piModelSelectorTip('gemini/gemini-2.5-flash'); + expect(tip).toContain('google/gemini-2.5-flash'); + expect(tip).toContain('WARDEN_GEMINI_API_KEY'); + }); + it('gives standard tip for plain model IDs', () => { const tip = piModelSelectorTip('gpt-5.5'); expect(tip).toContain('anthropic/claude-sonnet-4-6'); diff --git a/packages/warden/src/sdk/runtimes/model-selectors.ts b/packages/warden/src/sdk/runtimes/model-selectors.ts index 92446e69..783ff84d 100644 --- a/packages/warden/src/sdk/runtimes/model-selectors.ts +++ b/packages/warden/src/sdk/runtimes/model-selectors.ts @@ -39,12 +39,16 @@ export interface InvalidPiModelSelector { /** * Format the user-facing error for an invalid Pi model selector. * - * Emits targeted guidance for known misuse patterns such as Cloudflare - * Workers AI native model IDs being passed without a Pi provider prefix. + * Emits targeted guidance for known misuse patterns: + * - @cf/... Cloudflare-native model IDs without a Pi provider prefix + * - Other @namespace/... provider-native IDs without a Pi provider prefix + * - Common wrong provider names (e.g. "gemini" instead of "google") + * - Valid-shape selectors with an unknown Pi provider name */ export function invalidPiModelSelectorMessage(invalid: InvalidPiModelSelector): string { const target = invalid.specName ? ` for ${invalid.specName}` : ''; const { model } = invalid; + const slashIndex = model.indexOf('/'); // Cloudflare Workers AI native model IDs start with @cf/. Users sometimes // set these directly instead of using the Warden Pi provider prefix. @@ -58,7 +62,6 @@ export function invalidPiModelSelectorMessage(invalid: InvalidPiModelSelector): } // Generic provider-native namespace prefix (@vendor/...) without a Pi provider. - const slashIndex = model.indexOf('/'); if (model.startsWith('@') && slashIndex > 0) { const namespace = model.slice(0, slashIndex + 1); return ( @@ -69,6 +72,29 @@ export function invalidPiModelSelectorMessage(invalid: InvalidPiModelSelector): ); } + // When the model already has provider/model shape, the selector is structurally + // valid but the provider segment is either wrong or unknown to Pi. + if (slashIndex > 0 && slashIndex < model.length - 1) { + const provider = model.slice(0, slashIndex); + + // Google Gemini: Pi provider name is "google", env var is GEMINI_API_KEY. + // Users commonly guess "gemini/..." from the product name. + if (provider === 'gemini') { + const modelId = model.slice(slashIndex + 1); + return ( + `Pi runtime ${invalid.option}${target} received "gemini/..." but Google Gemini's Pi provider name is "google", not "gemini". ` + + `Use google/${modelId} as the selector and set WARDEN_GEMINI_API_KEY or GEMINI_API_KEY.` + ); + } + + // Valid shape but unknown provider — give a pointer to the docs rather than + // saying "must use provider/model format" (which is already met). + return ( + `Pi runtime ${invalid.option}${target} uses an unknown Pi provider: "${provider}" in ${model}. ` + + `See https://warden.sentry.dev/config/models for supported providers and selectors.` + ); + } + return `Pi runtime ${invalid.option}${target} must use provider/model format: ${model}`; } @@ -83,7 +109,14 @@ export function piModelSelectorTip(model: string): string { if (model.startsWith('@')) { return 'Prefix the model with its Pi provider name, e.g. provider-name/@vendor/model-id.'; } - return 'Set a Pi model selector such as anthropic/claude-sonnet-4-6.'; + const slashIndex = model.indexOf('/'); + if (slashIndex > 0) { + const provider = model.slice(0, slashIndex); + if (provider === 'gemini') { + return `Use google/${model.slice(slashIndex + 1)} (Pi provider name is "google") and set WARDEN_GEMINI_API_KEY.`; + } + } + return 'Set a Pi model selector such as anthropic/claude-sonnet-4-6 or google/gemini-2.5-flash.'; } /** diff --git a/packages/warden/src/utils/index.test.ts b/packages/warden/src/utils/index.test.ts index b73d5b97..20894580 100644 --- a/packages/warden/src/utils/index.test.ts +++ b/packages/warden/src/utils/index.test.ts @@ -164,4 +164,38 @@ describe('bridgeWardenProviderApiKeyEnv', () => { expect(env['CLOUDFLARE_ACCOUNT_ID']).toBeUndefined(); }); + + it('bridges WARDEN_VERCEL_AI_GATEWAY_API_KEY to AI_GATEWAY_API_KEY', () => { + const env: NodeJS.ProcessEnv = { + WARDEN_VERCEL_AI_GATEWAY_API_KEY: 'vercel-key', + }; + + bridgeWardenProviderApiKeyEnv(env); + + expect(env['AI_GATEWAY_API_KEY']).toBe('vercel-key'); + }); + + it('does not overwrite AI_GATEWAY_API_KEY when already set via WARDEN_AI_GATEWAY_API_KEY', () => { + const env: NodeJS.ProcessEnv = { + WARDEN_AI_GATEWAY_API_KEY: 'direct-alias', + WARDEN_VERCEL_AI_GATEWAY_API_KEY: 'provider-alias', + }; + + bridgeWardenProviderApiKeyEnv(env); + + // Generic API-key bridge runs first: WARDEN_AI_GATEWAY_API_KEY -> AI_GATEWAY_API_KEY + // Explicit bridge then does not overwrite the already-set value. + expect(env['AI_GATEWAY_API_KEY']).toBe('direct-alias'); + }); + + it('does not overwrite native AI_GATEWAY_API_KEY when Vercel alias is set', () => { + const env: NodeJS.ProcessEnv = { + WARDEN_VERCEL_AI_GATEWAY_API_KEY: 'warden-key', + AI_GATEWAY_API_KEY: 'native-key', + }; + + bridgeWardenProviderApiKeyEnv(env); + + expect(env['AI_GATEWAY_API_KEY']).toBe('native-key'); + }); }); diff --git a/packages/warden/src/utils/index.ts b/packages/warden/src/utils/index.ts index 5991e8dd..5470b555 100644 --- a/packages/warden/src/utils/index.ts +++ b/packages/warden/src/utils/index.ts @@ -68,12 +68,16 @@ export function getAnthropicApiKey(): string | undefined { * * Each entry is [warden-alias, native-env-var]. */ -const WARDEN_PROVIDER_ENV_BRIDGE: ReadonlyArray = [ +const WARDEN_PROVIDER_ENV_BRIDGE = [ // Cloudflare Workers AI: requires account ID in addition to API key ['WARDEN_CLOUDFLARE_ACCOUNT_ID', 'CLOUDFLARE_ACCOUNT_ID'], // Cloudflare AI Gateway: additionally requires a gateway ID ['WARDEN_CLOUDFLARE_GATEWAY_ID', 'CLOUDFLARE_GATEWAY_ID'], -]; + // Vercel AI Gateway: Pi uses AI_GATEWAY_API_KEY, not VERCEL_AI_GATEWAY_API_KEY. + // Accept the provider-name-derived alias so users following the WARDEN_{PROVIDER}_API_KEY + // convention can discover the correct native env var through Warden's bridging. + ['WARDEN_VERCEL_AI_GATEWAY_API_KEY', 'AI_GATEWAY_API_KEY'], +] as const; /** * Mirrors WARDEN-prefixed provider credentials to the env names expected by SDKs. diff --git a/skills/warden/references/config-schema.md b/skills/warden/references/config-schema.md index 2331605a..230b3153 100644 --- a/skills/warden/references/config-schema.md +++ b/skills/warden/references/config-schema.md @@ -126,11 +126,15 @@ Always skipped (cannot be overridden): | Variable | Purpose | |----------|---------| | `WARDEN_MODEL` | Default model (lowest priority) | -| `WARDEN_OPENAI_API_KEY` | OpenAI API key for OpenAI Pi models | -| `WARDEN_ANTHROPIC_API_KEY` | Anthropic API key for Anthropic Pi models or Claude runtime | -| `WARDEN_CLOUDFLARE_API_KEY` | Cloudflare API key (bridged to `CLOUDFLARE_API_KEY`) | -| `WARDEN_CLOUDFLARE_ACCOUNT_ID` | Cloudflare account ID (required for Cloudflare providers; bridged to `CLOUDFLARE_ACCOUNT_ID`) | -| `WARDEN_CLOUDFLARE_GATEWAY_ID` | Cloudflare AI Gateway ID (required for `cloudflare-ai-gateway`; bridged to `CLOUDFLARE_GATEWAY_ID`) | +| `WARDEN_OPENAI_API_KEY` | OpenAI API key (Pi provider `openai`); bridged to `OPENAI_API_KEY` | +| `WARDEN_ANTHROPIC_API_KEY` | Anthropic API key (Pi provider `anthropic` or Claude runtime); bridged to `ANTHROPIC_API_KEY` | +| `WARDEN_GEMINI_API_KEY` | Google Gemini API key (Pi provider `google`); bridged to `GEMINI_API_KEY`. **Note:** Pi provider name is `google`, not `gemini`; use `google/` selectors. | +| `WARDEN_OPENROUTER_API_KEY` | OpenRouter API key (Pi provider `openrouter`); bridged to `OPENROUTER_API_KEY` | +| `WARDEN_AI_GATEWAY_API_KEY` | Vercel AI Gateway API key (Pi provider `vercel-ai-gateway`); bridged to `AI_GATEWAY_API_KEY` | +| `WARDEN_VERCEL_AI_GATEWAY_API_KEY` | Convenience alias for Vercel AI Gateway; bridged to `AI_GATEWAY_API_KEY`. Accepted in addition to `WARDEN_AI_GATEWAY_API_KEY`. | +| `WARDEN_CLOUDFLARE_API_KEY` | Cloudflare API key for `cloudflare-workers-ai` or `cloudflare-ai-gateway`; bridged to `CLOUDFLARE_API_KEY` | +| `WARDEN_CLOUDFLARE_ACCOUNT_ID` | Cloudflare account ID (required for all Cloudflare providers); bridged to `CLOUDFLARE_ACCOUNT_ID` | +| `WARDEN_CLOUDFLARE_GATEWAY_ID` | Cloudflare AI Gateway ID (required for `cloudflare-ai-gateway`); bridged to `CLOUDFLARE_GATEWAY_ID` | | `WARDEN_STATE_DIR` | Override cache location (default: `~/.local/warden`) | | `WARDEN_SKILL_CACHE_TTL` | Cache TTL in seconds for unpinned remotes (default: 86400) | From ff3a2c281240af85f317185fc8909da3dde00a9b Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Fri, 5 Jun 2026 08:31:56 +0000 Subject: [PATCH 3/5] fix(sdk): check all model lanes for Cloudflare env, fix registry-miss message MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two issues from inline review feedback: 1. findMissingCloudflareEnv used target.model ?? auxiliaryModel ?? synthesisModel, so only the first set lane was checked. A config mixing anthropic/... as the primary model with cloudflare-workers-ai/... as auxiliaryModel would pass the preflight without CLOUDFLARE_ACCOUNT_ID. Fix: loop over all three lanes independently and return on the first lane that needs a missing var. 2. When Pi's registry cannot resolve a selector that already passes isPiModelSelector, throwModelNotFound raised InvalidPiModelSelectorError whose message said 'unknown Pi provider: openai' for a known provider with a stale model ID. Fix: change the generic valid-shape fallback to 'could not find provider or model: ...' — accurate for both unknown-provider and known-provider/stale-model cases without mis-diagnosing the root cause. Fixes two medium-severity issues from sentry-warden and Cursor Bugbot review. Co-authored-by: immutable dcramer --- .../src/sdk/runtimes/model-selectors.test.ts | 46 ++++++++++++++++-- .../src/sdk/runtimes/model-selectors.ts | 47 +++++++++++-------- 2 files changed, 71 insertions(+), 22 deletions(-) diff --git a/packages/warden/src/sdk/runtimes/model-selectors.test.ts b/packages/warden/src/sdk/runtimes/model-selectors.test.ts index 1ddd244a..3bec4cfa 100644 --- a/packages/warden/src/sdk/runtimes/model-selectors.test.ts +++ b/packages/warden/src/sdk/runtimes/model-selectors.test.ts @@ -63,11 +63,21 @@ describe('invalidPiModelSelectorMessage', () => { expect(msg).not.toContain('must use provider/model format'); }); - it('emits unknown-provider guidance for valid-shape selectors with unrecognized providers', () => { + it('emits provider-or-model-not-found guidance for valid-shape selectors (unknown or stale model)', () => { + // Covers both: unknown provider name and known provider with stale/wrong model ID const msg = invalidPiModelSelectorMessage({ option: 'model', model: 'unknown-provider/gpt-5.5' }); - expect(msg).toContain('unknown Pi provider'); - expect(msg).toContain('unknown-provider'); + expect(msg).toContain('could not find provider or model'); + expect(msg).toContain('unknown-provider/gpt-5.5'); expect(msg).not.toContain('must use provider/model format'); + expect(msg).not.toContain('unknown Pi provider'); + }); + + it('uses same could-not-find wording for known providers with stale model IDs', () => { + const msg = invalidPiModelSelectorMessage({ option: 'model', model: 'openai/nonexistent-model' }); + expect(msg).toContain('could not find provider or model'); + expect(msg).toContain('openai/nonexistent-model'); + // Does not claim openai is an unknown provider + expect(msg).not.toContain('unknown Pi provider'); }); it('emits standard format guidance for plain model IDs without a provider', () => { @@ -226,4 +236,34 @@ describe('findMissingCloudflareEnv', () => { ); expect(result).toBeUndefined(); }); + + it('detects missing account ID when Cloudflare provider is in auxiliaryModel but not model', () => { + const result = findMissingCloudflareEnv( + [{ + runtime: 'pi', + model: 'anthropic/claude-sonnet-4-6', + auxiliaryModel: 'cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6', + }], + { CLOUDFLARE_API_KEY: 'key' }, + ); + expect(result).toMatchObject({ + provider: 'cloudflare-workers-ai', + missing: ['CLOUDFLARE_ACCOUNT_ID'], + }); + }); + + it('detects missing account ID when Cloudflare provider is in synthesisModel only', () => { + const result = findMissingCloudflareEnv( + [{ + runtime: 'pi', + model: 'openai/gpt-5.5', + synthesisModel: 'cloudflare-workers-ai/@cf/moonshotai/kimi-k2.6', + }], + { CLOUDFLARE_API_KEY: 'key' }, + ); + expect(result).toMatchObject({ + provider: 'cloudflare-workers-ai', + missing: ['CLOUDFLARE_ACCOUNT_ID'], + }); + }); }); diff --git a/packages/warden/src/sdk/runtimes/model-selectors.ts b/packages/warden/src/sdk/runtimes/model-selectors.ts index 783ff84d..3f71a8ee 100644 --- a/packages/warden/src/sdk/runtimes/model-selectors.ts +++ b/packages/warden/src/sdk/runtimes/model-selectors.ts @@ -87,10 +87,14 @@ export function invalidPiModelSelectorMessage(invalid: InvalidPiModelSelector): ); } - // Valid shape but unknown provider — give a pointer to the docs rather than - // saying "must use provider/model format" (which is already met). + // Valid shape but provider/model could not be resolved — this happens either + // when the provider is not registered in Pi (unknown provider) or when the + // provider is known but the model ID is wrong or stale. Avoid saying + // "unknown provider" because it may mislead users with a valid provider + // name and a stale model ID. return ( - `Pi runtime ${invalid.option}${target} uses an unknown Pi provider: "${provider}" in ${model}. ` + + `Pi runtime ${invalid.option}${target} could not find provider or model: ${model}. ` + + `Verify the Pi provider name and model ID are correct. ` + `See https://warden.sentry.dev/config/models for supported providers and selectors.` ); } @@ -158,29 +162,34 @@ export function findMissingCloudflareEnv( for (const target of targets) { if ((target.runtime ?? 'pi') !== 'pi') continue; - // Check the model fields in precedence order; use the first one set. - const model = target.model ?? target.auxiliaryModel ?? target.synthesisModel; - if (!model || !isPiModelSelector(model)) continue; + // Check each model lane independently. A target can mix providers across + // lanes (e.g. anthropic for `model`, cloudflare-workers-ai for + // `auxiliaryModel`), so each lane that resolves to a Cloudflare provider + // must be validated independently. + for (const lane of ['model', 'auxiliaryModel', 'synthesisModel'] as const) { + const model = target[lane]; + if (!model || !isPiModelSelector(model)) continue; - const slashIndex = model.indexOf('/'); - const provider = model.slice(0, slashIndex); + const slashIndex = model.indexOf('/'); + const provider = model.slice(0, slashIndex); - if (provider !== 'cloudflare-workers-ai' && provider !== 'cloudflare-ai-gateway') continue; + if (provider !== 'cloudflare-workers-ai' && provider !== 'cloudflare-ai-gateway') continue; - const missing: string[] = []; + const missing: string[] = []; - if (!env['CLOUDFLARE_ACCOUNT_ID'] && !env['WARDEN_CLOUDFLARE_ACCOUNT_ID']) { - missing.push('CLOUDFLARE_ACCOUNT_ID'); - } + if (!env['CLOUDFLARE_ACCOUNT_ID'] && !env['WARDEN_CLOUDFLARE_ACCOUNT_ID']) { + missing.push('CLOUDFLARE_ACCOUNT_ID'); + } - if (provider === 'cloudflare-ai-gateway') { - if (!env['CLOUDFLARE_GATEWAY_ID'] && !env['WARDEN_CLOUDFLARE_GATEWAY_ID']) { - missing.push('CLOUDFLARE_GATEWAY_ID'); + if (provider === 'cloudflare-ai-gateway') { + if (!env['CLOUDFLARE_GATEWAY_ID'] && !env['WARDEN_CLOUDFLARE_GATEWAY_ID']) { + missing.push('CLOUDFLARE_GATEWAY_ID'); + } } - } - if (missing.length > 0) { - return { provider, missing }; + if (missing.length > 0) { + return { provider, missing }; + } } } From c99f4352795bdadf682f587e7364ed090c27b9e1 Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Fri, 5 Jun 2026 08:55:51 +0000 Subject: [PATCH 4/5] fix(sdk): add Cloudflare preflight to runSkills, fix uppercase-provider message Two issues from Cursor Bugbot inline review: 1. findMissingCloudflareEnv was called in runConfigMode (PR/trigger path) but not in runSkills, which backs --skill, file-mode, and git-ref runs. Any Cloudflare Pi config using those paths would skip the startup preflight and fail later with per-chunk errors instead of the fast targeted message. Added the same check to runSkills immediately after findInvalidPiModelSelector. 2. invalidPiModelSelectorMessage entered the 'valid shape' branch (slashIndex in bounds) for selectors like OPENAI/gpt-5.5 where the provider segment fails the Pi naming regex (uppercase). That produced 'could not find provider or model' which implies a registry miss rather than a format error. Added an explicit regex check at the top of the branch: providers with invalid characters get 'invalid provider segment ... Pi provider names use lowercase letters, digits, and hyphens'; only providers passing the regex reach the registry-miss or known-alias branches. Co-authored-by: immutable dcramer --- packages/warden/src/cli/main.ts | 5 ++++ .../src/sdk/runtimes/model-selectors.test.ts | 10 +++++++ .../src/sdk/runtimes/model-selectors.ts | 28 +++++++++++++------ 3 files changed, 35 insertions(+), 8 deletions(-) diff --git a/packages/warden/src/cli/main.ts b/packages/warden/src/cli/main.ts index c16710b8..2ae6ac93 100644 --- a/packages/warden/src/cli/main.ts +++ b/packages/warden/src/cli/main.ts @@ -1198,6 +1198,11 @@ export async function runSkills( emitInvalidPiModelSelectorRunLog(repoPath ?? cwd, options, invalidModelSelector); return 1; } + const missingCloudflare = findMissingCloudflareEnv(specs.map((s) => ({ ...s.runnerOptions }))); + if (missingCloudflare) { + reportMissingCloudflareEnv(reporter, missingCloudflare); + return 1; + } let tasks: SkillTaskOptions[]; const concurrency = options.parallel ?? DEFAULT_CONCURRENCY; try { diff --git a/packages/warden/src/sdk/runtimes/model-selectors.test.ts b/packages/warden/src/sdk/runtimes/model-selectors.test.ts index 3bec4cfa..5cf0c6b2 100644 --- a/packages/warden/src/sdk/runtimes/model-selectors.test.ts +++ b/packages/warden/src/sdk/runtimes/model-selectors.test.ts @@ -63,6 +63,16 @@ describe('invalidPiModelSelectorMessage', () => { expect(msg).not.toContain('must use provider/model format'); }); + it('emits invalid-provider-segment guidance when provider fails Pi naming rules', () => { + // e.g. uppercase, underscores — valid shape but invalid provider segment format + const msg = invalidPiModelSelectorMessage({ option: 'model', model: 'OPENAI/gpt-5.5' }); + expect(msg).toContain('invalid provider segment'); + expect(msg).toContain('OPENAI'); + expect(msg).toContain('lowercase'); + expect(msg).not.toContain('could not find provider or model'); + expect(msg).not.toContain('must use provider/model format'); + }); + it('emits provider-or-model-not-found guidance for valid-shape selectors (unknown or stale model)', () => { // Covers both: unknown provider name and known provider with stale/wrong model ID const msg = invalidPiModelSelectorMessage({ option: 'model', model: 'unknown-provider/gpt-5.5' }); diff --git a/packages/warden/src/sdk/runtimes/model-selectors.ts b/packages/warden/src/sdk/runtimes/model-selectors.ts index 3f71a8ee..7cceb01c 100644 --- a/packages/warden/src/sdk/runtimes/model-selectors.ts +++ b/packages/warden/src/sdk/runtimes/model-selectors.ts @@ -72,12 +72,27 @@ export function invalidPiModelSelectorMessage(invalid: InvalidPiModelSelector): ); } - // When the model already has provider/model shape, the selector is structurally - // valid but the provider segment is either wrong or unknown to Pi. + // The model has provider/model shape (contains a slash with content on both sides). + // Distinguish between three sub-cases: + // (a) provider segment has invalid format (fails Pi naming rules) + // (b) known wrong provider alias (e.g. "gemini" instead of "google") + // (c) valid format and provider alias, but not found in Pi's registry if (slashIndex > 0 && slashIndex < model.length - 1) { const provider = model.slice(0, slashIndex); - // Google Gemini: Pi provider name is "google", env var is GEMINI_API_KEY. + // (a) Provider segment contains characters Pi doesn't accept. + // Pi provider names use lowercase letters, digits, and hyphens only. + // This catches e.g. OPENAI/gpt-5.5 or My_Provider/model being rejected + // by the regex — the issue is the format, not a registry miss. + if (!/^[a-z0-9][a-z0-9-]*$/.test(provider)) { + return ( + `Pi runtime ${invalid.option}${target} has an invalid provider segment "${provider}" in ${model}. ` + + `Pi provider names use lowercase letters, digits, and hyphens (e.g. openai, cloudflare-workers-ai). ` + + `See https://warden.sentry.dev/config/models for supported providers and selectors.` + ); + } + + // (b) Google Gemini: Pi provider name is "google", env var is GEMINI_API_KEY. // Users commonly guess "gemini/..." from the product name. if (provider === 'gemini') { const modelId = model.slice(slashIndex + 1); @@ -87,11 +102,8 @@ export function invalidPiModelSelectorMessage(invalid: InvalidPiModelSelector): ); } - // Valid shape but provider/model could not be resolved — this happens either - // when the provider is not registered in Pi (unknown provider) or when the - // provider is known but the model ID is wrong or stale. Avoid saying - // "unknown provider" because it may mislead users with a valid provider - // name and a stale model ID. + // (c) Valid format, valid name, but provider or model not in Pi's registry. + // Covers both unknown providers and known providers with stale model IDs. return ( `Pi runtime ${invalid.option}${target} could not find provider or model: ${model}. ` + `Verify the Pi provider name and model ID are correct. ` + From 89a4d6128129068674c7494bb3c8c701a71e9b77 Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Fri, 5 Jun 2026 09:05:37 +0000 Subject: [PATCH 5/5] fix(cli): emit run log when Cloudflare env preflight fails When findMissingCloudflareEnv fails, both call sites (runSkills and runConfigMode) reported the error to the reporter and returned 1, but neither wrote an on-disk JSONL run log. The invalid-model-selector check directly above calls emitInvalidPiModelSelectorRunLog for the same purpose; missing that step breaks the audit trail other early-exit config errors use (auth_failed, invalid_model_selector) and leaves warden runs --log with no record of the failure. Add emitMissingCloudflareEnvRunLog (code: auth_failed) and call it at both runSkills and runConfigMode after reportMissingCloudflareEnv. Co-authored-by: immutable dcramer --- packages/warden/src/cli/main.ts | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/packages/warden/src/cli/main.ts b/packages/warden/src/cli/main.ts index 2ae6ac93..7dce7de9 100644 --- a/packages/warden/src/cli/main.ts +++ b/packages/warden/src/cli/main.ts @@ -745,6 +745,18 @@ function reportMissingCloudflareEnv(reporter: Reporter, missing: MissingCloudfla reporter.tip(tip); } +function emitMissingCloudflareEnvRunLog( + repoPath: string, + options: CLIOptions, + missing: MissingCloudflareEnv, +): void { + emitEmptyRunLog(repoPath, options, { + code: 'auth_failed', + message: missingCloudflareEnvMessage(missing), + timestamp: new Date().toISOString(), + }); +} + function emitInvalidPiModelSelectorRunLog( repoPath: string, options: CLIOptions, @@ -1201,6 +1213,7 @@ export async function runSkills( const missingCloudflare = findMissingCloudflareEnv(specs.map((s) => ({ ...s.runnerOptions }))); if (missingCloudflare) { reportMissingCloudflareEnv(reporter, missingCloudflare); + emitMissingCloudflareEnvRunLog(repoPath ?? cwd, options, missingCloudflare); return 1; } let tasks: SkillTaskOptions[]; @@ -1536,6 +1549,7 @@ async function runConfigMode(options: CLIOptions, reporter: Reporter): Promise ({ ...s.runnerOptions }))); if (missingCloudflare) { reportMissingCloudflareEnv(reporter, missingCloudflare); + emitMissingCloudflareEnvRunLog(repoPath, options, missingCloudflare); return 1; } let tasks: SkillTaskOptions[];