fix(agent): kind-aware retry with Retry-After and a real delay cap

.pylon/features.yaml

@@ -281,3 +281,18 @@ decisions:
     revisit_when:
       - Prime routes subagent and derived-request provider hooks through the owning session's extension runner.
       - An upstream extension contract exposes per-agent provider identity that supersedes the scoped session-id view.
+  kind-aware-provider-retry:
+    area: runtime-reliability
+    state: shipped
+    owner: pylon-prime-integration
+    decision: redesign
+    pylon_refs:
+      - https://github.com/pylon-code/prime-agent/issues/22
+      - https://github.com/pylon-code/prime-agent/issues/24
+    upstream_refs:
+      - https://github.com/PrimeIntellect-ai/prime-agent/tree/a903d4b6768f484bd6d459b7b0aa7dee38e461e2
+    fork_change: kind-aware-session-retry-v1
+    upstream_support: Prime through a903d4b6768f classifies provider stream failures but retries every one of them on the same fixed 2s/4s/8s ladder, stacks provider SDK retries underneath the session loop, never reads Retry-After, and threads maxRetryDelayMs through StreamOptions, Agent, and the proxy without any provider reading it.
+    revisit_when:
+      - Prime upstream makes the session retry loop failure-kind aware and honors Retry-After with a delay ceiling.
+      - Prime upstream makes exactly one retry layer own policy so provider SDK retries cannot multiply session retries.

.pylon/upstream-review.md

@@ -109,6 +109,17 @@ This ledger records Prime upstream evidence and the decision taken for each over
 - Cross-repository merge order: Prime issue #17 and a reproducible artifact, Pylon issue #190 consuming the exact post-attach proof, then Comet issue #7. Exact committed-head API/security/test review and trusted hosted CI remain mandatory. Revisit when upstream offers an equivalent generation-scoped proof and both consumers can remove this token without weakening fail-closed negotiation.
 
 
+## 2026-08-30 — kind-aware provider retry candidate
+
+- Upstream baseline: `PrimeIntellect-ai/prime-agent@a903d4b6768f484bd6d459b7b0aa7dee38e461e2`; latest audited release remains `v0.8.1`. This candidate does not advance `reviewed_upstream_commit`.
+- Searched current Prime issues, pull requests, and source for retry backoff, `Retry-After`, and `maxRetryDelayMs`. Upstream has no kind-aware retry work and still carries the same dead `maxRetryDelayMs` threading through `packages/ai/src/types.ts`, `packages/ai/src/providers/simple-options.ts`, `packages/agent/src/agent.ts`, `packages/agent/src/proxy.ts`, `packages/coding-agent/src/core/sdk.ts`, and `packages/coding-agent/src/core/side-question.ts`. The behavior is therefore not superseded by an upstream API.
+- `kind-aware-provider-retry`: **redesign**. Prime classifies provider stream failures but retries every one on the same fixed 2s/4s/8s ladder, stacks SDK retries under the session loop, and never reads `Retry-After`. Pylon replaces both layers with one policy: the session loop owns retry, provider `maxRetries` defaults to `0` while `retry.enabled` is true, and `retry.provider.maxRetryDelayMs` becomes the enforced ceiling on provider-requested waits instead of a silent no-op setting.
+- Policy: `auth` aborts the ladder immediately; `rate_limit` and `overloaded` back off from 15s and 10s with full jitter floored at `retry.baseDelayMs` and capped by `maxRetryDelayMs`, honoring `Retry-After` plus up to 1s of spread when the provider sends one, and failing fast when the requested wait exceeds the cap; every other kind keeps the historical ladder unchanged.
+- Wire compatibility: the `auto_retry_start` and `auto_retry_end` session events keep their exact shapes, so there is no daemon protocol, capability, or schema-revision change. `retry.provider.maxRetryDelayMs` keeps its settings key and existing `retry.maxDelayMs` migration.
+- Behavior change accepted deliberately: structured auth failures previously retried once before marking auth stale. They now abort on the first failure while still marking the auth source stale and appending login guidance. The affected assertions in `agent-session-retry-events.test.ts` and regression `4491-provider-stale-after-401.test.ts` were updated; the unstructured 401 path continues to exercise mid-backoff credential rotation and cancellation.
+- Validation: `npm run check` clean. Focused runs pass `packages/ai` stream-failure 35/35 and Anthropic SSE parsing 7/7; `packages/coding-agent` retry-backoff 10/10, issue #24 regression 7/7, and the retry/auth suites 38/38; `packages/agent` 60/60; settings and telemetry 55/55.
+- Revisit when Prime upstream makes its session retry loop failure-kind aware, honors `Retry-After` with a delay ceiling, and gives exactly one layer ownership of retry policy.
+
 ## 2026-08-31 — negotiated proof shipped and mixed-version snapshot catch-up follow-up
 
 - Upstream evidence remains fully audited through `PrimeIntellect-ai/prime-agent@a903d4b6768f484bd6d459b7b0aa7dee38e461e2`, the product base used by PR #19. The only later upstream-main commit currently visible is `c382f09856d4a8c8d2b765179657047d58691f25` (PR #1893, terminal Mermaid rendering); its changed paths do not overlap daemon snapshot, worker, supervisor, framing, or recovery code and it does not supersede this boundary.

packages/agent/.changes/24-kind-aware-retry.md

@@ -0,0 +1 @@
+- Removed the unused `maxRetryDelayMs` agent and proxy option; it was threaded to providers that never read it ([#24](https://github.com/pylon-code/prime-agent/issues/24)).

packages/agent/src/agent.ts

@@ -112,7 +112,6 @@ export interface AgentOptions {
 	sessionId?: string;
 	thinkingBudgets?: ThinkingBudgets;
 	transport?: Transport;
-	maxRetryDelayMs?: number;
 	toolExecution?: ToolExecutionMode;
 }
 
@@ -216,7 +215,6 @@ export class Agent {
 	public sessionId?: string;
 	public thinkingBudgets?: ThinkingBudgets;
 	public transport: Transport;
-	public maxRetryDelayMs?: number;
 	public toolExecution: ToolExecutionMode;
 
 	constructor(options: AgentOptions = {}) {
@@ -237,7 +235,6 @@ export class Agent {
 		this.sessionId = options.sessionId;
 		this.thinkingBudgets = options.thinkingBudgets;
 		this.transport = options.transport ?? "auto";
-		this.maxRetryDelayMs = options.maxRetryDelayMs;
 		this.toolExecution = options.toolExecution ?? "parallel";
 	}
 
@@ -470,7 +467,6 @@ export class Agent {
 			onResponse: this.onResponse,
 			transport: this.transport,
 			thinkingBudgets: this.thinkingBudgets,
-			maxRetryDelayMs: this.maxRetryDelayMs,
 			toolExecution: this.toolExecution,
 			beforeToolCall: this.beforeToolCall,
 			afterToolCall: this.afterToolCall,

packages/agent/src/proxy.ts

@@ -57,7 +57,6 @@ type ProxySerializableStreamOptions = Pick<
 	| "metadata"
 	| "transport"
 	| "thinkingBudgets"
-	| "maxRetryDelayMs"
 >;
 
 export interface ProxyStreamOptions extends ProxySerializableStreamOptions {
@@ -96,7 +95,6 @@ function buildProxyRequestOptions(options: ProxyStreamOptions): ProxySerializabl
 		metadata: options.metadata,
 		transport: options.transport,
 		thinkingBudgets: options.thinkingBudgets,
-		maxRetryDelayMs: options.maxRetryDelayMs,
 	};
 }
 

packages/ai/.changes/24-kind-aware-retry.md

@@ -0,0 +1,2 @@
+- Added upstream `Retry-After` capture to provider stream-failure diagnostics, including Anthropic overloads delivered as mid-stream SSE errors, so higher-level retry can honor it ([#24](https://github.com/pylon-code/prime-agent/issues/24)).
+- Removed the unused `maxRetryDelayMs` stream option; no provider ever read it ([#24](https://github.com/pylon-code/prime-agent/issues/24)).

packages/ai/src/providers/anthropic.ts

@@ -36,6 +36,7 @@ import {
 	classifyStreamFailure,
 	formatStreamFailureMessage,
 	recordStreamFailure,
+	retryAfterMsFromHeaders,
 	StreamFailureError,
 	streamFailureFromStopReason,
 	streamFailureMessage,
@@ -384,22 +385,33 @@ async function* iterateSseMessages(
 }
 
 /** Turn an in-stream `error` SSE event (how Anthropic delivers overloads etc.) into a classified failure. */
-function anthropicSseError(data: string, requestId?: string): StreamFailureError {
+function anthropicSseError(data: string, requestId?: string, retryAfterMs?: number): StreamFailureError {
 	let errorType: string | undefined;
 	let detail: string | undefined;
 	try {
-		const parsed = parseJsonWithRepair<{ error?: { type?: string; message?: string }; request_id?: string }>(data);
+		const parsed = parseJsonWithRepair<{
+			error?: { type?: string; message?: string; retry_after?: number };
+			request_id?: string;
+		}>(data);
 		errorType = parsed.error?.type;
 		detail = parsed.error?.message;
 		// Proxies may strip the request-id header; the error body carries it too.
 		requestId ??= typeof parsed.request_id === "string" ? parsed.request_id : undefined;
+		// A stream's headers are sent before the error is known, so proxies that
+		// compute a wait mid-stream (e.g. Meridian) carry it in the frame as
+		// seconds. The in-frame value describes this exact failure; prefer it.
+		const frameRetryAfter = parsed.error?.retry_after;
+		if (typeof frameRetryAfter === "number" && Number.isFinite(frameRetryAfter) && frameRetryAfter >= 0) {
+			retryAfterMs = Math.round(frameRetryAfter * 1000);
+		}
 	} catch {
 		detail = data;
 	}
 	const info = {
 		kind: classifyStreamFailure(errorType),
 		providerErrorType: errorType,
 		requestId,
+		retryAfterMs,
 		raw: truncateRawPayload(data),
 	};
 	return new StreamFailureError(streamFailureMessage(info, detail), info);
@@ -416,10 +428,13 @@ async function* iterateAnthropicEvents(
 
 	let sawMessageStart = false;
 	let sawMessageEnd = false;
+	// Overload/rate-limit SSE errors arrive on an otherwise-200 response, so the
+	// only Retry-After the caller ever sees is the one on the response headers.
+	const retryAfterMs = retryAfterMsFromHeaders(response.headers);
 
 	for await (const sse of iterateSseMessages(response.body, signal)) {
 		if (sse.event === "error") {
-			throw anthropicSseError(sse.data, requestId);
+			throw anthropicSseError(sse.data, requestId, retryAfterMs);
 		}
 
 		if (!ANTHROPIC_MESSAGE_EVENTS.has(sse.event ?? "")) {

packages/ai/src/providers/simple-options.ts

@@ -15,7 +15,6 @@ export function buildBaseOptions(model: Model<Api>, options?: SimpleStreamOption
 		onResponse: options?.onResponse,
 		timeoutMs: options?.timeoutMs,
 		maxRetries: options?.maxRetries,
-		maxRetryDelayMs: options?.maxRetryDelayMs,
 		metadata: options?.metadata,
 	};
 }

packages/ai/src/types.ts

@@ -123,14 +123,6 @@ export interface StreamOptions {
 	 * For example, OpenAI and Anthropic SDK clients default to 2.
 	 */
 	maxRetries?: number;
-	/**
-	 * Maximum delay in milliseconds to wait for a retry when the server requests a long wait.
-	 * If the server's requested delay exceeds this value, the request fails immediately
-	 * with an error containing the requested delay, allowing higher-level retry logic
-	 * to handle it with user visibility.
-	 * Default: 60000 (60 seconds). Set to 0 to disable the cap.
-	 */
-	maxRetryDelayMs?: number;
 	/**
 	 * Optional metadata to include in API requests.
 	 * Providers extract the fields they understand and ignore the rest.

packages/ai/src/utils/stream-failure.ts

@@ -25,6 +25,8 @@ export interface StreamFailureInfo {
 	providerErrorType?: string;
 	status?: number;
 	requestId?: string;
+	/** Upstream `Retry-After` translated to milliseconds, when the provider sent one. */
+	retryAfterMs?: number;
 	/** Truncated raw provider payload for post-mortems. */
 	raw?: string;
 }
@@ -114,6 +116,39 @@ export function truncateRawPayload(raw: string): string {
 	return raw.length > MAX_RAW_LENGTH ? `${raw.slice(0, MAX_RAW_LENGTH)}…` : raw;
 }
 
+/**
+ * Translate an HTTP `Retry-After` value (delta-seconds or HTTP-date) into a
+ * non-negative millisecond wait. Returns undefined when the header is absent or
+ * unparseable so callers fall back to their own backoff.
+ */
+export function parseRetryAfterMs(value: string | null | undefined, now = Date.now()): number | undefined {
+	const raw = value?.trim();
+	if (!raw) return undefined;
+	const seconds = Number(raw);
+	if (Number.isFinite(seconds)) return seconds > 0 ? Math.round(seconds * 1000) : 0;
+	const retryAt = Date.parse(raw);
+	return Number.isFinite(retryAt) ? Math.max(0, retryAt - now) : undefined;
+}
+
+type HeaderSource = Headers | Record<string, unknown>;
+
+function readHeader(headers: unknown, name: string): string | undefined {
+	if (!headers || typeof headers !== "object") return undefined;
+	const source = headers as HeaderSource;
+	if (typeof (source as Headers).get === "function") {
+		return (source as Headers).get(name) ?? undefined;
+	}
+	const record = source as Record<string, unknown>;
+	const key = Object.keys(record).find((candidate) => candidate.toLowerCase() === name);
+	const value = key === undefined ? undefined : record[key];
+	return typeof value === "string" ? value : undefined;
+}
+
+/** Read `Retry-After` off any header carrier a provider SDK error might expose. */
+export function retryAfterMsFromHeaders(headers: unknown, now = Date.now()): number | undefined {
+	return parseRetryAfterMs(readHeader(headers, "retry-after"), now);
+}
+
 function extractStreamFailureParts(error: unknown): { info: StreamFailureInfo; detail?: string } {
 	if (error instanceof StreamFailureError) return { info: error.info };
 	if (!(error instanceof Error)) return { info: { kind: "unknown" } };
@@ -150,13 +185,7 @@ function extractStreamFailureParts(error: unknown): { info: StreamFailureInfo; d
 					: undefined;
 
 	const headers = err.headers;
-	const headerRequestId =
-		headers && typeof (headers as Headers).get === "function"
-			? ((headers as Headers).get("request-id") ?? (headers as Headers).get("x-request-id"))
-			: headers && typeof headers === "object"
-				? ((headers as Record<string, unknown>)["request-id"] ??
-					(headers as Record<string, unknown>)["x-request-id"])
-				: undefined;
+	const headerRequestId = readHeader(headers, "request-id") ?? readHeader(headers, "x-request-id");
 	const rawRequestId = err.requestID ?? err.request_id ?? err.$metadata?.requestId ?? headerRequestId;
 	const requestId = typeof rawRequestId === "string" ? rawRequestId : undefined;
 
@@ -166,6 +195,7 @@ function extractStreamFailureParts(error: unknown): { info: StreamFailureInfo; d
 			providerErrorType,
 			status,
 			requestId,
+			retryAfterMs: retryAfterMsFromHeaders(headers),
 		},
 		detail: typeof bodyMessage === "string" ? bodyMessage : undefined,
 	};

packages/ai/test/anthropic-sse-parsing.test.ts

@@ -153,6 +153,51 @@ describe("Anthropic raw SSE parsing", () => {
 		expect(result.usage.cost.cacheWrite).toBeCloseTo(testCase.expectedCacheWriteCost);
 	});
 
+	it("carries Retry-After from an overload delivered as a mid-stream SSE error", async () => {
+		const model = getModel("anthropic", "claude-haiku-4-5");
+		const response = new Response(
+			`event: error\ndata: ${JSON.stringify({ type: "error", error: { type: "overloaded_error", message: "Overloaded" } })}\n`,
+			{ status: 200, headers: { "content-type": "text/event-stream", "retry-after": "30" } },
+		);
+
+		const result = await streamAnthropic(
+			model,
+			{ messages: [{ role: "user", content: "Say hello.", timestamp: Date.now() }] },
+			{ client: createFakeAnthropicClient(response) },
+		).result();
+
+		expect(result.stopReason).toBe("error");
+		expect(result.diagnostics?.[0]).toMatchObject({
+			type: "provider_stream_failure",
+			details: { kind: "overloaded", retryAfterMs: 30_000 },
+		});
+	});
+
+	it("prefers an in-frame retry_after over the response header for a mid-stream SSE error", async () => {
+		const model = getModel("anthropic", "claude-haiku-4-5");
+		// A stream's headers go out before the failure is known, so proxies
+		// (e.g. Meridian) put the wait in the error frame itself, in seconds.
+		const response = new Response(
+			`event: error\ndata: ${JSON.stringify({
+				type: "error",
+				error: { type: "rate_limit_error", message: "Rate limited", retry_after: 45 },
+			})}\n`,
+			{ status: 200, headers: { "content-type": "text/event-stream", "retry-after": "30" } },
+		);
+
+		const result = await streamAnthropic(
+			model,
+			{ messages: [{ role: "user", content: "Say hello.", timestamp: Date.now() }] },
+			{ client: createFakeAnthropicClient(response) },
+		).result();
+
+		expect(result.stopReason).toBe("error");
+		expect(result.diagnostics?.[0]).toMatchObject({
+			type: "provider_stream_failure",
+			details: { kind: "rate_limit", retryAfterMs: 45_000 },
+		});
+	});
+
 	it("preserves configured cache write pricing for non-Anthropic models", async () => {
 		const model = getModel("minimax", "MiniMax-M2.7-highspeed");
 		const response = createSseResponse(

packages/ai/test/stream-failure.test.ts

@@ -5,7 +5,9 @@ import {
 	classifyStreamFailure,
 	extractStreamFailureInfo,
 	formatStreamFailureMessage,
+	parseRetryAfterMs,
 	recordStreamFailure,
+	retryAfterMsFromHeaders,
 	StreamFailureError,
 	streamFailureFromStopReason,
 } from "../src/utils/stream-failure.js";
@@ -102,12 +104,64 @@ describe("extractStreamFailureInfo", () => {
 		expect(extractStreamFailureInfo(awsError)).toMatchObject({ requestId: "aws_req" });
 	});
 
+	test("carries Retry-After from rate-limit response headers", () => {
+		const sdkError = Object.assign(new Error("429 rate limited"), {
+			status: 429,
+			headers: new Headers({ "retry-after": "30", "request-id": "req_429" }),
+		});
+		expect(extractStreamFailureInfo(sdkError)).toMatchObject({
+			kind: "rate_limit",
+			status: 429,
+			requestId: "req_429",
+			retryAfterMs: 30_000,
+		});
+	});
+
+	test("leaves retryAfterMs unset when the provider sent no Retry-After", () => {
+		const sdkError = Object.assign(new Error("529 overloaded"), { status: 529 });
+		expect(extractStreamFailureInfo(sdkError).retryAfterMs).toBeUndefined();
+	});
+
 	test("falls back to classifying the message text", () => {
 		expect(extractStreamFailureInfo(new Error("provider overloaded, retry later")).kind).toBe("overloaded");
 		expect(extractStreamFailureInfo("not an error").kind).toBe("unknown");
 	});
 });
 
+describe("parseRetryAfterMs", () => {
+	test("reads delta-seconds", () => {
+		expect(parseRetryAfterMs("30")).toBe(30_000);
+		expect(parseRetryAfterMs(" 1.5 ")).toBe(1500);
+	});
+
+	test("clamps non-positive delta-seconds to zero", () => {
+		expect(parseRetryAfterMs("0")).toBe(0);
+		expect(parseRetryAfterMs("-5")).toBe(0);
+	});
+
+	test("reads an HTTP-date relative to now", () => {
+		const now = Date.parse("2026-01-01T00:00:00Z");
+		expect(parseRetryAfterMs("Thu, 01 Jan 2026 00:00:45 GMT", now)).toBe(45_000);
+		expect(parseRetryAfterMs("Wed, 31 Dec 2025 23:59:00 GMT", now)).toBe(0);
+	});
+
+	test("returns undefined for missing or unparseable values", () => {
+		expect(parseRetryAfterMs(undefined)).toBeUndefined();
+		expect(parseRetryAfterMs(null)).toBeUndefined();
+		expect(parseRetryAfterMs("   ")).toBeUndefined();
+		expect(parseRetryAfterMs("soon")).toBeUndefined();
+	});
+});
+
+describe("retryAfterMsFromHeaders", () => {
+	test("reads Headers objects and plain records", () => {
+		expect(retryAfterMsFromHeaders(new Headers({ "retry-after": "12" }))).toBe(12_000);
+		expect(retryAfterMsFromHeaders({ "Retry-After": "12" })).toBe(12_000);
+		expect(retryAfterMsFromHeaders({})).toBeUndefined();
+		expect(retryAfterMsFromHeaders(undefined)).toBeUndefined();
+	});
+});
+
 describe("formatStreamFailureMessage", () => {
 	test("condenses a classified SDK error to a one-liner instead of the raw payload", () => {
 		const sdkError = Object.assign(
@@ -160,6 +214,21 @@ describe("recordStreamFailure", () => {
 		});
 	});
 
+	test("persists Retry-After in the diagnostic so session retry can honor it", () => {
+		setLogSink(() => {});
+		const output = makeOutput({ errorMessage: "Provider rate limit exceeded" });
+		recordStreamFailure(
+			model,
+			output,
+			new StreamFailureError("x", { kind: "rate_limit", status: 429, retryAfterMs: 30_000 }),
+		);
+
+		expect(output.diagnostics?.[0]).toMatchObject({
+			type: "provider_stream_failure",
+			details: { kind: "rate_limit", retryAfterMs: 30_000 },
+		});
+	});
+
 	test("does nothing for user aborts", () => {
 		const logged: unknown[] = [];
 		setLogSink((entry) => logged.push(entry));