perf(coding-agent): keep the prompt-cache prefix stable across harness updates, date flips, and tool refreshes

.pylon/features.yaml

@@ -316,3 +316,22 @@ decisions:
     revisit_when:
       - Prime makes a user-facing turn abort cancel the child runs it owns, or exposes an equivalent turn-scoped cancellation contract.
       - Prime introduces a real detached or background spawn mode, which would need a per-child survival decision instead of one retained-versus-active rule.
+
+  stable-prompt-cache-prefix:
+    area: prompt-ownership
+    state: shipped
+    owner: shared
+    decision: retain
+    pylon_refs:
+      - https://github.com/pylon-code/prime-agent/issues/22
+      - https://github.com/pylon-code/prime-agent/issues/26
+      - https://github.com/pylon-code/prime-agent/pull/36
+    upstream_refs:
+      - https://github.com/PrimeIntellect-ai/prime-agent/commit/4b9e6006fad553dcca5aa69b23a039088064589f
+      - https://github.com/PrimeIntellect-ai/prime-agent/commit/f81acc6679ab214f1c40821588d9e5e97c47bdb1
+      - https://github.com/PrimeIntellect-ai/prime-agent/tree/c382f09856d4a8c8d2b765179657047d58691f25
+    fork_change: stable-prompt-cache-prefix-v1
+    upstream_support: Prime through c382f09856d4 reduced the system-prompt date to a day stamp for cache stability, but it still serializes mutable continual-harness state and that date into the cached system prompt, has no provider-neutral volatile-content channel, no per-model placement contract for session-cached proxy backends, and does not pin active tool order behind the `cache_control` marker on the last tool definition.
+    revisit_when:
+      - Prime upstream provides an equivalent provider-neutral channel that keeps mutable state and clock-derived content out of the cached tools/system/history prefix, including a declared constraint for append-only-history backends.
+      - Pylon can drop the fork behavior without regressing measured cache-hit rates through a Claude-Max proxy or Meridian lineage continuation rates.

.pylon/upstream-review.md

@@ -148,3 +148,17 @@ This ledger records Prime upstream evidence and the decision taken for each over
 - Validation: new faux-provider regression `packages/coding-agent/test/suite/regressions/25-request-abort-rlm-cascade.test.ts` (2 tests) proves the child's provider stream is cut, that no further child request reaches the provider, that the parent settles and its next turn runs, and pins retained-child survival; it fails on the pre-change implementation. The upstream characterization test was inverted to "cancels active rlm children when the parent turn is interrupted", and the ACP-close terminal-notice retention test now settles the child before the scheduler cut, since a live child no longer survives it. `test/agent-session-recursion.test.ts` passes 112/112 and a 17-file affected batch across abort, RLM, subagent, queue, prompt, compaction, ACP, and correlated-lifecycle suites passes 396/396. `npm run check` is clean.
 - Fork change: [pylon-code/prime-agent#35](https://github.com/pylon-code/prime-agent/pull/35).
 - Revisit when Prime makes a user-facing turn abort cancel the children it owns, or introduces a real detached/background spawn mode that needs a per-child survival decision.
+
+## 2026-08-31 — stable prompt-cache prefix
+
+- Upstream baseline reviewed: `PrimeIntellect-ai/prime-agent@c382f09856d4a8c8d2b765179657047d58691f25` (one commit past the audited `a903d4b6768f`; PR #1893 renders Mermaid diagrams and does not touch prompt assembly or provider payloads). Latest audited compatibility release remains stock `v0.8.1`. This is a product candidate and does not advance `reviewed_upstream_commit`.
+- Searched upstream history and current source for cache-prefix stability work. Prime already recognized the problem for the clock: `4b9e6006f` reduced `Current date and time` to an ISO day stamp and `f81acc667` replaced `toISOString` with local date parts, both to stop the system prompt changing more often than necessary. Nothing else exists: upstream `buildSystemPrompt` still serializes `harnessState` into the cached system prompt, upstream has no `Context.volatileContext` or equivalent provider-neutral channel, and `convertTools` still marks the last tool definition with `cache_control` without pinning active tool order.
+- `stable-prompt-cache-prefix`: **retain** a Pylon-owned split. Adopt upstream's day-granularity date decision as-is; it is already in the fork. Add the missing piece upstream lacks: the volatile content leaves the cached region entirely instead of merely changing less often.
+- Design: `Context.volatileContext` carries continual-harness state and the current date. The Anthropic provider appends it as the final content block after the `cache_control` breakpoint on the last history block, so the cached tools -> system -> history prefix is unchanged by a harness write or a date flip. The OpenAI-completions provider appends it after its Anthropic-style markers and after the automatically cached prefix of providers without markers. The API registry folds the content into the end of the message list for every other provider, so no provider can silently drop it. `AgentSession` pins each tool to the slot it first occupied, so a registry refresh or an extension reload serializes an unchanged tool set to identical bytes.
+- Behavior preservation: the harness-state section and the `Current date` line are byte-identical to what the system prompt previously inlined, including the empty-state rendering and the custom-prompt-only scope of the date. Only their position changed. The subagent guidance still precedes the harness-state roster in the assembled request.
+- Compatibility classification: **backward-compatible and additive**. `Context.volatileContext`, `AgentState.volatileContext`, `AgentLoopConfig.getVolatileContext`, and `ApiProvider.handlesVolatileContext` are all optional; a caller that sets none behaves exactly as before. No daemon command, event, protocol version, or schema revision changes.
+- Validation: 15 `packages/ai` payload tests assert a byte-identical cached prefix across volatile changes for both the Anthropic and OpenAI-completions payload builders, correct marker placement, the trailing-user-message case, blank-input handling, and the registry fallback. 5 `packages/coding-agent/test/suite/regressions/26-stable-cache-prefix.test.ts` cases assert an unchanged prefix across a mid-session harness memory write, an unchanged prefix across a mocked two-day clock advance, identical tool bytes across repeated registry refreshes, first-activation tool-order pinning across reordering and removal, and — for an `appendOnlyHistory` model registered through `registerProvider` — an append-only history with the volatile content in the system prompt. `test/suite` (79 files, 960), `test/system-prompt.test.ts` (25), `test/model-registry.test.ts`, `test/agent-session-recursion.test.ts` (112), `packages/agent` (60), and 13 focused `packages/ai` provider files (93 passed, 9 gated skips) are green. `npm run check` passes. `test/extensions-runner.test.ts` fails 21 cases identically on unmodified `origin/pylon`, so it is pre-existing and unrelated.
+- Review correction — two placements, declared per model. The first candidate always used trailing placement, which regressed the Meridian proxy path from "cold cache when harness state or the date changes" to a full fresh replay every turn. Meridian's lineage matcher (`meridian/src/proxy/session/lineage.ts`) resumes an SDK session only when the incoming history matches the stored history; its boundary branch at `cached.messageCount - 1` tolerates a last-message block append only when every stored block hash is preserved and every appended block is a new `tool_result` (`hasOnlyNewToolResults`, deliberately narrow per Meridian #689/#692). A payload-only volatile text block fails that on the turn it appears, and on the next turn the stored boundary message holds more blocks than the incoming replay because the block was never persisted, so `incomingBlockHashes.length > storedBlocks.length` fails too. Classification falls through to `diverged: modified-history` and forces a full-history replay on the exact deployment #22 targets.
+- The fix is a declared per-model constraint, not a workaround. `Model.appendOnlyHistory` marks a backend that caches by session rather than by request prefix and therefore requires the request message array to stay byte-identical to the persisted history. When set, the API registry routes `volatileContext` into the system prompt and adds no payload-only message block; the backend's own session cache absorbs the system-prompt change. Default stays trailing placement, which is correct for direct prefix-cached APIs. The flag is settable from extension `registerProvider` model entries and from `models.json` model definitions and `modelOverrides`, and it is documented for extension authors in `packages/coding-agent/docs/extensions.md`.
+- Deferred: the optional debug-level prefix-drift check from issue #26 is not implemented. The regression tests assert the invariant directly, and a cross-turn byte comparison would need new mutable state inside otherwise pure prompt assembly.
+- Revisit when Prime upstream offers an equivalent provider-neutral channel that keeps mutable state and clock-derived content out of the cached prefix, and Pylon can drop the fork behavior without regressing measured cache-hit rates through a Claude-Max proxy.

packages/agent/.changes/stable-prompt-cache-prefix.md

@@ -0,0 +1 @@
+- Added `AgentState.volatileContext`, resolved before each model request and forwarded to the provider outside the cached prompt prefix ([#26](https://github.com/pylon-code/prime-agent/issues/26)).

packages/agent/src/agent-loop.ts

@@ -487,6 +487,7 @@ async function streamAssistantResponse(
 
 		const llmContext: Context = {
 			systemPrompt: config.getSystemPrompt?.() ?? context.systemPrompt,
+			volatileContext: config.getVolatileContext?.() ?? context.volatileContext,
 			messages: llmMessages,
 			tools: context.tools,
 		};

packages/agent/src/agent.ts

@@ -72,6 +72,7 @@ function createMutableAgentState(
 
 	return {
 		systemPrompt: initialState?.systemPrompt ?? "",
+		volatileContext: initialState?.volatileContext,
 		model: initialState?.model ?? DEFAULT_MODEL,
 		thinkingLevel: initialState?.thinkingLevel ?? "off",
 		serviceTier: initialState?.serviceTier ?? "default",
@@ -451,6 +452,7 @@ export class Agent {
 	private createContextSnapshot(): AgentContext {
 		return {
 			systemPrompt: this._state.systemPrompt,
+			volatileContext: this._state.volatileContext,
 			messages: this._state.messages.slice(),
 			tools: this._state.tools.slice(),
 		};
@@ -475,6 +477,7 @@ export class Agent {
 			convertToLlm: this.convertToLlm,
 			transformContext: this.transformContext,
 			getSystemPrompt: () => this._state.systemPrompt,
+			getVolatileContext: () => this._state.volatileContext,
 			getApiKey: this.getApiKey,
 			getSteeringMessages: async () => {
 				if (skipInitialSteeringPoll) {

packages/agent/src/types.ts

@@ -171,6 +171,9 @@ export interface AgentLoopConfig extends SimpleStreamOptions {
 	/** Resolves the system prompt immediately before each LLM call. */
 	getSystemPrompt?: () => string;
 
+	/** Resolves the volatile, never-cached context immediately before each LLM call. */
+	getVolatileContext?: () => string | undefined;
+
 	/**
 	 * Resolves an API key dynamically for each LLM call.
 	 *
@@ -306,6 +309,12 @@ export type AgentMessage = Message | CustomAgentMessages[keyof CustomAgentMessag
 export interface AgentState {
 	/** System prompt sent with each model request. */
 	systemPrompt: string;
+	/**
+	 * Content the model needs but that must stay out of every cached prefix,
+	 * such as mutable harness state or the current date. Providers place it after
+	 * their final prompt-cache breakpoint.
+	 */
+	volatileContext?: string;
 	/** Model used for future turns. */
 	model: Model<any>;
 	/** Requested reasoning level for future turns. */
@@ -374,6 +383,8 @@ export interface AgentTool<TParameters extends TSchema = TSchema, TDetails = any
 export interface AgentContext {
 	/** System prompt included with the request. */
 	systemPrompt: string;
+	/** Volatile content kept out of the cached prefix. See {@link AgentState.volatileContext}. */
+	volatileContext?: string;
 	/** Transcript visible to the model. */
 	messages: AgentMessage[];
 	/** Tools available for this run. */

packages/ai/.changes/stable-prompt-cache-prefix.md

@@ -0,0 +1,2 @@
+- Added `Context.volatileContext` for content that must stay out of every cached prompt prefix; Anthropic and OpenAI-completions place it after their cache breakpoints and other providers receive it appended to the message list ([#26](https://github.com/pylon-code/prime-agent/issues/26)).
+- Added the `Model.appendOnlyHistory` flag for session-cached backends, which routes volatile content into the system prompt so the request message array stays byte-identical to the persisted history ([#26](https://github.com/pylon-code/prime-agent/issues/26)).

packages/ai/README.md

@@ -46,6 +46,7 @@ Unified LLM API with automatic model discovery, provider configuration, token an
   - [OpenAI Compatibility Settings](#openai-compatibility-settings)
   - [Type Safety](#type-safety)
 - [Cross-Provider Handoffs](#cross-provider-handoffs)
+- [Volatile Context](#volatile-context)
 - [Context Serialization](#context-serialization)
 - [Browser Usage](#browser-usage)
   - [Browser Compatibility Notes](#browser-compatibility-notes)
@@ -977,6 +978,35 @@ This enables flexible workflows where you can:
 - Use specialized models for specific tasks
 - Maintain conversation continuity across provider outages
 
+## Volatile Context
+
+Prompt caching only pays off when the request prefix is byte-stable. Content that changes for reasons unrelated to the conversation — mutable agent state, the current date — invalidates the cache for the whole prefix when it sits in the system prompt.
+
+Put that content in `Context.volatileContext` instead:
+
+```typescript
+const context: Context = {
+  systemPrompt: 'You are a helpful assistant.',
+  messages: [{ role: 'user', content: 'What changed?', timestamp: Date.now() }],
+  volatileContext: `Current date: ${new Date().toISOString().slice(0, 10)}`
+};
+```
+
+The model still sees it, but never inside a cached region:
+
+- Anthropic and OpenAI-completions append it after their `cache_control` breakpoints, so the cached tools, system prompt, and conversation history are unaffected when it changes.
+- Every other provider receives it appended to the end of `Context.messages`, which is after the automatically cached prefix. Set `handlesVolatileContext: true` when registering a custom provider that positions the content itself.
+
+### Session-cached backends
+
+Trailing placement assumes the backend caches by request prefix. A proxy in front of an agent SDK does not: it matches the incoming message array against the history it already holds and resumes that session, so a payload-only trailing block reads as a modified history and forces a full replay.
+
+Set `appendOnlyHistory: true` on those models. The volatile content then goes into the system prompt and the message array stays byte-identical to the caller's history:
+
+```typescript
+const model = { ...getModel('anthropic', 'claude-opus-4-6'), baseUrl: 'https://my-sdk-proxy.example.com', appendOnlyHistory: true };
+```
+
 ## Context Serialization
 
 The `Context` object can be easily serialized and deserialized using standard JSON methods, making it simple to persist conversations, implement chat history, or transfer contexts between services:

packages/ai/src/api-registry.ts

@@ -7,6 +7,7 @@ import type {
 	StreamFunction,
 	StreamOptions,
 } from "./types.js";
+import { foldVolatileContext, foldVolatileContextIntoSystemPrompt } from "./utils/volatile-context.js";
 
 export type ApiStreamFunction = (
 	model: Model<Api>,
@@ -24,6 +25,12 @@ export interface ApiProvider<TApi extends Api = Api, TOptions extends StreamOpti
 	api: TApi;
 	stream: StreamFunction<TApi, TOptions>;
 	streamSimple: StreamFunction<TApi, SimpleStreamOptions>;
+	/**
+	 * Set when the provider positions `Context.volatileContext` itself, after its
+	 * final prompt-cache breakpoint. Providers that leave this unset receive the
+	 * volatile content folded into the end of `Context.messages`.
+	 */
+	handlesVolatileContext?: boolean;
 }
 
 interface ApiProviderInternal {
@@ -39,39 +46,58 @@ type RegisteredApiProvider = {
 
 const apiProviderRegistry = new Map<string, RegisteredApiProvider>();
 
+/**
+ * Decide where `Context.volatileContext` goes before the provider sees it.
+ *
+ * Prefix-cached backends want it after the last cache breakpoint, which the
+ * provider places. Session-cached `appendOnlyHistory` backends need the message
+ * array untouched, so it goes into the system prompt instead.
+ */
+function resolveVolatilePlacement(model: Model<Api>, context: Context, handlesVolatileContext: boolean): Context {
+	if (model.appendOnlyHistory) {
+		return foldVolatileContextIntoSystemPrompt(context);
+	}
+	return handlesVolatileContext ? context : foldVolatileContext(context);
+}
+
 function wrapStream<TApi extends Api, TOptions extends StreamOptions>(
 	api: TApi,
 	stream: StreamFunction<TApi, TOptions>,
+	handlesVolatileContext: boolean,
 ): ApiStreamFunction {
 	return (model, context, options) => {
 		if (model.api !== api) {
 			throw new Error(`Mismatched api: ${model.api} expected ${api}`);
 		}
-		return stream(model as Model<TApi>, context, options as TOptions);
+		const resolved = resolveVolatilePlacement(model, context, handlesVolatileContext);
+		return stream(model as Model<TApi>, resolved, options as TOptions);
 	};
 }
 
 function wrapStreamSimple<TApi extends Api>(
 	api: TApi,
 	streamSimple: StreamFunction<TApi, SimpleStreamOptions>,
+	handlesVolatileContext: boolean,
 ): ApiStreamSimpleFunction {
 	return (model, context, options) => {
 		if (model.api !== api) {
 			throw new Error(`Mismatched api: ${model.api} expected ${api}`);
 		}
-		return streamSimple(model as Model<TApi>, context, options);
+		const resolved = resolveVolatilePlacement(model, context, handlesVolatileContext);
+		return streamSimple(model as Model<TApi>, resolved, options);
 	};
 }
 
 export function registerApiProvider<TApi extends Api, TOptions extends StreamOptions>(
 	provider: ApiProvider<TApi, TOptions>,
 	sourceId?: string,
 ): void {
+	const handlesVolatileContext = provider.handlesVolatileContext === true;
 	apiProviderRegistry.set(provider.api, {
 		provider: {
 			api: provider.api,
-			stream: wrapStream(provider.api, provider.stream),
-			streamSimple: wrapStreamSimple(provider.api, provider.streamSimple),
+			stream: wrapStream(provider.api, provider.stream, handlesVolatileContext),
+			streamSimple: wrapStreamSimple(provider.api, provider.streamSimple, handlesVolatileContext),
 		},
 		sourceId,
 	});