@ingram-tech/anthropic-cache

Automatic prompt caching for Anthropic models through the Vercel AI SDK.

Anthropic's prefix cache is opt-in per request: without cache_control breakpoints, every step of an agent loop pays the full input rate on a prefix that has not changed since the previous step. This middleware places the breakpoints on every request, so no call site needs them. It works on the direct API and on Amazon Bedrock, which has no automatic caching.

npm install @ingram-tech/anthropic-cache

import { anthropic } from "@ai-sdk/anthropic";
import { wrapLanguageModel } from "ai";
import { anthropicCache } from "@ingram-tech/anthropic-cache";

const model = wrapLanguageModel({
	model: anthropic("claude-sonnet-5"),
	middleware: anthropicCache(),
});

generateText, streamText and agent loops built on them get cached prefixes from the second step on.

Where the breakpoints go

Three of the four breakpoints Anthropic allows:

• The last system message. Tools are rendered before the system prompt, so this one point caches the tool definitions and the system prompt together.
• The last two non-system messages. The trailing point caches the whole conversation up to this step. The one before it keeps the previous step's entry inside the provider's 20-block lookback even when a single step appends many tool-result blocks.

Existing providerOptions on a message are kept; cacheControl is merged in.

Lifetime

anthropicCache({ ttl: "1h" });

The default 5-minute cache is refreshed by every step of a turn and read by follow-up turns inside the window. The 1-hour cache costs more to write and suits long idle gaps between turns.

Caveats

Caching only pays when the prefix is byte-stable between requests. A system prompt that embeds the current time, or tools whose order changes, defeats it. Cache writes are billed at a premium over plain input, so a one-shot request with no follow-up pays the write premium and reads nothing back.

License

MIT