Documentation · Package · npm

The LLM call framework. Resilience, observability, and control for every call.

One interface across OpenAI-compatible, Anthropic, Gemini, and Bedrock, with retry budgets, circuit breaking, multi provider fallback, rate limiting, caching, and middleware built in, all running in your own process rather than a new network hop.

import Anthropic from '@anthropic-ai/sdk';
import OpenAI from 'openai';
import { VernLLM } from 'vern-llm';
import { fromAnthropic, fromOpenAI } from 'vern-llm/adapters';

const openai = fromOpenAI(new OpenAI({ apiKey: process.env.OPENAI_API_KEY }));
const anthropic = fromAnthropic(new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY }));

const llm = new VernLLM({
  client: openai,
  model: 'gpt-6-sol',
  fallback: { client: anthropic, model: 'claude-opus-5-5', circuitBreaker: true },
  rateLimit: { requestsPerMinute: 500, tokensPerMinute: 100_000, maxConcurrent: 20 },
  retryBudget: { windowMs: 60_000, minCalls: 20, retryRatio: 0.2 },
  maxRetries: 3,
  timeoutMs: 10_000,
  defaultMaxTokens: 1000,
  defaultReasoningEffort: 'medium',
});

const result = await llm.call({ userContent: "What's the weather in New York?" });

Works with OpenAI, Groq, Mistral, DeepSeek, Cerebras, Together AI, Fireworks AI, Ollama, Anthropic, Gemini, AWS Bedrock (via vern-llm-bedrock), or any provider reachable over HTTP via a fromFetch adapter.

Repository layout

This is a pnpm monorepo with five workspaces, managed with Nx for task caching and affected-based CI:

Path	Description
packages/vern-llm	The vern-llm npm package: source, tests, and its own README with the full API reference.
packages/vern-llm-redis	The vern-llm-redis npm package: Redis backed circuit breaker, rate limiter, and cache adapters for vern-llm.
packages/vern-llm-otel	The vern-llm-otel npm package: OpenTelemetry traces and metrics for vern-llm.
packages/vern-llm-bedrock	The vern-llm-bedrock npm package: the AWS Bedrock Converse adapter for vern-llm, built on the AWS SDK client.
apps/docs	The Fumadocs-powered documentation site.

License

MIT © LakBud