semantic-cache

Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode. Works with any SDK or agent and adds under 200µs of overhead at 5k RPS.

A LLM semantic caching system aiming to enhance user experience by reducing response time via cached query-result pairs.

Redis Vector Library. The AI-native Python client for Redis.

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

SmarterRouter: An intelligent LLM gateway and VRAM-aware router for Ollama, llama.cpp, and OpenAI. Features semantic caching, model profiling, and automatic failover for local AI labs.

Drop-in AI memory layer with 2x faster retrieval and 10x lower cost. Fully compatible with Mem0 API. Migrate in 5 minutes without any code changes. Self-host for free.

企业级 AI Agent 平台 | Spring Boot 3 + LangChain4j | ReAct 推理 + 多路召回 RAG + 语义缓存 + 多智能体协作

Semantic caching layer for your LLM applications. Reuse responses and reduce token usage.

mimir is a drop-in proxy that caches LLM API responses using semantic similarity, reducing costs and latency for repeated or similar queries.

Reliable and Efficient Semantic Prompt Caching with vCache

Own your LLM's web search: a local search->fetch->rank pipeline that replaces hosted web-search tools. Measured: matches hosted accuracy at 66% lower cost and up to 88% fewer tokens, plus a precision-tuned semantic caching with query-dependant TTL that no API offers.

RAMen is a fast in-memory data store like Redis, but built for AI: drop-in Redis protocol, native vector search, semantic caching, and a built-in MCP server for agents. Single Go binary, BSD-3.

Transparent, transport-layer semantic cache for LLM API calls, powered by Redis 8 Vector Sets.

Unified multi-layer caching library for AI/agent pipelines — LangChain, LangGraph, AutoGen, CrewAI, Agno, A2A

Redis integration for Google Agent Development Kit (ADK) - Memory, Sessions, Search Tools, MCP

Enterprise AI traffic gateway — unified compliance, routing across 20+ LLM providers, semantic cache, quotas, and audit. SDK / network / OS-layer intercept.

Redis Vector Library (RedisVL) -- the AI-native Java client for Redis.

⚡ Adaptive multi-model LLM router — 80+ providers, Jev System One single-pass routing (model=jev-auto), pheromone-trail failover, parallel ensemble merge. npm: adaptive-memory-multi-model-router

This is a RAG based chatbot in which semantic cache and guardrails have been incorporated.

Semantic caching layer for LLM apps: serves cached answers for paraphrased prompts, with pluggable vector stores and streaming support.