RAG Deep Agent

A local-first retrieval-augmented generation (RAG) API for indexing web documents and answering questions from their contents. It combines local sentence-transformers embeddings and a persistent Chroma vector store with an OpenAI-compatible chat model. For complex questions, it can use LangChain Deep Agents (deepagents) to search, delegate chunk analysis, and synthesize an answer.

How it works

1. Fetches HTTP(S) URLs, extracts text from HTML, and splits content into overlapping chunks.
2. Embeds and stores the chunks locally in Chroma, retaining the source URL as metadata.
3. Retrieves the closest chunks for a question and returns a grounded answer with citations—or runs a multi-step deep-agent workflow and returns its trace.

Embeddings are computed locally. Chat completions are sent to the configured OpenAI-compatible endpoint. The embedding model is downloaded on first use.

Requirements

• Python 3.13 or newer
• uv
• An OpenAI-compatible chat endpoint and API key to answer questions. The API can start without a key, but chat and agent requests need one.

Get started

git clone <repository-url>
cd rag-deep-agent
uv sync --dev
cp .env.example .env

Edit .env with the chat endpoint, model, and key for your provider. The checked-in example is set up for an OpenAI-compatible endpoint at http://localhost:11434/v1; change it if you use another provider.

Start the development server:

uv run fastapi dev app/main.py

Open http://127.0.0.1:8000/docs for the interactive API reference. The first embedding operation downloads sentence-transformers/all-mpnet-base-v2 into model-home/; indexed data is persisted in chroma_langchain_db/.

Warning

This API has no authentication and allows cross-origin requests. Run it only in a trusted environment; do not expose it to an untrusted network without adding appropriate access controls.

Configuration

Settings are read from environment variables or the project-root .env file. Environment variables take precedence. Relative storage paths resolve from the project root.

Variable	Default	Description
OPENAI_API_KEY	Empty	Chat API key; required for chat and agent requests. The legacy variable name api_key is also accepted.
CHAT_BASE_URL	https://llm.wi8.ir/v1	OpenAI-compatible chat endpoint.
CHAT_MODEL	auto	Model identifier sent to the endpoint.
CHAT_TEMPERATURE	0	Chat model temperature.
CHAT_TIMEOUT_SECONDS	120	Chat request timeout.
EMBEDDING_MODEL_NAME	sentence-transformers/all-mpnet-base-v2	Local embedding model.
EMBEDDING_CACHE_FOLDER	model-home/	Embedding model download/cache directory.
CHROMA_PERSIST_DIRECTORY	chroma_langchain_db/	Persistent vector-store directory.
CHROMA_COLLECTION_NAME	rag_documents	Chroma collection name.
FETCH_TIMEOUT_SECONDS	20	Timeout for each source request.
MAX_SOURCE_BYTES	5000000	Maximum response size per source.
CHUNK_SIZE / CHUNK_OVERLAP	1000 / 200	Text splitting size and overlap.
DEFAULT_TOP_K / MAX_TOP_K	4 / 25	Default and maximum number of retrieved chunks.
SNIPPET_LENGTH	400	Citation preview length.
MAX_CONCURRENT_ANALYSTS	3	Maximum parallel chunk-analyst tasks for the deep agent.
LOG_LEVEL	INFO	Application log level.

Configuration is validated at startup. Invalid values are reported by /health; the server still starts so the problem can be diagnosed. A missing chat key also leaves the API available, but requests that need the model return 503.

HTTP API

Method	Path	Description
GET	/	Service banner.
GET	/health	Readiness checks and configuration problems.
GET	/api/system/info	Effective non-secret configuration.
GET	/api/corpus/stats	Source/chunk counts and index settings.
POST	/api/documents	Fetch and index one or more URLs (201).
GET	/api/documents	List indexed sources and chunk counts.
GET	/api/documents/search	Semantic search without a chat completion.
POST	/api/documents/remove	Remove all chunks for one source.
DELETE	/api/documents?confirm=true	Clear the entire index; confirmation is required.
POST	/api/chat	Answer from retrieved chunks with citations.
POST	/api/agent/runs	Run the deep agent and return its steps.

Ingestion accepts 1–20 URLs per request, removes duplicates, and skips failed URLs when other sources can still be indexed. Each source must be HTTP(S) and is capped at MAX_SOURCE_BYTES.

Examples

Index URLs:

curl -X POST http://127.0.0.1:8000/api/documents \
  -H 'content-type: application/json' \
  -d '{"urls":["https://example.com/guide.html"]}'

List indexed sources:

curl http://127.0.0.1:8000/api/documents

Search without calling the chat model. Lower scores indicate closer matches:

curl 'http://127.0.0.1:8000/api/documents/search?q=streaming&top_k=3'

Ask a grounded question:

curl -X POST http://127.0.0.1:8000/api/chat \
  -H 'content-type: application/json' \
  -d '{"question":"How does this guide configure the service?","top_k":4}'

Run the deep agent:

curl -X POST http://127.0.0.1:8000/api/agent/runs \
  -H 'content-type: application/json' \
  -d '{"question":"Summarize the guide and compare its setup options."}'

Remove one source, or clear the full index (the confirmation query is required):

curl -X POST http://127.0.0.1:8000/api/documents/remove \
  -H 'content-type: application/json' \
  -d '{"source":"https://example.com/guide.html"}'

curl -X DELETE 'http://127.0.0.1:8000/api/documents?confirm=true'

top_k defaults to 4 and is limited to 25. The chat and search endpoints also accept max_distance to exclude weak matches. The agent uses the configured default retrieval count.

Responses and errors

Chat responses contain answer, unique sources, and citations with a source URL, similarity distance, and snippet. Agent responses contain the final answer and ordered steps. Search responses include ranked chunks and their scores.

Errors use a stable JSON shape:

{"detail":"No indexed documents found. Ingest documents first.","code":"empty_corpus"}

Code	HTTP status	Meaning
not_configured	503	Invalid configuration or missing chat API key.
source_fetch_failed	502	A source could not be fetched.
invalid_source	422	Invalid URL or unsupported scheme.
unsupported_content	415	Source content is unsupported or too large.
empty_document	422	No indexable text was produced.
empty_corpus	409	No documents are indexed.
unknown_source	404	The requested source is not indexed.
confirmation_required	400	Whole-index deletion needs ?confirm=true.
validation_error	422	Invalid request; includes an errors array.
internal_error	500	An unclassified domain error.

POST /api/ingest and POST /api/query remain available as deprecated aliases for document ingestion and chat.

Command-line interface

The rag-demo command uses the same local index:

uv run rag-demo stats
uv run rag-demo ask "How do I stream?"
uv run rag-demo agent "Summarize the indexed guide."

Index documents through the API before running ask or agent.

Development

Run the test suite:

uv run pytest

Run Ruff:

uvx ruff check app tests

Tests mock service boundaries and do not require downloading the embedding model or contacting a chat provider.

Project structure

app/
├── agents/      Deep-agent setup, prompts, and retrieval tool
├── api/         FastAPI routers, request helpers, and error mapping
├── core/        Settings and domain errors
├── services/    Ingestion, vector storage, retrieval, and answering
├── cli.py       rag-demo command
├── main.py      FastAPI application
└── schemas.py   HTTP request and response models
tests/           Unit and API tests