An advanced, production-grade Self-Corrective Retrieval-Augmented Generation (CRAG) application built with LangGraph, LangChain, and Ollama. CorrectiveRAG combines multi-agent validation, dynamic document relevance grading, hallucination detection, and real-time Google web search fallback to deliver high-precision answers from PDFs and web documents.
- Overview
- System Architecture
- How Self-Correction Works
- Benchmark & Evaluation
- Getting Started
- Docker & Containerized Deployment
- AWS EC2 Deployment Guide
- Project Structure
- Configuration Reference
- Roadmap
- Author & License
Standard RAG systems suffer from three primary failure modes:
- Irrelevant Context Retrieval: Semantic similarity does not guarantee relevance to the query.
- LLM Hallucinations: Generation ungrounded in retrieved context.
- Out-of-Scope Queries: Questions where indexed documents do not contain the answer.
CorrectiveRAG solves these challenges through an autonomous multi-agent state graph:
- Evaluates retrieved documents and filters out non-relevant chunks before generation.
- Dynamically falls back to live web search when internal documents lack necessary information.
- Verifies that generated answers are grounded in context facts and directly answer the question.
- Self-corrects and iterates if hallucinations are detected.
- User Input / Ingestion: User uploads PDF documents or inputs web URLs. Documents are chunked and indexed into ChromaDB using Ollama vector embeddings.
- Conditional Query Routing:
summary: Triggers a Map-Reduce summary chain across the entire ingested document set.rag: Routes to the corrective question-answering workflow.
- Vector Store Retrieval: Retrieves top-$k$ nearest document chunks from Chroma.
- Document Relevance Grader: Evaluates each retrieved document chunk against the query with binary scoring (
yes/no). - Dynamic Search Fallback: If retrieved chunks are deemed irrelevant, the agent automatically executes a Google Custom Search to obtain live external context.
- RAG Answer Generation: Synthesizes an answer strictly based on graded context.
- Hallucination Detection: Verifies if generated claims are fully supported by context facts.
- Answer Relevancy Grader: Checks if the answer resolves the user's intent. If ungrounded, the system initiates a retry or triggers web search.
Evaluation conducted using the RAGAS framework and the ragas-wikiqa dataset comparing CorrectiveRAG against a standard baseline RAG chain (retriever | prompt | llm | parser):
| Evaluation Metric | CorrectiveRAG | Baseline Naive RAG | Improvement |
|---|---|---|---|
| Faithfulness (Factuality) | 0.9139 | 0.8730 | +4.68% |
| Answer Relevancy | 0.7952 | 0.6624 | +20.05% |
| Average Response Time | ~26.2 s | ~9.6 s | (Trade-off for multi-step verification) |
Key Takeaway: CorrectiveRAG provides a significant gain in answer faithfulness and relevancy, making it ideal for high-stakes enterprise applications (finance, legal, compliance, medical) where accuracy is paramount.
- Python: 3.10, 3.11, or 3.12
- Ollama: Download and install Ollama
- Pull your chosen LLM and embedding model (default is
llama3):ollama pull llama3
Clone the repository:
git clone https://github.com/shrridharshan27/CorrectiveRAG.git
cd CorrectiveRAGCreate and activate a virtual environment:
# Windows
python -m venv venv
.\venv\Scripts\activate
# Linux / macOS
python3 -m venv venv
source venv/bin/activateInstall dependencies:
pip install -r requirements.txtCopy .env.example to .env:
cp .env.example .envFill in your API keys (optional for local document RAG; required for Google search fallback):
OLLAMA_URL=http://localhost:11434
MODEL_NAME=llama3
EMBEDDING_MODEL=llama3
# Optional: For Web Search Fallback
GOOGLE_CSE_ID=your_custom_search_engine_id
GOOGLE_API_KEY=your_google_cloud_api_key
# Optional: For Running RAGAS Evaluation
OPENAI_API_KEY=your_openai_api_keyLaunch the modern Streamlit interface:
streamlit run app.pyOpen your browser at http://localhost:8501.
Run the complete stack (Streamlit App + ChromaDB) using Docker Compose:
docker compose up --build -dThe app will be accessible at:
- Streamlit Frontend:
http://localhost:8501 - ChromaDB API:
http://localhost:8000
To tear down:
docker compose downFor production or cloud GPU hosting:
- GPU Instance Setup:
- Launch an AWS EC2 instance (e.g.,
g4dn.xlargewith NVIDIA T4 GPU and 50+ GB SSD). - Inbound security group rules: Open port
11434(Ollama) and8501(Streamlit).
- Launch an AWS EC2 instance (e.g.,
- Install Ollama & GPU Drivers:
curl -fsSL https://ollama.com/install.sh | sh ollama pull llama3 - Run Containerized Application:
- Point
OLLAMA_URLin your.envorapplication.yamlto the EC2 host IP or private DNS. - Run
docker compose up -d.
- Point
CorrectiveRAG/
βββ .env.example # Configuration template
βββ .gitignore # Git ignore rules
βββ Dockerfile # Production multi-stage Docker build
βββ LICENSE # MIT License
βββ README.md # Project documentation
βββ agent.py # LangChain agent declarations & tool definitions
βββ app.py # Streamlit frontend with interactive chat & tracing
βββ application.yaml # Model & environment settings
βββ arch.png # Architecture & state machine diagram
βββ demo.gif # UI walkthrough animation
βββ docker-compose.yml # Docker Compose configuration (App + ChromaDB)
βββ document_loader.py # Multi-format document loader (PDFs & Web URLs)
βββ embeddings.py # Custom Ollama embeddings integration
βββ evaluate.py # RAGAS benchmarking & metric evaluation script
βββ prompts.yaml # Prompt templates for grading and generation
βββ rag_app.py # Core application orchestrator
βββ requirements.txt # Python dependencies
βββ vectorstore_retriever.py # Chroma vector store manager
βββ workflow.py # LangGraph state machine & conditional edges
βββ yaml_reader.py # Environment-aware YAML configuration parser
| Parameter | Default | Description |
|---|---|---|
model |
llama3 |
Main reasoning LLM served via Ollama |
embedding_model |
llama3 |
Vector embedding model |
collection_name |
rag-chroma |
Target collection name in ChromaDB |
ollama_url |
http://localhost:11434 |
Ollama service endpoint |
GOOGLE_CSE_ID |
"" |
Google Programmable Search Engine ID |
GOOGLE_API_KEY |
"" |
Google Cloud API key for Web Search |
- LangGraph Multi-Agent Self-Correction Graph
- PDF & Web URL ingestion support
- Google Search API fallback
- Streamlit Chat Interface with real-time reasoning trace
- Docker & Docker Compose setup
- Multi-modal support (charts, images, audio transcription)
- Semantic query caching with Redis
- Streaming tokens directly into chat bubbles
Developed and maintained by shrridharshan27.
Distributed under the MIT License. See LICENSE for more details.

