Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

🧠 CorrectiveRAG: Multi-Step Agentic Self-Corrective RAG

Python Version LangChain LangGraph Ollama ChromaDB Streamlit License: MIT

An advanced, production-grade Self-Corrective Retrieval-Augmented Generation (CRAG) application built with LangGraph, LangChain, and Ollama. CorrectiveRAG combines multi-agent validation, dynamic document relevance grading, hallucination detection, and real-time Google web search fallback to deliver high-precision answers from PDFs and web documents.


πŸ“Έ Interactive Demo

CorrectiveRAG Demo


πŸ“‘ Table of Contents


🌟 Overview

Standard RAG systems suffer from three primary failure modes:

  1. Irrelevant Context Retrieval: Semantic similarity does not guarantee relevance to the query.
  2. LLM Hallucinations: Generation ungrounded in retrieved context.
  3. Out-of-Scope Queries: Questions where indexed documents do not contain the answer.

CorrectiveRAG solves these challenges through an autonomous multi-agent state graph:

  • Evaluates retrieved documents and filters out non-relevant chunks before generation.
  • Dynamically falls back to live web search when internal documents lack necessary information.
  • Verifies that generated answers are grounded in context facts and directly answer the question.
  • Self-corrects and iterates if hallucinations are detected.

πŸ—οΈ System Architecture

Architecture

Multi-Agent State Machine:

  1. User Input / Ingestion: User uploads PDF documents or inputs web URLs. Documents are chunked and indexed into ChromaDB using Ollama vector embeddings.
  2. Conditional Query Routing:
    • summary: Triggers a Map-Reduce summary chain across the entire ingested document set.
    • rag: Routes to the corrective question-answering workflow.
  3. Vector Store Retrieval: Retrieves top-$k$ nearest document chunks from Chroma.
  4. Document Relevance Grader: Evaluates each retrieved document chunk against the query with binary scoring (yes/no).
  5. Dynamic Search Fallback: If retrieved chunks are deemed irrelevant, the agent automatically executes a Google Custom Search to obtain live external context.
  6. RAG Answer Generation: Synthesizes an answer strictly based on graded context.
  7. Hallucination Detection: Verifies if generated claims are fully supported by context facts.
  8. Answer Relevancy Grader: Checks if the answer resolves the user's intent. If ungrounded, the system initiates a retry or triggers web search.

πŸ“Š Benchmark & Evaluation

Evaluation conducted using the RAGAS framework and the ragas-wikiqa dataset comparing CorrectiveRAG against a standard baseline RAG chain (retriever | prompt | llm | parser):

Evaluation Metric CorrectiveRAG Baseline Naive RAG Improvement
Faithfulness (Factuality) 0.9139 0.8730 +4.68%
Answer Relevancy 0.7952 0.6624 +20.05%
Average Response Time ~26.2 s ~9.6 s (Trade-off for multi-step verification)

Key Takeaway: CorrectiveRAG provides a significant gain in answer faithfulness and relevancy, making it ideal for high-stakes enterprise applications (finance, legal, compliance, medical) where accuracy is paramount.


πŸš€ Getting Started

1. Prerequisites

  • Python: 3.10, 3.11, or 3.12
  • Ollama: Download and install Ollama
  • Pull your chosen LLM and embedding model (default is llama3):
    ollama pull llama3

2. Installation

Clone the repository:

git clone https://github.com/shrridharshan27/CorrectiveRAG.git
cd CorrectiveRAG

Create and activate a virtual environment:

# Windows
python -m venv venv
.\venv\Scripts\activate

# Linux / macOS
python3 -m venv venv
source venv/bin/activate

Install dependencies:

pip install -r requirements.txt

3. Configuration

Copy .env.example to .env:

cp .env.example .env

Fill in your API keys (optional for local document RAG; required for Google search fallback):

OLLAMA_URL=http://localhost:11434
MODEL_NAME=llama3
EMBEDDING_MODEL=llama3

# Optional: For Web Search Fallback
GOOGLE_CSE_ID=your_custom_search_engine_id
GOOGLE_API_KEY=your_google_cloud_api_key

# Optional: For Running RAGAS Evaluation
OPENAI_API_KEY=your_openai_api_key

4. Run the Application

Launch the modern Streamlit interface:

streamlit run app.py

Open your browser at http://localhost:8501.


🐳 Docker & Containerized Deployment

Run the complete stack (Streamlit App + ChromaDB) using Docker Compose:

docker compose up --build -d

The app will be accessible at:

  • Streamlit Frontend: http://localhost:8501
  • ChromaDB API: http://localhost:8000

To tear down:

docker compose down

☁️ AWS EC2 Deployment Guide

For production or cloud GPU hosting:

  1. GPU Instance Setup:
    • Launch an AWS EC2 instance (e.g., g4dn.xlarge with NVIDIA T4 GPU and 50+ GB SSD).
    • Inbound security group rules: Open port 11434 (Ollama) and 8501 (Streamlit).
  2. Install Ollama & GPU Drivers:
    curl -fsSL https://ollama.com/install.sh | sh
    ollama pull llama3
  3. Run Containerized Application:
    • Point OLLAMA_URL in your .env or application.yaml to the EC2 host IP or private DNS.
    • Run docker compose up -d.

πŸ“ Project Structure

CorrectiveRAG/
β”œβ”€β”€ .env.example              # Configuration template
β”œβ”€β”€ .gitignore                # Git ignore rules
β”œβ”€β”€ Dockerfile                # Production multi-stage Docker build
β”œβ”€β”€ LICENSE                   # MIT License
β”œβ”€β”€ README.md                 # Project documentation
β”œβ”€β”€ agent.py                  # LangChain agent declarations & tool definitions
β”œβ”€β”€ app.py                    # Streamlit frontend with interactive chat & tracing
β”œβ”€β”€ application.yaml          # Model & environment settings
β”œβ”€β”€ arch.png                  # Architecture & state machine diagram
β”œβ”€β”€ demo.gif                  # UI walkthrough animation
β”œβ”€β”€ docker-compose.yml        # Docker Compose configuration (App + ChromaDB)
β”œβ”€β”€ document_loader.py        # Multi-format document loader (PDFs & Web URLs)
β”œβ”€β”€ embeddings.py             # Custom Ollama embeddings integration
β”œβ”€β”€ evaluate.py               # RAGAS benchmarking & metric evaluation script
β”œβ”€β”€ prompts.yaml              # Prompt templates for grading and generation
β”œβ”€β”€ rag_app.py                # Core application orchestrator
β”œβ”€β”€ requirements.txt          # Python dependencies
β”œβ”€β”€ vectorstore_retriever.py  # Chroma vector store manager
β”œβ”€β”€ workflow.py               # LangGraph state machine & conditional edges
└── yaml_reader.py            # Environment-aware YAML configuration parser

βš™οΈ Configuration Reference

Parameter Default Description
model llama3 Main reasoning LLM served via Ollama
embedding_model llama3 Vector embedding model
collection_name rag-chroma Target collection name in ChromaDB
ollama_url http://localhost:11434 Ollama service endpoint
GOOGLE_CSE_ID "" Google Programmable Search Engine ID
GOOGLE_API_KEY "" Google Cloud API key for Web Search

πŸ—ΊοΈ Roadmap

  • LangGraph Multi-Agent Self-Correction Graph
  • PDF & Web URL ingestion support
  • Google Search API fallback
  • Streamlit Chat Interface with real-time reasoning trace
  • Docker & Docker Compose setup
  • Multi-modal support (charts, images, audio transcription)
  • Semantic query caching with Redis
  • Streaming tokens directly into chat bubbles

πŸ‘€ Author & License

Developed and maintained by shrridharshan27.

Distributed under the MIT License. See LICENSE for more details.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages