RAG: From Naive to Graph

Five Retrieval-Augmented Generation systems, built from scratch, over real financial filings.

No LangChain. No LlamaIndex. No API keys. Every line of every pipeline is visible, local, and measured against the same benchmark.

Why this exists · The modules · Quick start · Benchmark · Roadmap · Contributing

---

Most RAG tutorials stop at "embed your docs and call an LLM." This repository is what comes after: a progressive series where each module is a complete, runnable QA system over corporate 10-K filings, and each new module exists to fix a failure mode you can actually observe in the previous one.

flowchart LR
    A["🧱 Naive RAG<br/><i>retrieve → read</i>"] --> B["⚡ Advanced RAG<br/><i>hybrid + rerank + guardrail</i>"]
    A --> C["🔀 Hybrid RAG<br/><i>dense + BM25 fusion</i>"]
    A --> D["🤖 Agentic RAG<br/><i>multi-hop ReAct loop</i>"]
    A --> E["🕸️ Graph RAG<br/><i>knowledge-graph retrieval</i>"]

Loading

Because every module shares the same embedding model, same LLM, same PDF parser, and same evaluation set, the differences you measure come from the pipeline design — not the components.

The modules

Module	Status	The problem it solves	Key techniques
Naive RAG	✅ Implemented	The baseline — and the source of every failure mode below	Fixed-size chunking, dense top-k, direct prompt injection
Advanced RAG	✅ Implemented	Hallucinations, bad chunk boundaries, weak ranking	Semantic chunking, hybrid search, cross-encoder reranking, grounding guardrail
Hybrid RAG	✅ Implemented	Dense retrieval misses exact terms (tickers, figures)	BM25 + dense fusion via RRF, three switchable retrieval modes
Agentic RAG	✅ Implemented	Single-shot retrieval can't answer multi-hop questions	Custom ReAct loop, tool use, iterative retrieval
Graph RAG	✅ Implemented	Similarity search can't follow entity relationships	LLM triple extraction, entity resolution, graph traversal, community summaries

All five pipelines are implemented and runnable. The cross-module FinanceBench comparison (one results table, every architecture, same questions) is the next milestone — see the Roadmap.

Every module is self-contained: its own README.md, requirements.txt, tests, and (for in-progress modules) a phased PLAN.md. Start with whichever tier matches what you're trying to learn.

Why this exists

1. As a learning path. The RAG literature is a zoo of techniques with no clear sense of which ones matter. Here, each technique is introduced only when a measured failure demands it — so you learn not just how but why.
2. As honest reference code. Framework abstractions hide exactly the parts worth understanding. Every pipeline here is plain Python you can read top to bottom in an afternoon.
3. As a controlled experiment. Shared components + shared benchmark = directly comparable numbers across all five architectures.

Quick start

Each module has its own setup guide, but they share two prerequisites:

# 1. A local LLM via Ollama (no API keys anywhere in this repo)
ollama pull qwen3.5:4b

# 2. Qdrant — most modules use Docker...
docker run -d --name qdrant -p 6333:6333 -p 6334:6334 \
  -v $(pwd)/qdrant_storage:/qdrant/storage:z \
  qdrant/qdrant:v1.18.2

Tip

Naive RAG needs no Docker at all — it runs Qdrant in embedded local mode (on-disk, in-process), so it works 100% offline. It's the fastest module to get running.

Then pick a module and follow its README:

cd "Naive RAG"          # or "Advanced RAG"
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

The benchmark

All modules are evaluated on FinanceBench — a published QA dataset over corporate 10-K filings, with baselines from GPT-4, GPT-4-Turbo, Claude-2, and Llama-2 to compare against. Financial QA is a deliberately hard target: it demands exact figures, table comprehension, and multi-document reasoning — precisely the things that separate the five architectures.

Repository structure

RAG/
├── Naive RAG/        Baseline retrieve-then-read pipeline (embedded Qdrant, fully offline)
├── Advanced RAG/     Production-grade pipeline: hybrid search, reranking, grounding
├── Hybrid RAG/       Dense + BM25 fusion via RRF, three switchable retrieval modes (embedded Qdrant, fully offline)
├── Agentic RAG/      Multi-hop retrieval with a custom ReAct agent (embedded Qdrant, fully offline)
├── Graph RAG/        LLM-extracted knowledge graph: triples, entity resolution, communities, traversal
├── ROADMAP.md        Build order and rationale
├── CONTRIBUTING.md   How to contribute (and the repo's design philosophy)
└── LICENSE           MIT

Contributing

Contributions are welcome — especially ones that make a module a better teacher. The one rule: don't make a module smarter than its tier (a grounding check in Naive RAG defeats its purpose). See CONTRIBUTING.md.

License

Released under the MIT License. Use it, fork it, learn from it — and if you build something on top of it, I'd love to hear about it.

---

_{Built by Jaysheel Dodia · If this helped you understand RAG, consider giving it a ⭐}