rag

Retrieve information from your own PDFs and see exactly where the answer comes from.

A small, local-first RAG stack designed to run end-to-end on a laptop. PDFs go through a parsing pipeline (Docling + MLX-accelerated formula / code models), the resulting chunks are embedded and stored in Postgres + Qdrant, and a React inspector lets you search the corpus and trace each answer back to its bounding box on the source page.

Built and benchmarked alongside the blog post — RAG on a Laptop — which covers the model choices, the MLX ports, and the per-stage numbers behind the pipeline you see here.

Quick start

Prerequisites: Docker, uv, pnpm, an Apple Silicon Mac (MLX is hard-coded in a few places). Python 3.12+.

git clone https://github.com/leonnoirclerc/rag
cd rag
cp .env.example .env

make setup       # uv sync + pnpm install
make up          # docker compose: Postgres + Qdrant
make db-migrate  # apply schema
make dev         # backend on :8100, frontend on :5173

Open http://localhost:5173, drop a PDF into the Library view, and watch it ingest. Press ⌘K (or Ctrl+K) anywhere to ask a question.

What's inside

The pipeline is a single FastAPI service that orchestrates:

• Parsing — Docling, with two stages re-routed through MLX:
  • doc_enrich (code + formula transcription): CodeFormulaV2 → MLX (mlx-community/CodeFormulaV2-mlx-q8), 24.9× faster than Docling's CPU default on math-heavy papers.
  • layout: docling-layout-heron (RT-DETRv2) is upstreamed in mlx-vlm#1195 and ready to plug in once Docling adds the matching ObjectDetectionEngineType.MLX. Currently still torch-on-MPS in this repo.
  • table_structure: TableFormerV2 on CPU (small enough that the cost is acceptable; do_cell_matching=False to keep the cell grid intact when the PDF text layer disagrees with the model).
• Embedding: BGE-M3 dense + sparse (MLX), Jina-ColBERT-v2 late interaction (MLX) for reranking.
• Storage: Postgres for documents / chunks / elements (SQLAlchemy + Alembic), Qdrant for dense vectors, on-disk blob store for PDFs + figures.
• Frontend: React 19 + Vite + TanStack Query + react-pdf + shadcn/ui. ⌘K search palette, side-by-side PDF inspector + chunk panel, bbox selection on the PDF page.

The blog post has the per-stage timings, the embedder comparison, and the recall numbers behind the picks above.

Project layout

src/rag/                  Backend
  api/                      FastAPI routers + SSE event bus
  db/                       SQLAlchemy models + Alembic migrations
  ingestion/                Docling pipeline + domain mapping
  embedding/                Dense / sparse / late-interaction backends
  retrieval/                Qdrant search + reranking
frontend/src/             React app
  components/inspect/       PDF viewer + chunk panel + table/formula render
  components/search/        ⌘K search palette
  api/                      Typed fetch hooks (TanStack Query)
benchmark/pdf_pipeline/   Reproducibility for the blog's stage benchmarks
docker-compose.yml        Postgres + Qdrant for local dev
Makefile                  Setup / dev / db / lint / test targets

Reproducing the blog's numbers

The benchmark/pdf_pipeline/ directory contains every pipeline variant the blog compared (traditional / VLM / codeformulav2_mlx / per-layout-model variants):

uv run python -m benchmark.pdf_pipeline --variants codeformulav2_mlx

Results land in benchmark/pdf_pipeline/results/ as timestamped JSON, one file per (variant, PDF). The blog walks through the comparison.

Tests + checks

make check                 # ruff + mypy + pytest
make frontend-typecheck

License

MIT — see LICENSE. Some components (Docling, the CodeFormulaV2 weights, the docling-layout-heron weights, TableFormerV2) ship under CDLA-Permissive-2.0; upstream model cards have the details.