RAG Agent V2

RAG Agent V2 is a Docker-first Retrieval-Augmented Generation demo application for asking questions over uploaded PDF documents. It combines a React frontend, FastAPI backend, PostgreSQL metadata storage, Qdrant vector search, Docling document parsing, OpenAI embeddings, and OpenAI chat completion into one end-to-end portfolio project.

The project is built as a practical RAG system rather than a notebook-only prototype. Admin users upload and manage documents, regular users ask questions through a chat interface, answers stream back to the browser, and citations show which document chunks supported the response.

Highlights

• Full-stack RAG application with React, FastAPI, PostgreSQL, and Qdrant.
• PDF ingestion pipeline using Docling hybrid chunking.
• OpenAI text-embedding-3-small embeddings stored in Qdrant.
• Optional dense + BM25 hybrid search with Qdrant named vectors.
• Optional local cross-encoder reranking before answer generation.
• Streaming chat answers through Server-Sent Events.
• Citation previews with document name, page, section/header, chunk ID, score, and retrieved text.
• JWT access tokens plus rotating refresh tokens stored in PostgreSQL.
• Admin document upload/delete, user creation, password reset, and audit log views.
• Standalone evaluation runner with JSONL datasets and saved result artifacts.
• Docker Compose setup for frontend, backend, PostgreSQL, and Qdrant.

Tech Stack

Area	Technology
Frontend	React, Vite, React Router
Backend	FastAPI, SQLAlchemy, Alembic
Database	PostgreSQL
Vector database	Qdrant
Parsing	Docling, pdfium fallback
Embeddings	OpenAI embeddings API
Answer generation	OpenAI chat completions
Reranking	sentence-transformers cross-encoder
Auth	Password hashing, JWT access tokens, rotating refresh tokens
Tooling	Docker Compose, uv, pytest, ESLint

Architecture

Browser
  |
  | React UI
  v
FastAPI backend
  |
  |-- PostgreSQL
  |     users, refresh tokens, documents, chunks, chat history, audit logs
  |
  |-- Qdrant
  |     dense OpenAI vectors and optional BM25 sparse vectors
  |
  |-- Docling / pdfium
  |     PDF parsing and chunking
  |
  |-- OpenAI
        embeddings and grounded answer generation

The app stores uploaded files in Documents/, parsed chunk metadata in PostgreSQL, and searchable vectors in Qdrant. During a question, the backend embeds the query, retrieves candidate chunks from Qdrant, optionally reranks them, builds a grounded prompt, streams the answer to the frontend, and saves the final answer to chat history.

Main Features

Document Management

• Admin-only PDF upload.
• Background ingestion after upload.
• Document status tracking: uploaded, processing, ready, failed.
• Shared document library for all logged-in users.
• Document download for all logged-in users.
• Admin-only delete that removes the raw file, PostgreSQL rows, and Qdrant vectors.

RAG Chat

• Authenticated chat UI.
• Streaming answers with progress events.
• Saved chat history per user.
• Fallback answer when retrieved context is not enough.
• Citations attached to each answer.
• Citation preview text for inspecting retrieved context.

Retrieval Controls

Runtime settings are controlled through environment variables:

• RAG_HYBRID_SEARCH_ENABLED
• RAG_RERANKER_ENABLED
• RAG_RETRIEVAL_CANDIDATE_LIMIT
• RAG_FINAL_CONTEXT_LIMIT
• RAG_MAX_CHUNKS_PER_DOCUMENT
• RAG_ADJACENT_CHUNK_COUNT
• RAG_MIN_SIMILARITY_SCORE

Authentication And Admin Tools

• Login with username and password.
• Password hashes stored in PostgreSQL.
• JWT access tokens for protected routes.
• Rotating refresh tokens with database-backed revocation.
• Logout revokes the current refresh-token session.
• Admin user creation and password reset.
• Audit log page for important user/admin actions.

Project Structure

backend/
  app/
    api/             FastAPI route modules
    core/            config, logging, middleware, error handling
    db/              SQLAlchemy models, sessions, Alembic integration
    ingestion/       PDF parsing, chunking, embedding, ingestion flow
    rag/             retrieval, reranking, prompt building, answer generation
    security/        password hashing, JWTs, refresh tokens, dependencies
    vector/          Qdrant collection and vector helpers
evaluation/          standalone RAG quality evaluation runner
docs/                public roadmap and portfolio notes
frontend/
  src/               React app, pages, auth helpers, API client
postgres/            PostgreSQL Docker image config
qdrant/              Qdrant Docker image config
deploy/              local/private deployment notes
Documents/           local document storage, ignored by git
docker-data/         local database/vector storage, ignored by git

See docs/roadmap.md for the portfolio-safe roadmap and completed milestone summary.

Prerequisites

• Docker and Docker Compose.
• Python 3.12.
• uv.
• Node.js 22 or compatible npm environment if running the frontend outside Docker.
• OpenAI API key.

Environment Setup

Create your local environment file:

cp .env.example .env

Update at least these values:

OPENAI_API_KEY=your-openai-api-key
OPENAI_CHAT_MODEL=your-chat-model
JWT_SECRET_KEY=replace-with-a-long-random-secret
ADMIN_USERNAME=admin
ADMIN_PASSWORD=change-this-admin-password

For Docker Compose, keep service URLs using Compose hostnames:

POSTGRES_PASSWORD=change-me
DATABASE_URL=postgresql+psycopg://rag_user:change-me@postgres:5432/rag_agent_v2
QDRANT_URL=http://qdrant:6333

The password inside DATABASE_URL must match POSTGRES_PASSWORD.

Quick Start With Docker Compose

Start the full application:

docker compose up -d --build

Initialize the database and seed the admin user:

docker compose exec backend uv run python -m backend.app.scripts.init_db

Open the app:

Frontend: http://localhost:5173
Backend API: http://localhost:8000
FastAPI docs: http://localhost:8000/docs
Qdrant: http://localhost:6334
PostgreSQL host port: 5433

Sign in with the admin credentials from .env.

Ingest Documents

You can upload PDFs from the admin document page in the frontend. The backend saves the file and runs ingestion in the background.

You can also ingest PDFs already placed in Documents/:

docker compose exec backend uv run python -m backend.app.scripts.ingest_demo_documents

The ingestion flow:

1. Reads PDF files from Documents/.
2. Parses and chunks them with Docling.
3. Stores document and chunk rows in PostgreSQL.
4. Generates OpenAI embeddings.
5. Stores dense vectors and BM25 sparse vectors in Qdrant.

API Overview

Area	Endpoints
Health	GET /health, GET /health/db, GET /health/qdrant
Auth	POST /auth/login, POST /auth/refresh, POST /auth/logout, GET /auth/me, POST /auth/change-password
Admin users	GET /admin/users, POST /admin/users, POST /admin/users/{user_id}/reset-password
Documents	GET /documents, POST /documents, GET /documents/{document_id}/download, DELETE /documents/{document_id}
RAG	POST /rag/ask, POST /rag/ask/stream, GET /rag/history
Audit logs	GET /admin/audit-logs

Local Development

Run only PostgreSQL and Qdrant in Docker:

docker compose up -d postgres qdrant

Run backend commands from the host with host-facing service URLs:

POSTGRES_USER=$(grep '^POSTGRES_USER=' .env | cut -d= -f2 | tr -d '"')
POSTGRES_PASSWORD=$(grep '^POSTGRES_PASSWORD=' .env | cut -d= -f2 | tr -d '"')
POSTGRES_DB=$(grep '^POSTGRES_DB=' .env | cut -d= -f2 | tr -d '"')
HOST_DATABASE_URL="postgresql+psycopg://${POSTGRES_USER}:${POSTGRES_PASSWORD}@localhost:5433/${POSTGRES_DB}"

DATABASE_URL="$HOST_DATABASE_URL" \
QDRANT_URL='http://localhost:6334' \
uv run python -m backend.app.scripts.init_db

Start the backend locally:

POSTGRES_USER=$(grep '^POSTGRES_USER=' .env | cut -d= -f2 | tr -d '"')
POSTGRES_PASSWORD=$(grep '^POSTGRES_PASSWORD=' .env | cut -d= -f2 | tr -d '"')
POSTGRES_DB=$(grep '^POSTGRES_DB=' .env | cut -d= -f2 | tr -d '"')
HOST_DATABASE_URL="postgresql+psycopg://${POSTGRES_USER}:${POSTGRES_PASSWORD}@localhost:5433/${POSTGRES_DB}"

DATABASE_URL="$HOST_DATABASE_URL" \
QDRANT_URL='http://localhost:6334' \
uv run uvicorn backend.app.main:app --reload

Start the frontend locally:

cd frontend
npm install
npm run dev

Evaluation Workflow

The evaluation runner imports the backend RAG pipeline directly. It does not require the FastAPI server to be running, but PostgreSQL and Qdrant must be available.

Host-run Python commands should use localhost:5433 for PostgreSQL and localhost:6334 for Qdrant. Docker-container commands should use postgres:5432 and qdrant:6333.

Start the required services:

docker compose up -d postgres qdrant

Check that documents and vectors already exist:

POSTGRES_USER=$(grep '^POSTGRES_USER=' .env | cut -d= -f2 | tr -d '"')
POSTGRES_PASSWORD=$(grep '^POSTGRES_PASSWORD=' .env | cut -d= -f2 | tr -d '"')
POSTGRES_DB=$(grep '^POSTGRES_DB=' .env | cut -d= -f2 | tr -d '"')
HOST_DATABASE_URL="postgresql+psycopg://${POSTGRES_USER}:${POSTGRES_PASSWORD}@localhost:5433/${POSTGRES_DB}"

DATABASE_URL="$HOST_DATABASE_URL" \
QDRANT_URL='http://localhost:6334' \
uv run python - <<'PY'
from sqlalchemy import func, select

from backend.app.core.config import get_settings
from backend.app.db import models
from backend.app.db.session import SessionLocal
from backend.app.vector.qdrant import get_qdrant_client

settings = get_settings()

with SessionLocal() as db:
    document_count = db.scalar(select(func.count(models.Document.id)))
    chunk_count = db.scalar(select(func.count(models.DocumentChunk.id)))

qdrant_count = get_qdrant_client().count(
    collection_name=settings.qdrant_collection_name,
    exact=True,
).count

print(f"documents={document_count}")
print(f"postgres_chunks={chunk_count}")
print(f"qdrant_points={qdrant_count}")
PY

Run a small smoke evaluation without judge calls:

POSTGRES_USER=$(grep '^POSTGRES_USER=' .env | cut -d= -f2 | tr -d '"')
POSTGRES_PASSWORD=$(grep '^POSTGRES_PASSWORD=' .env | cut -d= -f2 | tr -d '"')
POSTGRES_DB=$(grep '^POSTGRES_DB=' .env | cut -d= -f2 | tr -d '"')
HOST_DATABASE_URL="postgresql+psycopg://${POSTGRES_USER}:${POSTGRES_PASSWORD}@localhost:5433/${POSTGRES_DB}"

DATABASE_URL="$HOST_DATABASE_URL" \
QDRANT_URL='http://localhost:6334' \
uv run python -m evaluation.evaluate_rag \
  --limit 3 \
  --no-judge

Run the full judged evaluation:

POSTGRES_USER=$(grep '^POSTGRES_USER=' .env | cut -d= -f2 | tr -d '"')
POSTGRES_PASSWORD=$(grep '^POSTGRES_PASSWORD=' .env | cut -d= -f2 | tr -d '"')
POSTGRES_DB=$(grep '^POSTGRES_DB=' .env | cut -d= -f2 | tr -d '"')
HOST_DATABASE_URL="postgresql+psycopg://${POSTGRES_USER}:${POSTGRES_PASSWORD}@localhost:5433/${POSTGRES_DB}"

DATABASE_URL="$HOST_DATABASE_URL" \
QDRANT_URL='http://localhost:6334' \
uv run python -m evaluation.evaluate_rag \
  --dataset evaluation/datasets/rag_quality_world_history.jsonl \
  --run-name baseline \
  --judge-model gpt-5

Evaluation results are written to:

evaluation/results/<timestamp>-<run-name>/
  run_config.json
  results.jsonl
  results.pretty.json
  results.<question_type>.pretty.json
  summary.json

Generated evaluation results are ignored by default. The repository keeps one curated example result at evaluation/results/20260702T044801Z-hybrid_rerank_ctx8_doc3_adj2_min02/, and evaluation/results/latest.json points to that example.

Validation

Backend tests:

uv run pytest

Frontend lint:

cd frontend
npm run lint

Frontend production build:

cd frontend
npm run build

Docker Compose config check:

docker compose config --quiet

Notes

• This project is scoped as a local or private-network demo, not a public SaaS deployment.
• Documents/ and docker-data/ are runtime data folders and should not be committed.
• New evaluation/results/ runs are ignored except for the curated portfolio example result.
• Host-run Python commands should use localhost:5433 for PostgreSQL and localhost:6334 for Qdrant.
• Docker-container commands should use postgres:5432 and qdrant:6333.
• If an old Qdrant collection uses the previous unnamed-vector schema, rebuild or recreate the collection before enabling hybrid search.

Demo Limitations

• An OpenAI API key and access to the configured embedding, answer, and judge models are required.
• Source documents are not bundled in this repository; upload or place PDFs in Documents/ before ingestion.
• The app uses one shared document library for all users and does not include self-registration or advanced permission groups.
• The deployment files are for local or private-network Docker use. Public TLS, public domain setup, and cloud object storage are out of scope.
• Runtime state is stored locally in Documents/ and docker-data/.
• Evaluation scores depend on the local documents, model settings, retrieval configuration, and judge model used for that run.

Portfolio Focus

This project demonstrates:

• Designing a full RAG workflow from document ingestion to grounded answer generation.
• Building a user-facing web app around retrieval and citations.
• Managing auth, audit logs, background ingestion, and operational error handling.
• Evaluating RAG quality with reproducible result artifacts.
• Keeping local development, Docker runtime, and host-run evaluation workflows separate and documented.