RAG

This repository contains a Django-based retrieval-augmented generation (RAG) app using pgvector extension.

Project flow

1. POST /retrieval/upload/ accepts a file.
2. The app hashes the file and reuses an existing document ID if the same file was already indexed.
3. Text is extracted from PDFs, DOCX files, or plain text files.
4. OCR text is collected from PDF image regions when available.
5. The app chunks extracted content, generates embeddings, and stores them in PostgreSQL with pgvector.
6. POST /retrieval/query/ retrieves the most relevant chunks for a given document and asks OpenAI to answer from that context only.
7. GET /retrieval/prev-docs/ returns previously indexed documents.

Setup

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
python manage.py migrate
python manage.py test
python manage.py runserver

The database must be PostgreSQL with the pgvector extension available. The repository includes a migration that creates the documents and chunks tables used by the app.

Environment variables

• DJANGO_SECRET_KEY
• DJANGO_DEBUG
• DJANGO_ALLOWED_HOSTS
• DB_ENGINE
• DB_NAME
• DB_USER
• DB_PASSWORD
• DB_HOST
• DB_PORT
• OPENAI_API_KEY
• OPENAI_CHAT_MODEL
• EMBEDDING_MODEL