📚 Pathok (পাঠক): Flask + LangChain + FAISS + Groq (Free API)

"Pathok" means "the reader" in Bengali.

A small, fully working Retrieval-Augmented Generation (RAG) project you can run locally in minutes. Upload a PDF/TXT/MD document, then ask questions about it - the app retrieves the most relevant chunks from a FAISS vector store and asks a free LLM (via Groq) to answer using that context.

Stack

Layer	Tool	Notes
Backend	Flask	Simple REST endpoints
Orchestration	LangChain	Loaders, splitters, chains
Embeddings	HuggingFace sentence-transformers/all-MiniLM-L6-v2	Runs locally, 100% free
Vector DB	FAISS	Local, free, persisted to disk
LLM	Groq API (llama-3.1-8b-instant)	Free tier, very fast
Frontend	Plain HTML/CSS/JS	No build step needed

No paid API keys are required. Groq's free tier is generous and fast enough for this project.

1. Get a free Groq API key

1. Go to https://console.groq.com/keys
2. Sign up (free) and create an API key.
3. Copy it — you'll need it in step 3 below.

(Alternative free LLM providers you could swap in: Google Gemini free tier, OpenRouter free models, or a local Ollama model — see "Swapping the LLM" below.)

2. Setup

# 1. Create and activate a virtual environment
python -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate

# 2. Install dependencies
pip install -r requirements.txt

3. Configure your API key

cp .env.example .env

Edit .env and paste your key:

GROQ_API_KEY=gsk_your_actual_key_here

4. Run the app

python app.py

Open your browser at http://127.0.0.1:5000

5. Use it

1. Upload a .pdf, .txt, or .md file — it gets chunked, embedded, and stored in FAISS (persisted under vectorstore/faiss_index/, so it survives restarts).
2. Ask a question in the box below — the app retrieves the top matching chunks and sends them
   • your question to the LLM, then displays the answer along with the source snippets used.
3. Click "Clear Vector Store" to wipe the index and start over.

Project structure

rag_project/
├── app.py                  # Flask routes (/, /upload, /ask, /reset, /health)
├── rag_utils.py             # RAG pipeline: loading, splitting, embedding, retrieval, LLM chain
├── requirements.txt
├── .env.example
├── templates/
│   └── index.html
├── static/
│   ├── style.css
│   └── app.js
├── uploads/                 # Uploaded source files land here
└── vectorstore/
    └── faiss_index/          # Persisted FAISS index (created on first upload)

How the RAG pipeline works (rag_utils.py)

1. Load — PyPDFLoader / TextLoader reads the uploaded file.
2. Split — RecursiveCharacterTextSplitter breaks it into ~800-character overlapping chunks.
3. Embed — each chunk is embedded locally using a small sentence-transformers model (no API call, no cost, runs on CPU).
4. Store — chunks + embeddings go into a FAISS index, saved to disk so it persists across runs.
5. Retrieve — on a question, FAISS returns the top-k most similar chunks.
6. Generate — LangChain's create_stuff_documents_chain stuffs those chunks into a prompt template and sends it to the Groq-hosted LLM, which answers grounded in that context only.

Swapping the LLM (optional)

rag_utils.py isolates the LLM in one place:

self.llm = ChatGroq(model=GROQ_MODEL, api_key=api_key, temperature=0.2)

To use a different free provider instead, replace this with e.g.:

• Google Gemini (free tier): from langchain_google_genai import ChatGoogleGenerativeAI
• Ollama (fully local, no API key at all): from langchain_ollama import ChatOllama
• OpenRouter free models: use ChatOpenAI pointed at OpenRouter's base URL with a free model id.

Everything else (retrieval, chunking, FAISS) stays the same.

Troubleshooting

• GROQ_API_KEY not set error — make sure you copied .env.example to .env and filled in a real key.
• Slow first run — the embedding model (~90MB) downloads once and is cached locally afterward.
• Large PDFs — increase chunk_size in rag_utils.py or reduce k in ask() if answers seem truncated or slow.
• Rate limits — Groq's free tier has generous but finite rate limits; wait a few seconds and retry if you hit one.
• ValueError: ... Keras 3 ... install tf-keras — this happens when TensorFlow is also installed in your Python environment and transformers tries to use it instead of PyTorch. rag_utils.py already sets USE_TF=0 / TRANSFORMERS_NO_TF=1 at the top to prevent this. If you still hit it, your environment likely has leftover packages from another project — create a fresh virtual environment dedicated to this project (see PyCharm setup below) rather than reusing one from elsewhere.
• 'cp' is not recognized (Windows) — cp is a Linux/Mac command. On Windows use copy .env.example .env (CMD) or Copy-Item .env.example .env (PowerShell) instead.

Running in PyCharm

1. Unzip the project and File → Open... the pathok folder (the one containing app.py).
2. Create a dedicated virtual environment for this project — don't reuse a venv from another project, that's what causes the TensorFlow/Keras conflict above:
   • PyCharm will usually prompt "No interpreter configured" → Add New Interpreter → Add Local Interpreter → Virtualenv Environment → New.
   • Or manually: File → Settings → Project: pathok → Python Interpreter → Add Interpreter.
3. Open the PyCharm terminal (Alt+F12), confirm the prompt shows (venv), then run:

   pip install -r requirements.txt
   

4. Create .env (right-click project root → New → File → name it .env) and add:

   GROQ_API_KEY=gsk_your_actual_key_here
   

5. Right-click app.py → Run 'app'. Ctrl+Click the http://127.0.0.1:5000 link in the console.

Extending this project

• Add authentication before deploying publicly.
• Swap FAISS for a hosted vector DB (Pinecone, Qdrant, Chroma Cloud) for multi-user / production use.
• Add conversation memory (ConversationBufferMemory) for multi-turn chat instead of single-shot Q&A.
• Support more file types (docx, csv) by adding more LangChain document loaders.