Agentic RAG Engine

An AI assistant that becomes an expert on a set of technical PDFs and answers grounded, cited questions about them — from a simple chat interface.

It ships pre-loaded with Wärtsilä 32 marine-engine service manuals: ask it a spec, a step-by-step procedure, or describe a fault and get a diagnosis built only from the manuals (never invented), with inline citations and diagrams.

Just want to run it? Jump to Quickstart — 5 steps, no Docker, no GPU. Everything fancy (local offline LLM, the Neo4j fault-cascade graph, the public tunnel) is optional and lives under Advanced.

---

Quickstart

Runs the full demo with Groq for answers, local embeddings, and an embedded on-disk Qdrant — no Docker, no Ollama, no separate database server.

What you need first

• Python 3.11+ — https://www.python.org/downloads/ (tick Add Python to PATH)
• A free Groq API key — https://console.groq.com → API Keys → create key
• ~3 GB free disk and internet (first run downloads ~600 MB of embedding/reranker models from HuggingFace, then caches them)

You do not need Node, Docker, a GPU, or Ollama for the quickstart.

5 steps

# 1. Get the code
git clone https://github.com/kartikdubeycoded/agentic-rag-engine.git
cd agentic-rag-engine

# 2. Create a virtual environment and install dependencies
python -m venv venv
venv\Scripts\pip install -r requirements.txt      # Windows
# source venv/bin/activate && pip install -r requirements.txt   # macOS/Linux

# 3. Configure — copy the example env and paste your Groq key
copy .env.example .env                              # Windows  (cp on macOS/Linux)
#   then edit .env and set:  GROQ_API_KEY=gsk_...

# 4. Download the knowledge base and build the vector index
#    (one-time; ~5 min on a GPU, ~20-25 min on CPU-only)
venv\Scripts\python scripts/setup_data.py

# 5. Run it
venv\Scripts\python -m backend.app

On Windows you can replace step 5 with a double-click on run.bat.

Open http://localhost:5000 and log in:

Username	Password	Role
engineer1	eng123	Engineer — create tickets, chat with the bot
admin	admin123	Admin — review & approve field fixes

Try asking: "What is the normal jacket water outlet temperature?"

---

Where does the data come from?

The Wärtsilä manuals themselves aren't in the repo, but the processed chunks (the searchable text extracted from them) are published as a GitHub Release asset. scripts/setup_data.py handles everything:

1. Downloads all_chunks.json from the Release (set CHUNKS_URL in .env, or it uses the default release asset) — skipped if the file is already present.
2. Embeds every chunk locally with nomic-embed-text-v1.5 (sentence-transformers) — fast on a GPU, ~20-25 min CPU-only for the full Wärtsilä set.
3. Indexes the vectors into an embedded Qdrant at db/qdrant_local/.

Default data source: https://github.com/kartikdubeycoded/agentic-rag-engine/releases/download/data-v1/all_chunks.json

To use your own PDFs instead, see Rebuild from your own PDFs.

---

Using the system

• Spec lookup: "What is the normal cooling water pressure range?"
• Procedure: "Walk me through replacing the CWP mechanical seal."
• Diagnosis: create a ticket with several alerts, then ask "what's causing these alarms and what should I do?" (Root-cause via the graph needs the Neo4j add-on; without it you still get a grounded RAG answer from the manuals.)
• Self-improvement: after fixing something, submit a Resolution Form. An admin approves it and it's embedded back into the knowledge base for the next engineer.

Answers are built only from retrieved manual text, graded before delivery, and carry inline citations. Nothing is invented.

---

Troubleshooting

Symptom	Fix
GROQ_API_KEY / "Groq API key not configured"	Put a valid key in .env (GROQ_API_KEY=gsk_...) and restart.
Setup can't find chunks	Set CHUNKS_URL in .env to the Release asset URL, or place all_chunks.json in data/chunks/.
Search returns nothing	You skipped step 4 — run python scripts/setup_data.py and confirm it reports ~12,000 vectors.
First answer is slow	First request loads the embedding + reranker models into memory (~20-40 s). Subsequent answers are fast.
Model download blocked / offline	The embedding + reranker download from HuggingFace on first run. On a locked-down machine, see Fully offline.
Port 5000 in use	Set FLASK_RUN_PORT=5001 in .env and open http://localhost:5001.
Blank page	The prebuilt UI ships in frontend/dist/. If you changed frontend code, rebuild: cd frontend && npm run build.

---

Tech stack

Layer	Quickstart default	Advanced option
Answers (LLM)	Groq llama-3.3-70b (cloud, free)	Local Ollama, or offline GGUF (llama-cpp)
Embeddings	nomic-embed-text-v1.5 (sentence-transformers, local)	Ollama nomic-embed-text
Vector DB	Qdrant embedded (on-disk, no server)	Qdrant server
Reranker	BAAI/bge-reranker-base	same
Graph DB	(off)	Neo4j Community (fault cascade)
Pipeline	LangGraph (rewrite → route → retrieve → rerank → grade → generate)	same
Backend / Frontend	Flask + prebuilt React	same
App DB	SQLite (auto-created, users seeded)	same

---

Results

Tested on 21 diagnostic questions (spec lookups, procedures, multi-symptom diagnosis):

Metric	Score
Recall@5	0.76 (16/21)
Hybrid search vs dense-only	+12% recall
Reranker	+18% top-5 precision

Full RAGAS results in evaluation/results/ragas_results.json (re-run with python evaluation/benchmark.py --ragas — needs requirements-advanced.txt).

---

Advanced

Everything below is optional — the quickstart does not need any of it.

Run fully offline (no internet)

For a locked-down machine with no outbound internet:

1. Pre-download the embedding + reranker models on a machine that has internet and copy them into models/embedding_model/ and models/reranker/, then point EMBEDDING_MODEL_PATH / RERANKER_MODEL_PATH at those folders in .env.
2. Use a local LLM instead of Groq — in .env:

   WORK_MODE=true
   LLM_BACKEND=llamacpp
   LLAMACPP_MODEL_PATH=C:\path\to\your\model.gguf
   

   (requires pip install llama-cpp-python).
3. Place all_chunks.json in data/chunks/ manually (no CHUNKS_URL needed).

This is the airgapped path — answers are generated on CPU and are slower (20-40 s).

Use a local LLM via Ollama

Install Ollama (https://ollama.com), then:

ollama pull llama3.1:8b
ollama pull nomic-embed-text

In .env:

LLM_BACKEND=ollama
EMBED_BACKEND=ollama
OLLAMA_BASE_URL=http://localhost:11434

Answers run locally on your GPU; the code still falls back to Groq if Ollama is busy.

Enable the Neo4j fault-cascade graph

This unlocks true multi-alarm root-cause detection (the CWP-A seal-failure demo). Requires Docker.

docker run -d --name neo4j-gt46mac -e NEO4J_AUTH=neo4j/gt46mac123 \
  -p 7474:7474 -p 7687:7687 neo4j:community
pip install -r requirements-advanced.txt
python scripts/06_load_neo4j.py

Then set in .env:

NEO4J_URI=bolt://localhost:7687
NEO4J_USER=neo4j
NEO4J_PASSWORD=gt46mac123

Leave NEO4J_URI blank to keep the graph disabled (the pipeline falls back to pure RAG).

Run against a Qdrant server instead of embedded

Download the Qdrant binary (https://github.com/qdrant/qdrant/releases), run it, then in .env set QDRANT_HOST=localhost and QDRANT_PORT=6333. Re-run scripts/setup_data.py to index into the server.

Rebuild the knowledge base from your own PDFs

The engine is domain-agnostic. To build a knowledge base from your own manuals:

# place PDFs in data/raw_pdfs/, then:
python scripts/01_extract_text.py     # PDF → text
python scripts/03_chunk_and_tag.py    # text → chunks with metadata (writes all_chunks.json)
python scripts/setup_data.py          # embed + index

For the fault-cascade graph, edit the rules in scripts/06_load_neo4j.py for your domain.

Expose it publicly / platform mode

Users can sign up and chat with their own uploaded PDFs (private per-account collections, page citations, extracted diagrams). Run start_public.bat to start everything behind a free Cloudflare tunnel and print a shareable https://….trycloudflare.com URL. See SCALE_PLAN.md.

---

Project structure

agentic-rag-engine/
├── backend/            # Flask backend
│   ├── api/            # chat, tickets, forms, admin, docs endpoints
│   ├── rag/            # LangGraph pipeline, retriever, confidence scoring
│   └── services/       # qdrant (embedded/server), neo4j (optional), bm25, reranker, llm
├── frontend/dist/      # prebuilt React app (served by Flask — no Node needed to run)
├── scripts/
│   ├── setup_data.py   # download chunks + embed + index  ← the one you run
│   └── 0x_*.py         # full rebuild-from-PDF pipeline (advanced)
├── evaluation/         # Recall@5 + RAGAS harness
├── requirements.txt            # quickstart deps
├── requirements-advanced.txt   # Neo4j / RAGAS / diagram-parse extras
├── .env.example        # copy to .env, add GROQ_API_KEY
├── run.bat / run.sh    # one-command launch (embedded Qdrant)
└── start.bat           # advanced launcher (Qdrant server + Ollama)

---

Built with Python, LangGraph, Qdrant, sentence-transformers, Groq, Flask, and React. Neo4j and Ollama optional.