Ask it a medical question. It finds the relevant passages in a small set of medical documents, writes an answer using only those passages, and then goes back through its own answer sentence by sentence and tests each one against the sources. You get the answer and a verdict for every claim in it.
Educational project. General information, not medical advice.
A normal RAG system does two things: find some text, hand it to a language model, print what comes back. Nothing in that loop ever asks whether the answer actually follows from the text. The model can quietly blend a retrieved fact with something it half-remembers from training, and the output looks exactly the same either way — fluent, confident, formatted, cited.
In most domains that is annoying. In a medical one it is dangerous, and the danger is specifically that you cannot tell from looking.
So this project adds a step. After the answer is written, it is split into individual claims, and each claim is checked against the retrieved passages by a model trained to judge exactly that question: does this passage entail this statement, contradict it, or neither? Each claim comes back labelled:
| Verdict | Meaning |
|---|---|
| Supported | A passage states this |
| Weak evidence | A passage leans this way but does not say it outright |
| Unsupported | Nothing in the sources establishes this |
| Contradicted | A passage says the opposite |
Those verdicts roll up into one number — the percentage of the answer that is actually grounded. That is the whole idea: make hallucination visible and countable instead of a thing you worry about.
Ingestion (once, offline) Answering (per question)
───────────────────────── ────────────────────────
documents question
↓ split into chunks ↓
↓ embed each chunk safety guardrail ── emergency? → stop, give a helpline
↓ ↓
┌───────────┬──────────┐ ┌────────────┬──────────┐
│ Chroma │ BM25 │ ──────────► │ semantic │ keyword │ search both ways
│ (meaning) │ (words) │ │ search │ search │
└───────────┴──────────┘ └─────┬──────┴─────┬────┘
└──── fuse ───┘ rank by agreement
↓
re-rank, keep the best 5
↓
LLM writes an answer from those 5
↓
split answer into claims
↓
NLI checks each claim vs each passage
↓
trust score + citations + contradictions
Why search twice. Embeddings understand meaning but are careless about exact strings — "metformin" and "metoprolol" sit close together in vector space and are completely different drugs. Keyword search (BM25) has the opposite problem: it nails the drug name and misses "what lowers blood sugar" when the document says "reduces glucose production". Running both and fusing the results gets you the answer either way.
Why fuse by rank instead of score. A BM25 score and a cosine similarity are not on the same scale and there is no principled way to convert one to the other. Reciprocal Rank Fusion sidesteps that entirely — it only asks how near the top did each method put this passage, which stays meaningful no matter what either method's raw numbers look like.
python smoke_test.pyNo installation needed at all. Runs in a second and checks the core logic.
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
# the local LLM — https://ollama.com
ollama pull qwen3:8b
python ingest.py --data ./data # build the indexes (once)
uvicorn app:app --reload # then open http://localhost:8000Everything runs on your machine. No question ever leaves it, which for medical queries is the point rather than a nice-to-have.
pytest96 tests, about a second, no machine-learning dependencies required — the pure-logic layer imports without torch or Chroma on purpose.
python -m eval.run_eval --no-generation # retrieval quality only, fast
python -m eval.run_eval # everything, needs OllamaAblations, to check the design earns its complexity:
USE_HYBRID=true ENABLE_RERANK=true python -m eval.run_eval # full system
USE_HYBRID=false ENABLE_RERANK=true python -m eval.run_eval # no keyword search
USE_HYBRID=true ENABLE_RERANK=false python -m eval.run_eval # no re-rankerWhat the numbers mean:
- Recall@k — was the right document in the top k? (1.0 = always)
- MRR — how near the top was it? (1.0 = always first)
- nDCG@k — rank quality, 0 to 1
- Hallucination rate — share of claims unsupported or contradicted
- Groundedness — share of claims supported or weakly supported
The live demo is a TypeScript port of the same pipeline, in web/. It
exists because the Python version cannot be hosted: torch, ChromaDB and
bart-large-mnli come to roughly 2.5 GB, and a Vercel function has a 250 MB
limit. Ten times over budget is not a packaging problem to solve, it is a
different deployment.
What is identical:
- the chunks — same documents, same splitter settings, same 44 chunks
- BM25, including the constants, and Reciprocal Rank Fusion
- dense search with the same embedding model (all-MiniLM-L6-v2)
- claim splitting, the thresholds, and the trust score arithmetic
- the safety guardrail
What had to change, and what it costs:
| Local | Demo | Consequence | |
|---|---|---|---|
| Generation | Ollama, on your machine | Llama 3.3 70B, hosted | Faster and better, but the question leaves your machine |
| Verification | bart-large-mnli |
An LLM prompted as an NLI judge | Handles messy passages better; less consistent near the threshold, and its numbers are self-reported confidence rather than calibrated probability |
| Re-ranker | cross-encoder | none | Slightly weaker ordering of the top passages |
| Query encoding | server | in your browser | ~25 MB model downloaded once; the server stays tiny |
The browser detail is the interesting one. The corpus vectors are precomputed
at build time, so the only thing needing a neural network at request time is
the question itself — one short string. Doing that on the server would mean
shipping onnxruntime-node, which is 208 MB by itself. Doing it in the browser
costs the server nothing, and if the model fails to load the demo falls back to
keyword-only search and says so on screen rather than pretending.
The verification substitution deserves honesty rather than a footnote. An LLM grading text it just wrote is a real conflict of interest, so the judge is given one passage and one claim — not the question, not the rest of the answer, not the fact that it wrote any of it — at temperature 0 in a separate request. That is a genuine check. It is still not the same instrument as a model trained for the task, and the local version remains the reference implementation.
One secret, one command. The demo calls a hosted chat model twice per question — once to write the answer, once to judge it — and that is the only thing it needs a key for. The embedding model runs in your browser and the corpus vectors are baked in at build time, so there is nothing else to provision.
cd web
npm install
cp .env.example .env.local # then put your key in it
npm run build-corpus # rebuild the index from ../data
npm run dev # http://localhost:3000Get a key from console.groq.com/keys — free
tier, no card. Any OpenAI-compatible endpoint works instead; set CHAT_API_BASE
and CHAT_MODEL and leave the rest alone:
CHAT_API_BASE |
CHAT_MODEL |
|
|---|---|---|
| Groq (default) | https://api.groq.com/openai/v1 |
llama-3.3-70b-versatile |
| OpenAI | https://api.openai.com/v1 |
gpt-4o-mini |
| Local vLLM | http://localhost:8000/v1 |
whatever you are serving |
To check the wiring without asking a question, GET /api/ask reports whether a
key is visible and what corpus is loaded:
curl localhost:3000/api/ask # {"status":"ok","configured":true,...}configured: false means the key is not reaching the process. Locally that is
usually a missing .env.local; on Vercel it is almost always an environment
variable added after the current deployment was built — the values are bound
at build time, so setting one does not change what is already running. Redeploy.
Deploying your own copy:
cd web
vercel link
vercel env add CHAT_API_KEY production
vercel --prodnpm test # 18 parity tests against the Python behaviourThe shared demo throttles to 8 questions a minute per visitor. One question can
cost seven upstream calls — the answer, then one per claim — so without a cap a
single loop drains the free tier and everyone after it gets an error instead of
a demo. Your own copy has the same limit; raise RATE_LIMIT in
app/api/ask/route.ts if you are paying for the key.
A full write-up is in docs/WHAT_CHANGED.md. The short
version — four bugs that were silently wrong, which is the dangerous kind:
1. The claim splitter was destroying claims. It broke sentences at any period followed by a space. Medical writing is full of periods that do not end sentences — "e.g.", "i.e.", "500 mg.", "45 yrs." — and each one cut a claim in half. Both halves were then usually too short to survive the minimum-length filter, so the claim did not merely get mangled, it disappeared:
"Diagnosis uses an A1C of 6.5% or higher, i.e. two separate tests."
before → ["two separate tests."] ← subject gone, scored UNSUPPORTED
after → the whole sentence, intact
The fragments that survived were unsupportable by construction, so they counted as hallucinations that never happened. This was corrupting the hallucination rate and groundedness — the two numbers the project exists to report.
2. The safety guardrail refused the questions the corpus answers. It matched alarming keywords anywhere in the question, and topic is not intent, so "What are the warning signs of a stroke?" was refused — by a system shipping a document about strokes. The rule now separates a question about the world from someone describing an emergency, and still refuses self-harm on topic alone, because there the cost of being wrong is not symmetric.
3. The search index was a pickle, and it was committed to git.
pickle.load executes whatever the file tells it to. Anyone who could put a
file at that path got code execution at startup. It is JSON now — a search
index is data and never needed that power — and it is a build artifact, so it
is out of git.
4. Editing a document left ghosts in the index. Chunk IDs depend on position in the file, so editing a document gave every later chunk a new ID. New chunks were written, old ones were never deleted. Text you removed stayed retrievable and kept coming back as evidence, with a citation pointing at a file that no longer contained it. Ingestion now prunes what is no longer there.
Also: dead code removed (a thread-safe LRU cache sat fully written and imported
by nothing, while the verifier kept its own unlocked copy), a latent
misalignment in the verifier that would have attributed claims to the wrong
evidence without raising, /health made capable of actually failing, and a
96-test suite pinning every one of these.
app.py FastAPI: /ask, /ask/stream, /health, /ui
ingest.py offline index builder
config.py every setting, read from .env
smoke_test.py zero-dependency sanity check
data/ the medical corpus
frontend/ the local UI
src/
types.py shared dataclasses, no heavy imports
loaders.py chunker.py embeddings.py bm25.py vectorstore.py
fusion.py retriever.py reranker.py generator.py
guardrail.py errors.py factory.py pipeline.py cache.py
trust/ claims, verifier, scorer, contradictions, attribution
eval/ metrics, runner, labeled dataset
tests/ 96 pytest tests
web/ the deployed TypeScript port
docs/ per-phase notes + WHAT_CHANGED.md
- The bundled corpus is 8 short documents. It is enough to demonstrate the mechanism and far too small for the metrics to mean much — point it at MedQuAD for numbers worth quoting.
- Answer quality depends on the model you run.
- NLI verification is approximate. It catches claims that plainly do not follow from the sources; it will not catch a subtly wrong dosage that is phrased like the passage.
- The guardrail screens wording, and wording misleads in both directions.
- Not a medical device. Do not use it for real clinical decisions.