A curator-assist tool that drafts a Reactome annotation for a gene: it finds the relevant literature, extracts the biochemical reactions from the full text, and converts them into draft Reactome data-model instances with a QA report telling a human curator exactly what to fix. It produces a reviewed draft, not a finished annotation — see How to read the result.
The whole pipeline runs from one command:
conda run -n reactome_ai python run_curator.py "GENE"There is a second way in: a RESTful service (curator_llm/) that annotates one paper (a PubMed ID
or an uploaded PDF), lists what needs a curator's attention, and lets the curator refine the draft by chat.
The curator-tool-frontend uses it, and you can call it directly. See
Running the REST service.
- What it does
- Architecture: the 3 agents
- Key Details ← read this
- Prerequisites
- Install
- Configuration (.env)
- Running it
- Running the REST service
- How to read the result
- How full-text extraction works (Tool 4)
- Repository layout
- Known limitations
- Experimental: dense-retrieval investigation
Given a gene symbol, the pipeline:
- Places the gene into a Reactome pathway (even if the gene has no existing annotation).
- Retrieves the most relevant papers from PubMed (or uses papers you supply).
- Extracts biochemical reactions from those papers' full text.
- Converts the reactions into Reactome data-model instances (entities, complexes, reactions, a pathway).
- QA-checks the draft and emits a per-instance report of what a curator must fix.
Output for each gene lands in results/:
results/<gene>_curator_instances.json— the draft annotationresults/<gene>_qa_report.json— the QA findings (overall score + per-instance verdicts)
run_curator.py runs three agents synchronously (no framework, no event loop). The loop
between the Curator and the Reviewer, and the retry cap, live in run_curator.py itself.
┌─────────────── adjustment (Reviewer's proposed knobs) ───────────────┐
▼ │
Curator.run(gene, adjustment) ──▶ Reviewer.review(result) ──sufficient?──────┤ no
(deterministic: 4 tools) (LLM: decide + adjust) │
│ yes / give_up / retry-cap hit │
▼ │
QA.check: build_instances → schema + Neo4j checks + LLM review → verdict ◀────┘
Agent 1 — Curator (reactome_llm/ReactomeCurator.py), deterministic, calls four tools:
| Tool | Module | Does |
|---|---|---|
| 1. Pathway placement | ReactomePathwayPlacement |
Predicts a Reactome pathway from interaction-partner enrichment (LLM-free). |
| 2. Literature retrieval | ReactomeLiteratureExtractor |
Union keyword query → cross-encoder rerank → LLM judge selects the top papers. |
| 3. Full-text resolver | ReactomeFullTextResolver |
Maps each selected PMID to a full-text source (local PDF / PMC XML / miss). |
| 4. Full-text analysis | ReactomeFullTextAnalyzer |
Extracts reactions from the resolved full text (see Tool 4). |
Placement is computed once and reused by retrieval (as the rerank target), by the Reviewer, and by the instance builder.
Agent 2 — Reviewer (ReactomeReviewer.py), LLM (or --no-llm-review for rules): looks at
the Curator's result and returns sufficient / retry (with a proposed adjustment, e.g. broaden
the query) / give_up. It can only propose a retry — run_curator.py owns the cap
(--max-attempts, default 3).
Agent 3 — QA (ReactomeQA.py): turns the approved reactions into instances and validates them:
build_instances(reaction_to_instances.py) — LLM converts reactions → Reactome data-model instances.- Deterministic schema check — required fields (
class,displayName) + referential integrity (every reaction/complex reference resolves to a defined entity — no "dangling" arrows) + optional JSON-schema validation againstresources/reactome_domain_model.json. Structure, not biology; an invalid schema is a hard fail. - Deterministic Neo4j merge check — if a generated pathway already exists in Reactome, it's a merge target, not a conflict: the check pulls the reactions already curated in that pathway so the extracted ones can be added as new reactions rather than duplicating the pathway.
- LLM expert review — one call scoring the whole draft: an overall
qa_scoreplus a per-instance verdict for every instance that isn't curator-ready (needs_revision/bad); anything unlisted isgood. A repair loop re-converts with QA's corrections up to a cap, keeping the best attempt.
-
A QA verdict of FAIL / NEEDS_REVISION is normal. This is a curator-assist tool: the deliverable is a draft plus a precise fix-list. A
FAILmeans "here's a draft and exactly what a human must fix," not "the pipeline broke." A genuinely clean draft is rare for a richly-studied gene. -
The overall
qa_scorehides good instances — so read the per-instance breakdown. One score of0.41can be dragged down by a few bad instances while dozens are fine. The terminal printsper-inst : 87 good · 43 needs-revision · 13 bad (of 143)and names every flagged one;flagged_instancesin the QA report has the same data. Unlisted = good. -
How a PMID is recovered from a PDF (there's no PMID printed on a paper). For a local PDF we read the DOI off page 1, then resolve DOI → PMID via NCBI esearch (
FullTextResolver.build_index). Chain:PDF → page-1 DOI → PMID. A PDF with no page-1 DOI can't be identified and is skipped (logged as "unresolvable"). -
"has-data" vs "cold-start", and the placement gate. The final
RESULTblock classifies the gene: has-data = the gene already has curated Reactome pathways (annotated); cold-start = none yet. The placement gate ispasswhen interaction-partner enrichment gives a confident pathway,failwhen it doesn't (reactions, if any, come back unplaced for manual placement). -
Full-text extraction runs as subprocesses. Tool 4 (
fulltext_extractor.py) shells out torun_extraction.py→run_merge.py→run_review.py(the tested CLIs). They look like standalone scripts but are a live part of the pipeline — don't delete them. -
Two MongoDBs, one server. MongoDB (
:27017) holds two databases the pipeline uses:PUBMED_MONGO_DB(cached PubMed abstracts) andFIS_MONGO_DB(Reactome functional-interaction partners, used for placement). Both must be present. -
--papers-onlyskips retrieval. When a curator already has the papers they want, point the tool at a folder of PDFs and it annotates those directly (placement + extraction + QA), skipping Tool 2. It's a single attempt (nothing to retry without retrieval). See Running it.
- Python 3.10 (conda env recommended; on this setup it's
reactome_ai) - Neo4j with a Reactome graph loaded (Bolt on
:7687) - MongoDB on
:27017with two databases (abstracts + functional interactions) - API keys: Anthropic (extraction/merge/convert/QA), OpenAI (
run_review.pyfull-text review only), NCBI/PubMed (E-utilities)
Three steps — all three are required; the Python install alone will not run the pipeline.
1. Python environment + dependencies:
conda create -n reactome_ai python=3.10
conda activate reactome_ai
pip install -r requirements.txtrequirements.txt is the single, authoritative dependency list — install from it. (There is
deliberately no conda environment.yaml: a stale export from another machine caused more confusion
than it solved.) Core libraries: anthropic, openai, langchain-* / langgraph, neo4j,
pymongo, sentence-transformers + flair + torch (cross-encoder rerank), PyMuPDF (PDF
parsing), pandas/numpy/scipy, pydantic, jsonschema.
2. Services + data — a Neo4j Reactome graph and two MongoDB databases that live outside this repo. This is the step people miss. See Services & data you must set up.
3. .env — API keys + connection strings. See Configuration (.env).
Copy .env.example to .env in the repo root (cp .env.example .env) and fill in your values (values loaded via python-dotenv; the Neo4j/Mongo vars can
alternatively live in your shell or conda env config vars):
# --- API keys ---
ANTHROPIC_API_KEY=sk-ant-...
OPENAI_API_KEY=sk-... # used only by run_review.py (full-text cross-model review)
PUBMED_API_KEY=... # NCBI E-utilities (raises rate limit to 10/sec)
NCBI_EMAIL=you@example.com # optional; NCBI politeness (default is hardcoded)
NCBI_TOOL=curator-tool-llm # optional
# --- Neo4j (Reactome graph) ---
REACTOME_NEO4J_URI=bolt://localhost:7687
REACTOME_NEO4J_USER=neo4j
REACTOME_NEO4J_PWD=your_password
REACTOME_NEO4J_DATABASE=graph.db # the Reactome graph DB name (NOT "reactome")
# --- MongoDB: PubMed abstract cache ---
PUBMED_MONGO_URI=mongodb://localhost:27017
PUBMED_MONGO_DB=pubmed_cache
PUBMED_MONGO_COLLECTION=abstracts
# --- MongoDB: Reactome functional-interaction partners (for placement) ---
FIS_MONGO_DB=idg_pairwise
FIS_MONGO_GENE_INDEX=... # collection name for the gene index
FIS_MONGO_RELATIONSHIPS=... # collection name for the pairwise relationshipsrun_curator.py sets TOKEN_PROFILE=1 itself, so per-gene token/cost accounting is on by default.
The pip install is not enough — the pipeline reads a Reactome graph and two Mongo databases
that are not shipped in this repo. This is what silently breaks a fresh setup, so do each explicitly:
1. Neo4j — the Reactome graph. Install Neo4j and load a Reactome graph database into it
(obtain it from Reactome's data downloads at https://reactome.org — the "Reactome Graph Database";
it is not in this repo). Point REACTOME_NEO4J_URI/USER/PWD/DATABASE at it. On this setup the DB
name is graph.db (not reactome). Sanity check: run_curator.py exits at startup if Bolt :7687
is unreachable.
2. MongoDB — two databases on :27017:
PUBMED_MONGO_DB(PubMed abstract cache) This may need to be pre-loaded. See the script for details on how to load: PubmedHandler.py.FIS_MONGO_DB=idg_pairwise(functional-interaction partners, used for placement) — must be loaded separately. This repo only reads it. The data is produced by the Reactomeorg.reactome.idg.pairwiseJava project (MainApp), or copied from a machine that already has it (see migration below). Without this DB, pathway placement finds no interaction partners and cold-start genes can't be placed.
3. Interaction flat files (resources/interactions/) — BioGRID + IntAct, gitignored (large).
Suggest to download the latest version from the web sites listed in resources/llm_interactions/README.txt. However, if there are any parsing errors, use the versions as described in the file. These feed the non-Mongo interaction path (ProteinProteinInteractionsLoader.load_interactions).
Moving the Mongo DBs between machines (e.g. from a dev box to curator.reactome.org):
mongodump --db idg_pairwise --out /path/to/backup
mongorestore --db idg_pairwise /path/to/backup/idg_pairwise # same for the abstract-cache DBresources/reactome_domain_model.json— the Reactome schema QA validates against.resources/ReactomePathwayGenes_Ver_91.txt— pathway↔gene map for enrichment.
fulltext_pdf/ (curator PDFs you point --papers-dir at), fulltext_cache/ (downloaded PMC XML),
abstract_cache/, and user_config.json (remembers your last --papers-dir).
Note: If you have already run conda activate reactome_ai (i.e. your local conda env has been activated) in the terminal, no need to add "conda run -n reactome_ai" before "python" below.
# One or more genes (full pipeline: retrieval → extraction → QA)
conda run -n reactome_ai python run_curator.py SHANK3
conda run -n reactome_ai python run_curator.py SHANK3 TANC1 CTTNBP2
# Point at a folder of curator PDFs (remembered for next time)
conda run -n reactome_ai python run_curator.py SHANK3 --papers-dir /path/to/pdfs
# Papers-only: skip retrieval, annotate ONLY the PDFs in the folder
conda run -n reactome_ai python run_curator.py SHANK3 --papers-only --papers-dir /path/to/pdfs
# Cheap smoke test (no full text, rule-based reviewer/QA — no LLM cost)
conda run -n reactome_ai python run_curator.py SHANK3 --no-full-text --no-llm-review --no-llm-qa| Flag | Default | Meaning |
|---|---|---|
--papers-dir PATH |
prompt/last used | Folder of local full-text PDFs. |
--papers-only |
off | Skip retrieval; annotate only the PDFs in --papers-dir. Implies full text on; single attempt. |
--max-attempts N |
3 | Curator↔Reviewer retries (full-text mode). |
--max-papers N |
5 | Papers the retrieval judge selects. |
--no-full-text |
off | Retrieval + placement only, no extraction. |
--no-llm-review |
off | Rule-based Reviewer (free, deterministic) instead of the LLM. |
--no-llm-qa |
off | QA's deterministic checks only (skips the LLM expert review + per-instance verdicts). |
Prereqs are checked at startup — Neo4j (:7687) and MongoDB (:27017) must be reachable or the
run exits with a clear message.
curator_llm/ is a FastAPI service for working on one paper at a time. It is
separate from the gene-first pipeline above, shares its install and .env, and is what the curator-tool-frontend's
Paper2Path page talks to. Given a PubMed ID or a PDF it extracts the reactions, checks every supporting quote against
the paper, resolves identifiers against Reactome and UniProt, and produces a draft as staged instances. A curator
can then review the issues it found, ask for changes by chat (edits are only proposed; nothing changes until they are
accepted), and load the result into the curator tool.
From the repo root, with the environment from Install:
conda activate reactome_ai
uvicorn curator_llm.main:app --port 8000Uvicorn is the web server; curator_llm.main:app names the module and the app inside it.
Add --reload while editing code. Check it is up: open http://localhost:8000/docs (it lists every route; it cannot
send the bearer token, so make the calls with curl as below), or
curl -i http://localhost:8000/api/llm/sessions # 401 Unauthorized: it is running and asking for a tokenWhat must be running first
| Needs | Why |
|---|---|
curator-tool-ws at WS_BASE_URL (default http://localhost:9090), with GET /api/auth/verify |
every request is checked there; with ws down every call is refused |
| Neo4j with the Reactome graph | identifier lookups and the check for reactions Reactome already has |
Anthropic key (ANTHROPIC_API_KEY) |
extraction, the draft, QA and chat |
Read from .env (the keys in Configuration apply too):
| Variable | Default | Meaning |
|---|---|---|
WS_BASE_URL |
http://localhost:9090 |
curator-tool-ws, which verifies tokens and roles. |
SESSION_STORE |
memory |
memory, or mongo to keep sessions across restarts (uses PUBMED_MONGO_URI). |
SNAPSHOT_MODE |
record |
record saves every successful run, replay reuses a saved result instead of calling the models, off does neither. See Fast iteration. |
SNAPSHOT_DIR |
data/snapshots |
Where saved results go. |
CHAT_MODEL |
claude-sonnet-5-5 |
Model used by the chat and the per-reaction review. |
CORS_ORIGINS |
http://localhost:4200 |
Comma-separated browser origins allowed to call the API. |
UPLOAD_DIR |
data/uploads |
Where uploaded PDFs are kept, per user. |
LLM_REVIEW |
off | 1 also runs the OpenAI cross-model review (needs OPENAI_API_KEY). |
There is no separate login. Use a curator-tool-ws token, which the service verifies with ws on each request. The
account must have the curator role: no token or an invalid one gets 401, a valid token without that role gets 403.
Tokens are short-lived (about five minutes); the frontend refreshes them for you.
TOKEN=$(curl -s -X POST http://localhost:9090/api/auth/login \
-H 'Content-Type: application/json' \
-d '{"username":"YOUR_USER","password":"YOUR_PASSWORD"}' | tr -d '"')Sessions are private: a session, its job and everything under it are visible only to the curator who started it
(anyone else gets 404).
API=http://localhost:8000/api/llm
AUTH="Authorization: Bearer $TOKEN"
# 1. Start from a PubMed ID (the paper must be open access in PubMed Central) ...
curl -s -X POST $API/sessions -H "$AUTH" -H 'Content-Type: application/json' \
-d '{"pmid":"24751536","focus":"PINK1"}'
# {"sessionId":"3f9c...","jobId":"a81d..."} "focus" (a gene) is optional
# ... or upload a PDF (up to 50 MB):
curl -s -X POST $API/sessions/upload -H "$AUTH" -F file=@paper.pdf -F focus=PINK1
# 2. Poll the job. It runs in the background: about 10-15 minutes the first time.
curl -s $API/jobs/JOB_ID -H "$AUTH"
# {"id":"a81d...","status":"running","progress":"extracting and merging reactions (several minutes)","error":null}
# status goes queued -> running -> done (or failed, with the reason in "error")
# 3. What needs a curator's attention
curl -s "$API/sessions/SESSION_ID/issues?status=open" -H "$AUTH"
# 4. The draft as staged instances: the JSON the curator tool's "load staged instances" takes
curl -s $API/sessions/SESSION_ID/export -H "$AUTH" > staged_instances.json
# 5. The quotes behind one instance (use a negative dbId from the export)
curl -s $API/sessions/SESSION_ID/instances/-5/evidence -H "$AUTH"Piping the output through jq makes it readable.
# Chat streams its reply as server-sent events (-N stops curl buffering them)
curl -N -X POST $API/sessions/SESSION_ID/chat -H "$AUTH" -H 'Content-Type: application/json' \
-d '{"message":"Which reactions does Reactome already have?","selectedDbIds":[]}'
# event: tool data: {"name":"find_existing_reactome","input":{}} what it is doing
# event: text data: {"delta":"Two of the draft reactions..."} the reply, in pieces
# event: proposal data: {"id":"p-001","summary":[...],"evidence":[...]} an edit awaiting your decision
# event: error data: {"message":"..."} a problem inside the turn
# event: done data: {"proposalIds":["p-001"],"usage":{"calls":2,"input_tokens":5878,"output_tokens":48,...}}
# ends every turn that ran, with its tokens
# The assistant only PROPOSES edits. You decide:
curl -s $API/sessions/SESSION_ID/proposals?status=pending -H "$AUTH"
curl -s -X POST $API/sessions/SESSION_ID/proposals/p-001/accept -H "$AUTH" # applies it
curl -s -X POST $API/sessions/SESSION_ID/proposals/p-001/reject -H "$AUTH"
# After accepting edits, export again to get the updated instancesA message refused up front (empty, or over 4,000 characters) gets a single error event and no done.
An edit that changes what a reaction is (its inputs, outputs, catalyst or regulators) must carry a quote that is found in the paper, or be marked as a curator assertion; a quote that is not in the paper is refused.
Every route is under /api/llm and needs the bearer token.
| Method and path | Purpose |
|---|---|
GET /whoami |
Who the token belongs to. |
POST /sessions |
Start from a PMID: {"pmid": "...", "focus": "GENE"} → 202 {sessionId, jobId}. |
POST /sessions/upload |
Start from a PDF (multipart file, optional focus) → 202. |
GET /sessions |
Your sessions. |
GET /sessions/{id} |
One session: status, progress, reactions, counts. |
GET /jobs/{jobId} |
Progress of the background job. |
GET /sessions/{id}/issues |
Everything needing attention (?status=open|resolved|dismissed). |
PATCH /sessions/{id}/issues/{issueId} |
{"status": "resolved"|"dismissed"|"open"}. |
GET /sessions/{id}/export |
The draft as staged-instances JSON. |
GET /sessions/{id}/instances/{dbId}/evidence |
The quotes behind one instance. |
GET /sessions/{id}/reactions/{key} |
One reaction with its participants and evidence. |
GET /sessions/{id}/paper/search |
Search the paper (?q=, §ion=, &limit=). |
GET /sessions/{id}/usage |
Language-model tokens spent, per step (extraction, merge, review, draft, reaction checks, chat) with totals. |
GET /sessions/{id}/network |
The reactions and their entities as nodes and edges, for drawing a network. |
GET /sessions/{id}/existing, POST /sessions/{id}/existing/check |
Draft reactions Reactome already seems to have; recheck. |
POST /sessions/{id}/qa/{key} |
Check one reaction (?llm=false for the rule checks only). |
GET, POST /sessions/{id}/proposals |
List, or submit your own edit as an RFC 6902 patch. |
POST /sessions/{id}/proposals/{pid}/accept , /reject |
Decide a proposed edit. |
GET, POST /sessions/{id}/chat |
Read the conversation; send a message (streams). |
The machine-readable contract is docs/openapi.json; regenerate it after changing the API with
python scripts/export_openapi.py.
Extraction and merging take 10-15 minutes of LLM calls. Every successful run is saved under data/snapshots/, and
with SNAPSHOT_MODE=replay the service serves a saved result instead of running the pipeline, so a ready session
appears in about a second:
How to use replay mode
-
Record once. Annotate the paper normally (the default
SNAPSHOT_MODE=record, see Example: annotate a paper). When the job isdone, the result is saved asdata/snapshots/pmid-<PMID>__<FOCUS>.json(orpdf-<hash>__<FOCUS>.jsonfor an upload). Check withls data/snapshots/. -
Start the service in replay mode, by the environment or in
.env:SNAPSHOT_MODE=replay uvicorn curator_llm.main:app --port 8000 # or put SNAPSHOT_MODE=replay in .env and start uvicorn as usual -
Make the same start request as before: the same PMID (or the same PDF file) and the same
focus. The job finishes in about a second, and its progress readsusing the saved result for <key> (replay mode, no models called). Issues, export, evidence, chat and the other routes then work as on a real session.
The service still needs curator-tool-ws for authentication, and the chat and per-reaction review still call the model.
To go back to real runs, restart without SNAPSHOT_MODE=replay (or set SNAPSHOT_MODE=off to also stop saving).
A replayed session shows the token usage of the run that was saved (flagged saved: nothing was spent on it now),
and anything you do in it afterwards, such as chat, is counted as spent now. GET /sessions/{id}/usage gives both.
A snapshot is found by the paper (PMID, or the PDF's content) and the focus gene, so a different focus, or none, is a
different snapshot. If there is none (or the file cannot be read), the pipeline runs as usual and the result is saved
for next time. Snapshots are not refreshed when prompts or code improve: delete the
file to extract again. scripts/make_snapshot.py builds one from files an earlier run left behind, without any model
call (see the script's header; --usage extraction=LOG --usage merge=LOG records those steps' token usage from their logs). Chat and the per-reaction review still call the model.
- One worker. Sessions (unless
SESSION_STORE=mongo) and background jobs live in the process, so do not start several workers (--workers). Restarting the service ends any job still running. WithSESSION_STORE=mongofinished sessions survive a restart, but one that was still running when the service stopped keeps showing "running" (its job is gone, and nothing marks it failed): start that paper again. - Behind a reverse proxy, do not let it buffer responses for the chat route, or the live steps arrive in one lump
(the service already sends
X-Accel-Buffering: nofor nginx), and allow long-running requests. - Browsers may only call it from the origins in
CORS_ORIGINS. - Keep it on the same network as curator-tool-ws; every call depends on ws being reachable and fails closed if it is not.
- Use a process manager (systemd, Docker) so it restarts if it stops.
pytest # unit and API tests, no services or models neededThe suite uses fakes for ws, Neo4j and the models. A few tests run against the real thing and skip themselves when it
is not available: the Neo4j graph (tests/test_neo4j_lookup.py, tests/test_existing_events.py), MongoDB
(tests/test_sessions_store.py, which uses a throwaway collection) and a running curator-tool-ws with WS_USERNAME /
WS_PASSWORD in .env (tests/test_ws_contract.py).
Each gene ends with a RESULT block and a batch summary:
======================================================================
RESULT — SHANK3
======================================================================
Gene type : ANNOTATED (has-data) — already has 7 curated Reactome pathway(s).
Placement : gate PASS — confident placement under "Neurexins and neuroligins".
Outcome : 48 reaction(s) drafted; QA verdict FAIL (qa_score 0.41).
Status : DRAFT -> needs curator review before acceptance (see the QA flags above).
Cost/time : ~$6.45 est (list price) · 1250k in + 180k out tokens · 30.4 min
- Gene type / Placement — see Key Details.
- Status — always a draft; the QA flags above it are the fix-list.
- Cost/time — combined token spend (retrieval + convert + QA and the full-text extraction subprocesses) and wall-clock. It's a list-price estimate (Sonnet-4.6 $3/$15 per Mtok, ignores cache discounts), so treat it as an upper-ish bound.
The artifacts are results/<gene>_curator_instances.json (the draft) and
results/<gene>_qa_report.json (score + flagged_instances).
Tool 4 turns resolved papers into reactions. For each paper, fulltext_extractor.py drives three
subprocesses in order:
run_extraction.py— isolates the paper's Results section (deterministic heading match, LLM fallback for PDFs; exact JATS (Journal Article Tag Suite) tags for PMC XML), then extracts reactions chunk by chunk with a LangGraph memory window (2 previous + 1 next chunk) so reactions spanning chunk boundaries are still captured.run_merge.py— a four-stage semantic merge (pairwise LLM judging → conflict-aware clustering → whole-cluster validation → subsection consolidation) that deduplicates reactions within a paper.run_review.py— a second-opinion quality read from a different model (OpenAI), with an automatic missed-reaction sweep when the score is below threshold.
Resolution sources (Tool 3): a curator's local PDF (matched by DOI→PMID), PMC JATS XML (fetched + cached for any PMID PMC serves), or a miss (falls back to mining the abstract, labeled so a curator can weight it below full text).
Supporting modules: PubMedFetcher (PMID/PMCID → JATS or local PDF), FullTextPDFSections
(Results-section isolation), FullTextPDFPrompts (prompt templates), abstract_extractor
(abstract fallback).
run_curator.py # ⭐ entry point — the 3-agent pipeline
reaction_to_instances.py # convert step: reactions → Reactome data-model instances
fulltext_extractor.py # drives the full-text subprocess pipeline (Tool 4)
run_extraction.py # stage 1 — chunk-by-chunk reaction extraction
run_merge.py # stage 2 — four-stage semantic merge/dedup
run_review.py # stage 3 — cross-model (OpenAI) quality review
abstract_extractor.py # abstract fallback when no full text is available
TextEmbedder.py # cross-encoder embeddings for retrieval rerank
reactome_llm/
ReactomeCurator.py # Agent 1 — deterministic 4-tool orchestration
ReactomeReviewer.py # Agent 2 — sufficiency decision + proposed adjustment
ReactomeQA.py # Agent 3 — build instances, validate, per-instance verdicts
ReactomePathwayPlacement.py # Tool 1 — interaction-partner pathway placement
ReactomeLiteratureExtractor.py # Tool 2 — retrieval (union query → rerank → LLM judge)
ReactomeFullTextResolver.py # Tool 3 — PMID → PDF / PMC XML / miss
ReactomeFullTextAnalyzer.py # Tool 4 — reactions from full text (wraps fulltext_extractor)
GenePathwayAnnotator.py # gene→accession + embedding/enrichment helpers
QueryBuilder.py # builds the retrieval queries + rerank targets
CuratorRubric.py # the LLM curator-judge rubric (paper selection)
ProteinProteinInteractionsLoader.py # functional-interaction partners (Mongo + BioGRID/IntAct)
ReactomeUtils.py # placement enrichment (binomial test + FDR)
FullTextResolver.py # PDF→DOI→PMID index; PMC XML fetch/cache
FullTextPDFSections.py # Results-section isolation (heuristic + LLM fallback)
FullTextPDFPrompts.py # full-text extraction/merge prompt templates
PubMedFetcher.py # NCBI E-utilities: abstracts, JATS full text, ID conversion
ReactomeModels.py # pydantic schemas (incl. QAReport + per-instance verdicts)
ReactomeNeo4jUtils.py # Reactome graph queries
ModelConfig.py # model settings (claude-sonnet-4-6)
ReactomePrompts.py, ReactomePubMed.py, ReactomeLLMErrors.py,
token_profiler.py, logging_config.py # infra
curator_llm/ # the REST service (see "Running the REST service")
api/ # FastAPI app: routes, auth dependency
services/ # evidence, draft, resolvers, emitter, sessions, chat, QA, snapshots
adapters/ # Neo4j, UniProt, OLS, ws, Mongo, Anthropic, job runner
ports/, models/ # interfaces and pydantic models
main.py # production wiring: uvicorn curator_llm.main:app
tests/ # pytest suite for the service (fakes; a few live tests skip themselves)
scripts/ # export_openapi.py, make_snapshot.py
docs/ # openapi.json (API contract), implementation-plan.md (design and status)
resources/ # schema, pathway↔gene map, interaction data (see above)
results/ # all run outputs (gitignored)
data/ # local caches: PDFs, PMC XML, abstracts, snapshots, uploads (gitignored)
- Very large annotations may still need batched conversion. The reactions→instances step is one LLM call capped at 64K output tokens (Sonnet-4.6's max). That fits genes up to ~50 reactions; a larger gene would need the conversion batched (convert N reactions at a time and stitch). The cap was raised from 32K→64K after SHANK3 truncated at 32K.
- No cross-paper reaction dedup. Merge deduplicates within a paper; the same reaction reported in multiple papers can appear more than once in the pooled set.
- Reaction-level duplicate detection vs. the graph is pathway-name-based. The Neo4j merge check matches whole pathways by name; it does not yet detect that an individual extracted reaction already exists in Reactome.
- A local PDF with no page-1 DOI can't be identified and is skipped.
Reference code only — not part of the live pipeline and not imported by it. These scripts are the investigation into whether a dense, embedding-based approach could improve on the current lexical retrieval and on matching extracted reactions to curated Reactome reactions:
cosine_retrieval_test.py,embedding_benchmark.py,cosine_similarity_score.py— measured the ceiling of embedding-only similarity and showed why it falls short (two reactions differing only in substrate score within ~0.02 of each other, so a wrong match can outrank the right one).pool_match.py+ReactionMatcher.py— the resulting "filter-then-rank" idea: a deterministic structural candidate pool (shared input/output/catalyst) → cross-encoder ranking → an LLM that can also answer "none".ReactomeFullCacheEmbeddingTest.py,check_sections.py— a bi-encoder positive-control eval and a chunk-preview diagnostic.
The written findings (dense_retrieval_investigation.md) are kept with the run outputs, outside
version control. These scripts are preserved as a record, not a maintained entry point — running them
as-is would need import fixes (they expect repo-root modules like TextEmbedder).