AI-powered support-ticket classification and resolution, built on LangGraph.
AutoTriage ingests a support ticket, classifies it (category / urgency / complexity / confidence), routes it, retrieves grounding articles from a vector knowledge base, generates a grounded response, and runs guardrails before delivery — with a supervisor agent that can send the ticket back for re-classification or another retrieval pass. A FastAPI admin panel exposes ticket ingest, a live streaming chat, KB browsing, dataset seeding, and LLM-call analytics.
The repository runs three parallel triage pipelines over different datasets — bitext, tobi, and kaludi — each with its own agents, prompts, and per-project Mongo KB collection.
Blue = API / ingress · green = LangGraph pipeline nodes · purple = model layer · grey = data & vector stores · orange = observability · red = human escalation. Solid edges are graph control flow; dashed edges are LLM / embedding calls and best-effort logging / tracing. Source: docs/architecture.dot (dot -Tpng -Gdpi=160 docs/architecture.dot -o docs/architecture.png); component table in docs/architecture.md.
Each ticket flows through a LangGraph state machine with an agentic-RAG retrieval stage and a supervisor correction loop:
ingest → classify ──(route)──┐
├─ human_only → escalate → END
└─ else → retrieve_kb → supervisor ──┐
├─ ready → resolve → guardrails → END
├─ reclassify → classify (loop)
├─ retry_rag → retrieve_kb (loop)
└─ human_only → escalate → END
- classify — task-specialized NVIDIA NIM model returns structured classification;
human_onlyrouting escalates immediately. - retrieve_kb — agentic RAG: an LLM expands the query into multiple variants → batch-embed → parallel category-scoped vector search → rerank → top articles.
- supervisor — reviews the classification + retrieved context and returns
ready,reclassify, orretry_rag; a retry counter breaks the loop at 3 attempts, and any unrecognized/error decision safely escalates to a human. - resolve — generates a grounded response from the retrieved articles.
- guardrails — checks the response before it is marked closed.
- escalate — routes the ticket to the human queue.
State is a TicketState TypedDict; every node persists progress to PostgreSQL.
| Concern | Backend |
|---|---|
| Classify / resolve / supervise / query-expand | NVIDIA NIM (OpenAI-compatible, task-routed models) |
| Reranking | NVIDIA NIM |
| Live chat streaming (SSE) | Gemini primary, NVIDIA NIM fallback on quota/error |
| Embeddings | nomic-embed-text-v1.5 via an external OpenAI-compatible endpoint |
| Legacy accuracy harness only | Claude Agent SDK |
- Orchestration: LangGraph (agent graph, state, conditional routing)
- API / UI: FastAPI + Jinja2 templates + Server-Sent Events
- Vector store: MongoDB Atlas native Vector Search (per-project, category-scoped collections)
- Relational store: PostgreSQL (tickets, chat messages, LLM call logs) via async SQLAlchemy
- Cache / queue: Redis
- Observability: self-hosted Langfuse (traces + token usage)
- Lint / format: Ruff
Brings up the app plus MongoDB Atlas Local (native vector search), PostgreSQL, Redis, and the Langfuse stack.
docker compose up --build # app on http://localhost:8000After the stack is up, seed a dataset from the /datasets admin page or via POST /seed/{project}.
uvicorn api.main:app --host 0.0.0.0 --port 8000(Requires reachable MongoDB, PostgreSQL, Redis, and the embedding endpoint — see environment variables below.)
| Route | Purpose |
|---|---|
/ |
Ticket list (optionally filtered by project) |
POST /ingest |
Ingest a ticket (Form or JSON) and run the matching pipeline in the background |
/ticket/{id} |
Ticket detail |
/chat/{session} + /chat/{session}/stream |
Live streaming support chat (SSE) |
/datasets + POST /seed/{project} |
Dataset overview and KB seeding |
/kb/{project} |
Browse a project's knowledge-base articles |
/analytics |
LLM-call analytics (calls, latency, token usage per model) |
Config is loaded from .env (.env is git-ignored). Template: autotriage/.env.example.
# NVIDIA NIM (primary reasoning engine)
NVIDIA_API_KEY=...
NVIDIA_BASE_URL=https://integrate.api.nvidia.com/v1
NVIDIA_MODEL=...
# Gemini (chat streaming)
GEMINI_API_KEY=...
GEMINI_MODEL=...
# MongoDB Atlas (vector store; per-project collections)
MONGODB_URI=mongodb://mongodb:27017/
MONGODB_DB_NAME=autotriage
MONGODB_VECTOR_INDEX_NAME=vector_index
# PostgreSQL (tickets, chat, LLM call logs)
POSTGRES_URI=postgresql://admin:password@postgres:5432/autotriage
# Redis
REDIS_URI=redis://redis:6379/0
# Embeddings (external OpenAI-compatible endpoint)
EMBEDDING_API_KEY=...
EMBEDDING_BASE_URL=http://embedding-srv:8000/v1
EMBEDDING_MODEL=nomic-embed-text-v1.5
# Langfuse (self-hosted tracing + token usage)
LANGFUSE_BASE_URL=http://localhost:3000
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...The stack ships with a self-hosted Langfuse instance for distributed tracing and token-usage tracking.
Dashboard: http://localhost:3000. On first visit, create an admin account (first signup becomes admin), create a project, and copy its public/secret keys into your .env.
Every ticket processed by the active pipelines becomes a Langfuse trace tagged with the project name, and NVIDIA NIM token usage is captured. The existing Postgres LLMCallLog table and the /analytics view remain the in-app logging path; Langfuse is an additive layer. (Dollar cost shows $0 until NVIDIA model prices are registered as Langfuse custom prices — token counts are accurate, dollar pricing is deferred; and Gemini-answered chat turns do not yet carry token usage on their span.)
api/ FastAPI app, templates, SSE chat (the running entry point)
projects/
bitext_triage/ active pipeline + agents + prompts + loader (per dataset)
tobi_triage/
kaludi_triage/
core/
llm_clients/ NVIDIA NIM, Gemini, Claude clients
agents/ supervisor, agentic retriever
kb/ MongoDB Atlas vector store (seed / search)
db/ async PostgreSQL models, CRUD, session
observability/ Langfuse integration
schemas.py Pydantic enums / models
autotriage/ legacy Milestone 1–2 single-dataset accuracy harness (not the live path)
infra/ , docker-compose.yml , Dockerfile
Two codebases: projects/* is the active path that api/main.py imports and runs. autotriage/ is the original single-dataset reference harness used for scored accuracy testing and is not wired into the API.
- Lint / format:
ruff check --fix . && ruff format . - The three
projects/*pipelines are intentional near-duplicates; changes to one usually need mirroring to the other two. - KB collections are per-project and category-scoped — a resolver only sees its own dataset's articles.