A production-ready, multi-agent AI assistant with RAG, long-term memory, tool use, guardrails, and real-time streaming — built from scratch with LangChain & LangGraph.
Features · Try It · Architecture · Run Locally · Tech Stack
No setup required — runs in your browser. Bring your own API key or use the default provider.
A Supervisor → Specialist architecture where a central supervisor analyzes your query and delegates it to the best-fit specialist agent:
- Researcher — handles factual questions using web search, Wikipedia, and RAG search
- Coder — writes, explains, and executes code in a sandboxed environment
- Generalist — handles casual conversation and math using a calculator tool
The supervisor observes each specialist's output and can re-delegate or chain multiple agents together for complex queries.
Upload your own documents (PDF, TXT, DOCX, Markdown) and the assistant uses them as a private knowledge base. Documents are chunked, embedded, and stored in ChromaDB for semantic retrieval.
The assistant remembers across conversations using three complementary systems:
- Short-term — recent conversation messages in a sliding window
- Long-term — LLM-powered fact extraction stored in ChromaDB (e.g., "User's name is Tony")
- Episodic — interaction patterns and session history tracked in SQLite
Input and output validation that protects against prompt injection attacks, toxic content, and spam. All triggered flags are visible as colored badges in the chat UI.
Responses stream token-by-token via Server-Sent Events, with live tool-use indicators and think-tag parsing for chain-of-thought models.
Switch between Ollama (local), Groq, Google Gemini, or any OpenAI-compatible API. If your primary provider is unavailable, the router automatically falls back to the next one.
- MCP Integration — connect external tool servers via the Model Context Protocol
- LLM-as-Judge Evaluation — optional quality scoring (relevance, helpfulness, accuracy) on every response
- JWT Auth & Rate Limiting — registration, login, role-based access, and per-endpoint rate limits
- LangSmith Observability — full tracing of every LLM call, agent step, and tool invocation
Open the Live Demo and try these queries to explore different capabilities:
Ask different types of questions and watch the supervisor route to the right specialist:
"What is the capital of France?" → Generalist
"Search the web for the latest AI news" → Researcher (web search)
"Write a Python function to reverse a string" → Coder
"Calculate 15% tip on a $85 bill" → Generalist (calculator)
Tell the assistant something personal, then ask about it in a later message:
"My name is Tony and I study Computer Science"
... later ...
"What do you know about me?"
Navigate to the Documents page, upload a PDF or text file, then ask questions about its content in the chat.
In the sidebar under Model Selection, choose custom and enter your own OpenAI-compatible API key and model name. This lets you test the full system with GPT-4o, Claude, or any provider that exposes an OpenAI-compatible endpoint — no server-side configuration needed.
After every response, look at the badge row beneath the answer:
- Agent trace — which specialist handled the query (e.g.,
researcher → supervisor) - Tools — which tools were called (e.g.,
web_search,calculator) - Memory — whether stored facts were injected into the prompt
- Time — total processing time in milliseconds
- Score — quality evaluation score (toggle "Answer Quality Scoring" in the sidebar to enable)
A user message flows through multiple layers before a response is streamed back:
flowchart TD
User(["User"]) -->|message| UI["Streamlit UI\nChat · Docs · Memory · MCP"]
UI -->|HTTP / SSE| API["FastAPI Backend"]
API --> Guards["Input Guardrails\nInjection · Toxicity · Spam"]
Guards -->|safe| Router["LLM Router\n(fallback chain)"]
Guards -->|blocked| Blocked["Blocked Response"]
Router --> Ollama["Ollama (local)"]
Router --> Groq["Groq (cloud)"]
Router --> Google["Google Gemini"]
Router --> Custom["Custom API"]
Router -->|selected LLM| Graph["Multi-Agent Graph\n(LangGraph StateGraph)"]
Graph --> Supervisor{"Supervisor"}
Supervisor -->|factual query| Researcher["Researcher Agent\nWeb Search · Wikipedia · RAG"]
Supervisor -->|code query| Coder["Coder Agent\nCode Executor · Calculator"]
Supervisor -->|general query| Generalist["Generalist Agent\nCalculator"]
Researcher --> Supervisor
Coder --> Supervisor
Generalist --> Supervisor
Supervisor -->|FINISH| Output
Output["Output Guardrails"] --> Eval["LLM-as-Judge\n(optional scoring)"]
Eval --> Memory["Memory Manager"]
Memory --> ShortTerm[("Short-Term\nConversation Buffer")]
Memory --> LongTerm[("Long-Term Facts\nChromaDB")]
Memory --> Episodic[("Episodic History\nSQLite")]
Eval -->|SSE stream| UI
Key design decisions:
- Guardrails run before and after the LLM — input is validated before any model call, and output is sanitized before delivery
- The supervisor can chain agents — e.g., researcher fetches data, then coder processes it, all within a single query
- Memory enrichment happens at prompt time — relevant facts and conversation history are injected into the LLM context before generation
- Streaming is end-to-end — tokens flow from LangGraph events through SSE directly to the UI with zero buffering
# Clone
git clone https://github.com/tonynagyy/ai-knowledge-assistant.git
cd ai-knowledge-assistant
# Setup
python -m venv .venv && .\.venv\Scripts\activate # Windows
# source .venv/bin/activate # macOS/Linux
pip install -r requirements.txt
# Configure — set PRIMARY_LLM and your API key in .env
cp .env.example .env
# Run (two separate terminals)
python -m uvicorn app.main:app --reload --port 8000 # Backend
python -m streamlit run ui/chat_app.py # FrontendOpen localhost:8501 for the UI, or localhost:8000/docs for the interactive API docs.
| Backend | FastAPI · Uvicorn · Pydantic |
| AI | LangChain · LangGraph · ChromaDB |
| LLMs | Ollama · Groq · Google Gemini · OpenAI-compatible |
| Frontend | Streamlit |
| Auth | JWT · bcrypt · SlowAPI |
| Infra | Docker · GitHub Actions · Azure Container Apps |
| Observability | LangSmith |
Built with LangChain, LangGraph, FastAPI & Streamlit