Docent helps teams inspect, search, evaluate, and understand large collections of AI-agent runs. It keeps full transcripts, tool calls, metadata, and scores together so you can move from a vague question about agent behavior to reviewable evidence and quantitative comparisons.
Open hosted Docent · Quickstart · Self-hosting · Get support
Docent is in alpha. APIs, schemas, and deployment interfaces may change.
- Browse complete agent runs, multi-transcript traces, messages, reasoning, and tool calls.
- Filter and compare runs using metadata, scores, labels, and charts.
- Ask natural-language questions about transcripts and follow answers back to cited evidence.
- Turn a behavior hypothesis into a rubric, run it across a collection, and spot-check structured results.
- Cluster matching results to identify recurring behaviors, failures, and edge cases.
- Generate action summaries and explore behavior structure with Hodoscope maps.
- Share collections and analysis links with collaborators.
Typical uses include investigating repeated failures, auditing evaluation or reinforcement-learning rollouts, comparing models or agent scaffolds, and finding behaviors that aggregate scores hide.
- Create a collection: a set of runs from one experiment, evaluation suite, or deployment.
- Add agent runs: complete executions containing one or more transcripts and their metadata.
- Describe the behavior you want to find and refine it into a rubric: a decision rule Docent can apply consistently.
- Review the evidence in matching transcripts, including explanations, citations, and human labels.
- Quantify or cluster results to compare steps, models, scaffolds, or other metadata.
Agent traces can contain prompts, model outputs, tool arguments, credentials, or user data. Review and redact runs before uploading them, and only send data permitted by your organization's policy.
- Create an account.
- Create a collection.
- Add data using one of the methods below.
- Open a run to inspect it, or create a rubric to search the collection.
In a collection, choose Add Data → Upload Inspect Log. The uploader accepts Inspect .eval files and Inspect-exported .json files and previews the runs before import.
Install the SDK, create an API key under Settings → API Keys, and expose it to your process:
pip install docent-python
export DOCENT_API_KEY="..."Then initialize tracing before your instrumented model calls:
from docent.trace import initialize_tracing
initialize_tracing("my-agent-runs") # collection nameThis creates or reuses the named collection and exports subsequent calls from supported OpenAI and Anthropic instrumentation. See Tracing for explicit agent-run contexts, metadata, scores, flushing, and the current instrumentation boundary.
import os
from docent import Docent
from docent.data_models import AgentRun, Transcript
from docent.data_models.chat import AssistantMessage, UserMessage
client = Docent(api_key=os.environ["DOCENT_API_KEY"])
collection_id = client.create_collection(name="My agent runs")
agent_run = AgentRun(
transcripts=[
Transcript(
messages=[
UserMessage(content="What is 1 + 1?"),
AssistantMessage(content="2"),
]
)
],
metadata={"model": "example-model"},
)
client.add_agent_runs(collection_id, [agent_run])Continue with the ingestion quickstart or the Python SDK guide to construct and upload AgentRun objects.
Docent supports English and Simplified Chinese (zh-CN). Choose Settings → Language to localize the interface and request the selected language for newly generated chats, rubric explanations, summaries, clusters, and Hodoscope text.
Language changes do not rewrite uploaded transcripts, metadata, quoted source material, or existing analysis results. See Language settings for the exact boundary.
The repository provides one deployment command and one user-edited configuration file. Install Docker Engine with Docker Compose v2, clone the repository, then run:
./deploy.shOn the first run, the script creates .env with mode 600 and stops. Replace every <...> placeholder in that file, review the Web port, model endpoints, credentials, and worker limits, then run ./deploy.sh again.
The script validates the Compose configuration, builds the images, applies Alembic migrations, waits for Postgres, Redis, the API, and the Web UI, then prints the published frontend address. The default application URL is http://localhost:3000; Postgres and Redis bind only to 127.0.0.1.
Stop the stack without deleting its database volume:
docker compose downSee Self-host Docent for local-model networking, upgrades, logs, and manual development.
The optional benchmark harness is isolated from the
Docent application stack. After cloning main, the following command pulls the
prebuilt linux/amd64 images, starts the dedicated DinD and LLM gateway services,
and runs the streaming and forced-tool-use checks:
git clone --branch main --single-branch https://github.com/LvDAO/docent-inner.git && \
cd docent-inner/benchmark-harness && \
export COMPOSE_FILE=compose.yaml:compose.prebuilt.yaml \
HARNESS_IMAGE_TAG=sha-85dd43129df9e287774fb6488049c3da68f2af6b \
LOCAL_LLM_BASE_URL=http://host.docker.internal:8000/v1 \
LOCAL_LLM_MODEL=Qwen/Qwen3-Coder && \
docker compose --profile tools pull && \
docker compose up -d --no-build docker llm-gateway && \
docker compose --profile tools run --rm runner doctorChange the endpoint and model for your local OpenAI-compatible server. This quickstart assumes the endpoint does not require an API key; see the harness README for key, CA, proxy, benchmark, logging, and Docent-import configuration. The target host must permit privileged containers because benchmark tasks run inside a dedicated DinD daemon.
This is the supported development and evaluation path in the current checkout; it is not a production deployment recipe. It runs Postgres and Redis in containers and runs the API, worker, and web application from the repository.
The Web UI and
/restAPI share the Web origin. The backend remains a separate local process, but browsers and SDK clients only need the Web port.
- Python 3.11 or newer
- uv
- Bun
- Docker Engine with Docker Compose
- An LLM endpoint and an embedding endpoint supported by your configuration
cp .env.template .env
uv sync --extra dev
docker compose -f docker-compose-db.yml up -d
uv run alembic upgrade headBefore starting Docent, replace placeholder credentials in .env. The default LLM route is DeepSeek; DOCENT_LLM_PROVIDER=custom selects a generic OpenAI-compatible chat-completions endpoint. Embeddings have their own endpoint, key, model, and dimensionality settings.
LLM-backed analysis sends relevant trace text to the configured LLM endpoint, and embedding jobs send text to the configured embedding endpoint. Select endpoints that meet your privacy, residency, and cost requirements.
See Environment variables and LLM providers and calls before using a custom provider or per-feature model overrides.
Run each command in a separate terminal:
uv run docent_core server --port 8889 --reloaduv run docent_core worker --workers 1uv run docent_core web --port 3001 --backend-url http://localhost:8889Open http://localhost:3001. In another terminal, verify the API with:
curl http://localhost:3001/rest/pingThe API check should return {"status":"ok","message":"pong"} through the same origin. When using SSH port forwarding, forward only port 3001. Stop the three foreground processes with Ctrl+C, then stop the data services without deleting their volume using:
docker compose -f docker-compose-db.yml down- Ingest agent runs
- Trace model calls and agent runs
- Search and cluster behaviors
- Understand Docent data models
- Configure a self-hosted deployment
- Select a language
Repository-specific guidance is kept close to each surface:
The core application consists of a FastAPI server, a Redis-backed worker, a Next.js frontend, PostgreSQL with pgvector, and the editable docent-python SDK.
Docent is licensed under the Apache License 2.0.