sprintstart-ai

The AI and RAG pipeline service for SprintStart, an AI-assisted onboarding and knowledge-retrieval platform for software development teams.

Prerequisites

• Python 3.12+
• uv
• Ollama running locally with the required models pulled:

ollama pull llama3.2
ollama pull nomic-embed-text

# Optional — required only for image captioning
ollama pull llava:7b        # lightweight, good for development
# ollama pull qwen2-vl:7b  # recommended for production (better on diagrams/text-heavy images)

Getting Started

Local

# 1. Install dependencies
uv sync

# 2. Configure environment
cp .env.example .env
# Edit .env and fill in the values

# 3. Run the service
uv run python -m src.main

The service runs on port 8000. Interactive docs are available at /docs.

Docker

# 1. Configure environment
cp .env.example .env
# Edit .env and fill in the values

# 2. Start the service
docker-compose up --build

The service runs on port 8000.

OLLAMA_BASE_URL is automatically overridden to http://host.docker.internal:11434 inside the container, so no manual change is needed.

Environment Variables

Variable	Example value	Description
LLM_BACKEND	ollama	LLM backend to use. Currently only ollama is supported.
OLLAMA_BASE_URL	http://localhost:11434	Base URL of the Ollama instance. Use http://host.docker.internal:11434 when running via Docker with Ollama on the host.
OLLAMA_MODEL	gemma4:e4b	Chat model to use for generation.
OLLAMA_EMBED_MODEL	nomic-embed-text:latest	Embedding model to use for ingestion and retrieval.
OLLAMA_VISION_MODEL	llava:7b (dev) / qwen2-vl:7b (prod)	Vision model for image captioning. Optional — if unset, image files are accepted but produce no chunks (chunk_count=0).
CHROMA_PATH	./data/chroma	Path for ChromaDB persistent storage. If unset, an in-memory store is used and data will not persist.
AGENT_DEBUG	0	When set to a truthy value, logs each agent's reasoning step (LLM text and tool calls) to stderr. Disabled by default and for 0/false/no/off/empty.
CHUNK_SIZE	512	Maximum number of characters per chunk
CHUNK_OVERLAP	64	Number of characters reused between consecutive chunks to preserve context when splitting large chunks
CONTEXT_AWARE_CHUNKING_MAX_CHARS	24000	Character-count ceiling (a proxy for a token limit — there is no tokenizer in this project) for text/PDF content sent to the LLM-based context-aware chunker. Above this, ingestion falls back to the plain chunk_text strategy. See Context-aware chunking.

API Endpoints

Method	Path	Description
GET	/api/v1/health	Reports service health including LLM backend status. Returns 503 if Ollama is unreachable.
POST	/api/v1/ingest	Parses, chunks, and embeds a document and stores it in the vector store. Re-ingesting the same artifact_id replaces existing chunks. Supports text files and images (send image content as base64; requires OLLAMA_VISION_MODEL). Text/PDF content can optionally be chunked using the LLM-based context-aware chunker (opt-in, off by default).
POST	/api/v1/chat	Retrieves relevant chunks and streams a generated answer as Server-Sent Events (SSE).
POST	/api/v1/title	Generates a short descriptive title from a user prompt.
POST	/api/v1/onboarding/path	Generates a personalized onboarding path (SSE). Blueprints are passed in by the backend on each request — the service is stateless.
POST	/api/v1/onboarding/blueprints/generate	Batch job: generates source: generated blueprints from the corpus and returns them. The backend owns persistence — it passes its active blueprints in and stores the results.

Chat SSE stream

The /api/v1/chat endpoint streams newline-delimited JSON events:

Event type	Description
tool_use	A capability the orchestrator invoked, in order. Has name and kind (agent for a sub-agent, tool for a leaf tool)
token	A single token fragment of the answer
citation	A source chunk used to generate the answer
done	Signals the end of the stream
error	Emitted on failure instead of the above

Context-aware chunking

POST /api/v1/ingest can optionally chunk text and PDF content using an LLM instead of the plain character-length chunk_text strategy (ingestion/chunker.py). This is opt-in per request via two independently toggleable flags on IngestRequest, both false by default:

Field	Default	Description
semantic_boundaries	false	Let the LLM choose chunk boundaries based on topic shifts / section boundaries instead of accumulating paragraphs up to CHUNK_SIZE.
contextualize	false	Let the LLM selectively prepend a short situating "context block" (Anthropic-style Contextual Retrieval) to chunks that would otherwise lack context; self-contained chunks are left untouched.

Both flags share a single LLM call (see ingestion/context_aware_chunker.py). Leaving both at false (the default) skips the LLM call entirely and behaves exactly like the legacy chunker — existing callers are unaffected unless they explicitly opt in. When either is enabled, the strategy still always falls back to chunk_text — no request ever fails because of it — when:

• the content exceeds CONTEXT_AWARE_CHUNKING_MAX_CHARS,
• the LLM backend is unreachable, or
• the LLM response is missing, malformed, or fails validation (e.g. out-of-range boundaries).

Resulting chunks carry extra metadata so context/overlap portions of a chunk's content are identifiable: has_context_block, context_block_range, has_overlap, overlap_range (character ranges as "start:end" strings, empty when not applicable).

Scope: only text and PDF content are affected; code and image parsing are unchanged. For PDFs, the LLM is invoked per page (pages are already parsed independently), so semantic boundaries and context blocks are page-scoped, not whole-document-scoped. The batch endpoint POST /api/v1/ingest/sync (GitHub run sync) does not expose these flags and always uses the plain chunker — changing that would mean changing the batch wire contract with the backend, which is a separate decision.

AI-proposed onboarding blueprints

Onboarding paths are assembled from versioned blueprints scoped by global or area:<name>. Blueprints carry a source: authored (human-written) or generated (drafted by the AI from the ingested corpus). The backend owns blueprint persistence, versioning, and rollback — the AI service is stateless and only returns generated data.

The generation job analyzes the corpus and returns proposed blueprints for the backend to persist. Every generated step is grounded (cites an ingested chunk), the job is idempotent (a blueprint records the corpus_fingerprint it was drafted from, so an unchanged corpus is a no-op), and human-owned invariants are protected — it may not remove or downgrade a required or invariant: true step; such changes are re-injected and the outcome is escalated, never silently applied.

Blueprint management (trigger generation, list versions, rollback) is exposed through the backend API at POST /api/v1/onboarding/blueprints/generate, GET /api/v1/onboarding/blueprints/{scope}/versions, and POST /api/v1/onboarding/blueprints/{scope}/rollback.

Cost / performance

A full refresh is O(number of scopes), not O(corpus size) or per onboarding request. Per scope it performs:

• 1 hybrid retrieval (one embedding call + a cached in-memory BM25 pass over the corpus), and
• 1 LLM generate call drafting the blueprint from the top ~12 retrieved chunks.

So a refresh of global + N areas is N+1 retrievals and N+1 generate calls — typically single-digit LLM calls for a whole organization. Scopes whose corpus fingerprint is unchanged are skipped entirely (no LLM call). The job is batch and schedulable; the latency-sensitive /onboarding/path request path never invokes generation.

Running Tests

uv run pytest

With coverage:

uv run pytest --cov=src --cov-report=term-missing