Codebase Intelligence Copilot

Ask a code repository questions in plain English — and get answers grounded in the actual code, with exact file:line citations.

$ cci ask ./my-service "how are auth tokens validated on each request?"

Each request passes through require_auth, which reads the Bearer token from the
Authorization header and calls verify_token; verify_token decodes the JWT and
checks its signature and expiry, rejecting the request on any failure.
(src/middleware.py:31-48, src/auth.py:52-70)

Sources (★ = cited in the answer):
  ★ src/middleware.py:31-48  require_auth
  ★ src/auth.py:52-70        verify_token
    src/auth.py:12-20        decode_jwt

---

Contents

• What is this? (in plain English)
• Who is this for?
• The idea in one minute
• Requirements
• Install
• Choose your AI model
• Quickstart
• Commands (full reference)
• Configuration reference
• How it works (under the hood)
• Evaluation & results
• Architecture
• Performance
• Privacy
• Limitations
• FAQ & troubleshooting
• Development
• License

---

What is this? (in plain English)

Imagine a new teammate who has read your entire codebase and never forgets a line. You can ask them "where do we check passwords?" or "how does a request get from the browser to the database?" — and they answer in plain language and point you to the exact file and line numbers, so you can jump straight there.

That's what this tool is. You point it at a folder of code, ask a question in normal English, and it answers using your code — not a guess, not a generic StackOverflow answer — with clickable-style file:line citations. If your code doesn't contain the answer, it says so instead of making something up.

It runs on your machine and works with whatever AI model you choose: a paid one (OpenAI, Claude, Gemini) or a free model running on your own computer.

Who is this for?

• Developers joining a new project who need to get oriented fast.
• Anyone exploring a large, old, or unfamiliar codebase (including your own from six months ago).
• Builders who want to drop code-question-answering into their own tool — the pieces are small, documented, and importable.

You do not need to be an AI expert. If you can run three commands in a terminal, you can use this. If you are an AI/RAG person, everything is here too — every command, flag, the architecture, and honest eval numbers.

The idea in one minute

Why not just paste your code into ChatGPT? Because a real repo is far too big to fit, it costs a lot per question, and the AI can't tell you the exact line — it just read a giant blob. And asking an AI "from memory" gets you a plausible answer about code it never actually saw (this is called a hallucination).

This tool uses a technique called RAG — think of it as an open-book exam for the AI:

1. First it finds the few pieces of your code that are actually relevant to your question.
2. Then it asks the AI to answer using only those pieces, and to cite where each fact came from.

The AI never has to memorize your codebase — it just reads the right page at the right time. That keeps answers cheap, accurate, and checkable.

Tiny glossary (optional — click to expand)

• RAG (Retrieval-Augmented Generation): the open-book-exam approach above — retrieve relevant text first, then generate an answer from it.
• Embedding: a way of turning a piece of text into a list of numbers so a computer can find similar-in-meaning pieces, not just exact word matches.
• Chunking: splitting your files into pieces before indexing. This tool splits at function/class boundaries so each piece is a complete, meaningful unit.
• Index: a prepared, searchable copy of your code (built once, reused for every question). Stored in a small local folder you can delete anytime.
• file:line citation: the source of a claim, e.g. src/auth.py:52-70 — the file and the line range the answer came from.

Requirements

• Python 3.11 or newer.
• About 300 MB of disk for a small embedding model that downloads automatically on first use (after that it works offline).
• An AI model to answer questions — your choice (see Choose your AI model). You only need this for the ask command; search works without any model.
• No GPU required. Everything runs on a normal CPU.

Install

git clone https://github.com/jafeeri/codebase-intelligence-copilot.git
cd codebase-intelligence-copilot

python -m venv .venv
# Windows:        .venv\Scripts\activate
# macOS / Linux:  source .venv/bin/activate

pip install -r requirements.txt

You run the tool as python -m copilot <command>. In this README we write it as cci for brevity — if you'd like that short name, add an alias:

# macOS / Linux (add to ~/.bashrc or ~/.zshrc):
alias cci="python -m copilot"
# Windows PowerShell (add to your $PROFILE):
function cci { python -m copilot @args }

Choose your AI model

The model that writes the answers is entirely your choice — the tool treats all of them the same way. Pick whichever suits your budget and privacy needs, then either create a .env file (copy .env.example) or pass the values as flags.

You want…	CCI_PROVIDER	CCI_MODEL (example)	CCI_BASE_URL	Key
OpenAI	openai	gpt-4o-mini	https://api.openai.com/v1	OPENAI_API_KEY
Anthropic (Claude)	anthropic	claude-sonnet-5	(default)	ANTHROPIC_API_KEY
Google Gemini	openai	gemini-2.0-flash	https://generativelanguage.googleapis.com/v1beta/openai	Google AI Studio key
A free local model (Ollama, LM Studio, …)	openai	llama3.1	http://localhost:11434/v1	(none needed)

Note: "openai" here means any service that speaks the OpenAI API format — that includes Gemini, Groq, Together, and local servers, not just OpenAI itself. Only Claude uses the separate anthropic provider.

Copy the template and edit one block:

cp .env.example .env      # then open .env and uncomment ONE provider

Keys are read from CCI_API_KEY (or OPENAI_API_KEY / ANTHROPIC_API_KEY) and are never printed or logged. Local servers need no key at all.

Quickstart

# 1. Build the index for a repo (creates a small .cci-index/ folder inside it)
python -m copilot index /path/to/your/repo

# 2. Search — find the most relevant code, ranked and cited. No AI model needed.
python -m copilot search /path/to/your/repo "where is the request routed?"

# 3. Ask — get a written, cited answer. Needs a model (see the section above).
cp .env.example .env          # configure a model once
python -m copilot ask /path/to/your/repo "how does password checking work?"

That's the whole loop: index once, then search or ask as much as you like. Re-run index (or use --watch) when the code changes.

Commands (full reference)

index — build or update the searchable index

python -m copilot index <repo> [--index-dir DIR] [--watch] [--interval SECONDS]

Flag	Meaning
--index-dir DIR	Where to store the index. Default: <repo>/.cci-index/.
--watch	Stay running and re-index automatically whenever a file changes.
--interval SECONDS	How often --watch checks for changes (default: 2).

Re-indexing is incremental — only files whose contents changed are re-processed, so updates are fast. Binary files and unreadable files are skipped automatically. Add .cci-index/ to your .gitignore.

search — retrieve the most relevant code (no AI model)

python -m copilot search <repo> "<query>" [-k N] [--index-dir DIR] [--rerank]

Flag	Meaning
-k N	How many results to return (default: 5).
--rerank	Add a second-pass reranker. Off by default — see Evaluation for why.

Great for a quick "where is X?" without spending any tokens. Results are ranked (#1, #2, …) and show each chunk's file:line and symbol name.

ask — get a grounded, cited answer

python -m copilot ask <repo> "<question>" [-k N] [--index-dir DIR] \
    [--model NAME] [--provider openai|anthropic] [--base-url URL]

Flag	Meaning
-k N	How many code chunks to give the model as context (default: 5).
--model / --provider / --base-url	Override the model for this one call (otherwise read from .env/environment).

The answer is written using only the retrieved code, every claim is cited as file:line, and if the code doesn't contain the answer you get Not in the retrieved code. A Sources list shows which chunks were used (★ = cited in the text).

eval — measure quality on a question set

python -m copilot eval <repo> --golden FILE.jsonl [-k N] [--index-dir DIR] \
    [--model NAME --provider … --base-url …]

Measures retrieval (did it fetch the right code?) and, if you pass a model, generation (is the answer faithful to the code? does it correctly say "I don't know" when it should?). See golden/example.jsonl for the file format. Details in Evaluation.

Configuration reference

Settings are read from (highest priority first): command-line flags → real environment variables → a .env file in the current directory.

Variable	Purpose	Example
CCI_PROVIDER	openai (any OpenAI-compatible API) or anthropic (Claude)	openai
CCI_MODEL	The model id to use	gpt-4o-mini
CCI_BASE_URL	The API endpoint	https://api.openai.com/v1
CCI_API_KEY	API key (falls back to OPENAI_API_KEY / ANTHROPIC_API_KEY)	sk-…

How it works (under the hood)

Two phases: an offline index you build occasionally, and an online query path you hit per question.

 ═══ INDEX  (offline — on change / --watch) ══════════════════════════════
 repo files ─▶ AST-chunk ─▶ embed (local) ─▶ store
              tree-sitter    code model      vectors + BM25 + metadata
              one chunk per                  (a small local index folder)
              function/class

 ═══ QUERY  (online — per question) ══════════════════════════════════════
 question ─▶ embed ─┬─ dense top-50 ┐
                    └─ BM25  top-50 ┴─▶ RRF fuse ─▶ [rerank] ─▶ top-k
                                                                  │
                    grounded prompt ◀─────────────────────────────┘
                    "answer ONLY from this code, cite file:line,
                     say 'not in the retrieved code' if unknown"
                                    │
                                    ▼
                            your chosen LLM ─▶ answer + citations

Three ideas do the heavy lifting:

1. Chunk by structure, not by size. Splitting code every N characters is the #1 reason naive code search fails — it cuts functions in half. This tool parses each file into a syntax tree (tree-sitter) and splits on real function / class boundaries, so every chunk is complete and carries its own file:line — which becomes the citation.
2. Two kinds of search, combined. Semantic search (embeddings) is great at meaning but fuzzy on exact names; keyword search (BM25) nails exact identifiers like validate_token. Running both and merging the results (reciprocal rank fusion) gives you both strengths.
3. A grounding contract. The prompt tells the model to answer only from the retrieved code, cite every claim, and admit when it doesn't know — turning "please don't hallucinate" from a hope into an enforced, measurable behavior.

Supported languages: Python, JavaScript, TypeScript, Rust. Adding one is a small, isolated change (a tree-sitter grammar + its declaration node types).

Evaluation & results

A wrong answer fails for one of two separable reasons — the search never found the right code, or the model had it and answered badly — so the harness measures them separately against a golden set of (question, correct file:line) pairs. (Format in golden/example.jsonl.)

On a sample repository (13 files, 229 chunks), recall@5 (how often the correct chunk was in the top 5) by search mode:

mode	recall@5	notes
semantic only	0.85	good, but misses exact-identifier queries
semantic + keyword (default)	0.92	best — keyword search rescues exact names
+ cross-encoder reranker	0.62	worse on code — see below

The default is semantic + keyword. A general-purpose reranker actually reduced accuracy here because it prefers prose (comments, tests, docs) over the real implementing function — so reranking is opt-in (--rerank), left in for anyone who plugs in a code-tuned reranker. This is exactly what the eval is for: change defaults based on measurement, not vibes.

Run it yourself:

python -m copilot eval /path/to/repo --golden golden/example.jsonl
# add --model … to also measure answer faithfulness and abstention

Architecture

Component	File	Responsibility
AST chunker	copilot/chunker.py	tree-sitter parse → function-granular chunks + file:line metadata
Embedder	copilot/embedder.py	local code-search embeddings (loaded lazily)
Index store	copilot/index.py	walk → chunk → embed → persist; incremental by content hash
Hybrid retriever	copilot/retrieve.py	semantic + BM25 (stdlib Okapi) → RRF → optional rerank
Grounded generator	copilot/answer.py	grounding prompt + provider-agnostic LLM call (stdlib urllib)
Eval harness	copilot/eval.py	recall@k, MRR, faithfulness (LLM-as-judge)
CLI	copilot/cli.py	index · search · ask · eval

The on-disk index is three files: chunks.jsonl (metadata + text), embeddings.npy (vectors), and manifest.json (repo, model, file hashes for incremental rebuilds).

Performance

• Answering/searching is fast once loaded — retrieval over a repo-scale index is a few milliseconds.
• First command in a session takes ~15 seconds to load the local embedding model. Each command is a fresh process, so you pay this once per run; for heavy interactive use keep a session open. (A persistent server mode is a natural future addition.)
• Tiny footprint — a typical repo's index is a few hundred KB.
• Designed for repositories, not billion-vector corpora — the search is a straightforward in-memory scan, which is plenty fast at that scale.

Privacy

With a local model (Ollama, LM Studio, …), your code and questions never leave your machine. With a cloud model (OpenAI, Claude, Gemini, …), the retrieved code snippets are sent to that provider's API to generate the answer — exactly like any cloud AI tool. Pick the option that matches your data-handling requirements. The local code-embedding model always runs on your machine regardless of which answer-model you choose.

Limitations

• Retrieval isn't perfect (~0.92 recall@5 on the sample). A vaguely-worded question, or a very generically-named function, can occasionally miss.
• Answer quality tracks the model you choose. Small local models are conservative and may over-refuse or skip inline citations; larger local models or a good cloud model answer more fully. The retrieval feeds the right code either way.
• Grounding is strong but probabilistic. The contract greatly reduces hallucination and resists instructions hidden inside the codebase, but it is a prompt, not a proof — which is why the eval measures faithfulness.
• Languages: Python, JavaScript, TypeScript, Rust today.
• Scale: single machine, in-memory search — ideal for repositories; very large multi-repo corpora would need a dedicated vector database.

FAQ & troubleshooting

Do I need an API key / to pay? No. Run a free local model (e.g. Ollama) and you never pay or send code anywhere. A paid cloud model is just one option.

Do I need a GPU? No — everything runs on a normal CPU.

Which model should I pick? For the best answers with no setup fuss, a cloud model (OpenAI/Claude/Gemini) is easiest. For zero cost and full privacy, a local model via Ollama. The tool works the same either way; bigger models give better, more consistent answers with cleaner citations.

ask says Not in the retrieved code. but I know the code has it. Two possibilities: (1) the relevant code wasn't retrieved — try rephrasing, or raise -k (e.g. -k 8); (2) a small local model is being over-cautious — try a larger or cloud model. Use search to confirm whether the code was found at all.

"connection failed" / the model errors. Check your CCI_BASE_URL and that the server is running. For a local model, make sure it's started (e.g. ollama serve) and you've pulled the model (ollama pull llama3.1). For a cloud model, check the API key.

The first command is slow. That's the one-time embedding-model load (~15 s) and, on the very first run ever, its ~300 MB download. Subsequent steps in the same run are fast.

How do I re-index after changing code? Run index again (it only re-processes changed files), or run index --watch to keep it updated automatically.

Where is my data stored? How do I remove it? In <repo>/.cci-index/. Delete that folder to remove the index; nothing else is written.

Development

Every non-trivial module ships with a runnable, offline self-check (fake embedder / fake LLM — no models, no network, no key):

python -m copilot.selfcheck     # runs every module's checks; this is the CI gate

Contributions welcome — adding a language is a good first PR (a tree-sitter grammar in chunker.py).

License

MIT © Ali Mehdi Jafeeri. See LICENSE.