codeindex-sync

Keep code indexes in sync with git activity — for any MCP backend.

Commit, merge, rebase or switch branches, and the affected repository is re-indexed in the background within seconds. Nothing to run by hand.

_{Recorded against a stub MCP backend, which is also the point: nothing in this project knows what a backend is. Regenerate with vhs demo/demo.tape.}

Any indexer that speaks MCP works, described entirely in config. SocratiCode is the backend this was built against and ships as a built-in preset, so that setup is four commands:

npm install -g codeindex-sync   # or: brew install mgd43b/taps/codeindex-sync
codeindex-sync init --preset socraticode   # configure a backend
codeindex-sync install                     # install the git hooks
codeindex-sync schedule                    # drain the queue on a timer
codeindex-sync doctor                      # check it all works

One step people miss: backend URLs go in the config's env block, not your shell — git hooks are not a login shell, so exported variables are invisible to the indexer, which then writes somewhere nothing reads. doctor checks the rest, including whether anything is actually draining the queue.

Guides: Using it with SocratiCode · Adding a backend or hook handler

Not affiliated with, or endorsed by, any backend it drives. codeindex-sync contains no third-party backend code — it spawns backends as separate processes and speaks MCP to them. Product names describe compatibility only.

Why this exists

Running a semantic index over a working tree sounds simple. It isn't. Every behaviour here comes from a failure seen in production:

Problem	What happens without this
Bursts	One git rebase fires dozens of hooks; naive tools run dozens of concurrent indexers against one GPU
Throwaway worktrees	A hook fires from a directory that has since been deleted; the indexer dies during interpreter startup with an opaque error
Agent worktrees	Every coding agent checks the repo out under its own directory and commits there. Each copy carries the committed marker file, so the backend builds a second index of the same code — under a path that will not exist in an hour
Redundant work	Worktree churn re-queues the same repo repeatedly, each costing a full tree walk to conclude nothing changed
Crashes	A worker that dies mid-job silently drops that repository forever
Backend contention	The backend's own lock is misread as failure, parking healthy repos in failed/
Hooks aren't a login shell	Config exported in ~/.bashrc is invisible to the indexer, which then writes to the wrong place — silently
Silent index damage	A partial write recorded as a success leaves a file claimed-but-empty forever; nothing re-indexes it, and search just quietly gets worse

Commands

init [--preset <id>]	Create a config from a preset
doctor	Diagnose config, hooks and backend health — every problem carries a remedy
status	Queue, worker and failures
list [repo] [--all] [--stale] [--json]	What the backend holds — one repo, or every index with status, age and file counts
sync [repo] [--full]	Index now, retrying a failure until maxAttempts attempts (the first included) have been made
verify [repo] [--repair]	Compare what an index claims to hold against what it actually holds
drain / once	Process the queue
retry / forget [--all]	Requeue or drop jobs that used up their attempts
unlock [--force]	Release a stale worker lock
providers [--example]	Configured providers and available presets
worktrees [--prune|--gone]	Inspect worktrees; drop dangling registrations or merged ones
cleanup [--apply]	Remove indexes whose directory is gone (dry run by default)
schedule / unschedule	Drain the queue on a timer (launchd or systemd)
claim [repo] [--replace] / unclaim [--remove]	Opt a repo in or out by writing the provider's marker file
install-repo / uninstall-repo	Cover a repo that sets its own core.hooksPath
completion [shell]	Shell completion for bash, zsh or fish
hook <name>	Entry point for git hooks (enqueues; never indexes inline)

Backends

Backend	Status
SocratiCode	Built-in preset — init --preset socraticode
Any other MCP indexer	Config only, no code — init --preset generic-mcp

There is one built-in preset because there is one backend this has been run against in earnest. That is a statement about what has been verified, not about what works: the worker knows nothing about SocratiCode, and the generic path is exercised in CI against stub MCP servers with arbitrary tool names.

The whole contract is one MCP tool that re-indexes a path. Point update at it and you have a working backend:

{
  "providers": [{
    "name": "my-backend",
    "command": "my-index-server",
    "args": ["--stdio"],
    "tools": { "update": "update_index", "index": "rebuild_index" },
    "detectFiles": [".my-backend.json"]
  }]
}

index (full rebuild) and status (chunk counts for list) are optional and improve behaviour where present. detectFiles is how a provider decides a repository is its own; without it, it claims every repository under root.

If you get another backend working, a preset is a few lines of config and a welcome PR — see docs/extending.md for the full field reference, plus how to write a hook handler for something other than indexing.

Design

Everything above provider.ts is backend-agnostic. If a backend's name appears in the worker, the abstraction has leaked.

• Serialised by design. Indexing backends are usually GPU- or network-bound singletons; one job at a time is faster than several.
• Nothing-changed is settled before any work. A (HEAD, dirty) fingerprint catches the common case: a hook fired for worktree activity that never touched this repo. Measured 1.20s → 0.12s, without contacting the backend at all. A dirty tree never counts as unchanged, since two sets of uncommitted edits are indistinguishable.
• Coalescing uses an invariant, not a timer. An index that started at T observed the filesystem as of T, so a job queued before T is already covered — this catches recovered and retried jobs. A real change can never be dropped, because its hook fires only after the file changed.
• busy is not failure. Contention requeues without burning an attempt.
• Worktrees are not projects. A commit in a linked worktree changes no file in the checkout that carries the index, so hooks firing there enqueue nothing. Agent tools are also excluded by path — excludePaths, defaulting to **/.claude/worktrees/** and **/.codex/worktrees/** — because a deleted directory is the normal case by the time a hook runs, and git can answer nothing about one. Neither rule asks "is this the repository root?": a path handed over as /repo/src — a nested project, a manual sync — must come back as /repo/src, neither discarded as a worktree nor widened to the whole repo. (Git fires these hooks at the working-tree root, so a nested project under an unmarked root is still reached by sync, not by a commit; finding it from the root would mean a tree walk on every commit.) Add a directory to excludePaths when a new tool appears; no release needed.
• The log is the diagnostic. Append-only, rotated not truncated, and writing to it never throws. It also carries whatever the backend logs during an index run (MCP notifications/message), as [<provider>:<level>] lines, since a backend's own account of a failure is usually more specific than its reply.
• verify is the one deliberate exception. Every other command drives a backend through the MCP tools it declares. Integrity has no such tool to call — "which files does this index claim, and which have content?" is answerable only from storage — so verify reads the backend's store directly, and is the only part of this project that is not backend-agnostic. It is confined to qdrant.ts and verify.ts, reached only from the verify command — the worker, the queue and the provider interface never touch it.

Releasing

release-please keeps a release PR open with the version bump and changelog it would produce; merging it tags, releases and publishes. Packages are published from CI with provenance, so a tarball can be traced back to the commit and workflow that built it. The Homebrew formula is generated from the published tarball by scripts/update-tap.sh.

See docs/releasing.md.

Requirements

Node >= 22.12. Works on macOS and Linux; CI covers both across Node 22 and 24.

Licence

MIT — see LICENSE.