Add an optional Neo4j graph read model behind a default-off flag - #6
Open
ShreeBohara wants to merge 1 commit into
Open
Add an optional Neo4j graph read model behind a default-off flag#6ShreeBohara wants to merge 1 commit into
ShreeBohara wants to merge 1 commit into
Conversation
Enables the traversals the SQL path structurally cannot do, without making a new
datastore load-bearing.
WHY
The graph questions that matter for understanding a codebase are transitive: what breaks
if I change this file, how does auth reach the database, are there import cycles. The
Python implementation cannot answer them -- hops is hard-capped at 2 in the API because
each hop rescans the whole edge list, so depth is quadratic in Python and a single
variable-length pattern in Cypher.
DESIGN: PROJECTION, NOT SOURCE OF TRUTH
code_dependencies (SQL) stays authoritative. Neo4j is projected from it during indexing
and can be rebuilt by re-indexing. Consequences, all deliberate:
- neo4j_enabled defaults to False, so nothing changes unless it is turned on.
- get_graph_store() returns None when disabled OR misconfigured (missing password)
rather than raising, so every caller treats "no graph store" as an ordinary path.
- Startup verifies connectivity and applies schema, but a failure only logs -- an
unreachable graph database must not stop the API booting.
- A sync failure during indexing is caught and logged; the SQL edges are unaffected.
- Repo deletion removes the subgraph before the SQL rows, so a failure leaves the
authoritative data intact and retryable rather than orphaning a subgraph.
- core/graph/neo4j_store.py: schema (uniqueness on (repo_id, path), which is also the
index that keeps MERGE off a label scan), batched UNWIND+MERGE ingest at 500 rows,
and reads. sync_repository deletes the subgraph first rather than merging, because a
MERGE-only sync leaves edges for deleted files and drifts into a union of every commit
ever indexed.
- Traversals: reachable_from, blast_radius, shortest_path, import_cycles. Depth is
clamped (1..10) -- an unbounded variable-length pattern is a trivial way to hang the
server.
- Degree and centrality use COUNT {} subqueries, NOT Graph Data Science: the in-database
GDS plugin requires AuraDB Professional or above, and Aura Graph Analytics sessions
are an offline batch shape (2GB, one concurrent session, 30-minute TTL) that does not
fit a synchronous request.
- Async driver held as a single long-lived instance (it owns the pool) and closed in the
lifespan.
- docker-compose.yml: neo4j:5-community as a fourth service at 512m heap + 512m
pagecache, plus the five settings forwarded to the API.
Docs record the AuraDB Free trap: a Free instance auto-pauses after 72h idle and a
paused instance's hostname stops resolving, so the graph would silently fall back to SQL
until someone resumed it by hand.
Also fixes stale drift in .env.example that still advertised
LOCAL_EMBEDDING_MODEL=nomic-ai/nomic-embed-text-v1.5 -- the HuggingFace id removed from
config.py earlier, which would have walked a self-hoster straight back into the
all-zero-vector index.
VERIFIED, AND WHAT IS NOT
15 new tests, 132 total (was 117), ruff clean, app boots with the store returning None
by default. The tests use a fake driver, so they cover the queries we send and every
fallback path -- including that indexing still persists SQL edges when the graph sync
fails -- but NOT that the Cypher returns correct results. No query in this commit has
executed against a Neo4j server. docker/README.md documents the two count queries that
confirm the projection matches SQL once an instance is available.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #5. Enables the traversals the SQL path structurally cannot do, without making a new datastore load-bearing.
Why
The graph questions that matter for understanding a codebase are transitive: what breaks if I change this file, how does auth reach the database, are there import cycles. The Python implementation can't answer them —
hopsis hard-capped at 2 in the API because each hop rescans the whole edge list, so depth is quadratic in Python and a single variable-length pattern in Cypher.Design: projection, not source of truth
code_dependencies(SQL) stays authoritative. Neo4j is projected from it during indexing and rebuildable by re-indexing. Every consequence is deliberate:neo4j_enableddefaults to False — nothing changes unless you turn it on.get_graph_store()returnsNonewhen disabled or misconfigured, rather than raising, so callers treat "no graph store" as an ordinary path.Notable implementation choices
sync_repositorydeletes the subgraph first rather than merging. A MERGE-only sync leaves edges for deleted files behind and drifts into a union of every commit ever indexed.(repo_id, path)— which is also the index that keepsMERGEoff a label scan.COUNT {}subqueries, not Graph Data Science. The in-database GDS plugin requires AuraDB Professional or above, and Aura Graph Analytics sessions are an offline batch shape (2GB, one concurrent session, 30-min TTL) that doesn't fit a synchronous request. There's a test assertinggds.appears nowhere.UNWIND+MERGEat 500 rows, so a large repo isn't one enormous transaction.New traversals:
reachable_from,blast_radius,shortest_path,import_cycles.What is verified — and what is not
132 tests pass (was 117), ruff clean, app boots with the store returning
Noneby default.But I want to be explicit: the tests use a fake driver. They cover the queries we send and every fallback path — including that indexing still persists SQL edges when the graph sync fails — but not that the Cypher returns correct results. No query in this PR has executed against a Neo4j server, because there's no Docker on the machine I worked on.
docker/README.mddocuments the verification path —docker compose up -d neo4j, setNEO4J_ENABLED=true, re-index, then:Those should equal
count(*)fromcode_filesandcode_dependencies. I'd treat this PR as unlanded until someone runs that. The default-off flag is what makes merging it safe in the meantime.Also
Fixes stale drift in
.env.examplethat still advertisedLOCAL_EMBEDDING_MODEL=nomic-ai/nomic-embed-text-v1.5— the HuggingFace id removed fromconfig.pyin the first batch, which would have walked a self-hoster straight back into the all-zero-vector index.AuraDB Free warning, in the docs
A Free instance auto-pauses after 72h idle and a paused instance's hostname stops resolving, so the graph would silently fall back to SQL until someone resumed it by hand. Deleted after 30 days paused. Don't point a public demo at it.
🤖 Generated with Claude Code