feat(knowledge): put the vector store in Postgres, off the disk that gets wiped

[sshlg]

The defect

CHROMA_PERSIST_DIR is /app/data/chroma — the Heroku container filesystem. It does not survive a dyno restart, and web and worker are separate process types with separate copies of it. Production, 2026-08-27, five hours after a deploy:

22:00:44  repair_embeddings  "Vector store empty but 758 docs in DB.
                              Forcing a full re-index."
22:01:21  code_symbol_embed  started   (all 8 646 files — force_full)
22:39:09  code_symbol_embed  completed (38 min)
23:59:50  heartbeat stops — the 7 200 s ceiling, at 380 of 758 documents
00:04     stale run reaped

A full rebuild costs 12 039 s and the nightly budget is 7 200 s, so the store was empty again the next night, and the night after:

	completed	runs
index_repo	16	94	17 %
daily_sync	13	91	14 %

The C3 self-repair (v1.13.0) was not wrong. The repair simply cost more than the budget allowed — which is not something a bigger ceiling can fix, and #237/#240 raising that ceiling did not fix it.

The change

PgVectorStore is a drop-in: same six methods, same shapes, same cosine distance. The collection handle still answers count(), because that is exactly what pipeline_runner.py:415 reads to decide whether a full re-index is owed — the decision at the centre of the loop.

Embeddings do not change. ChromaDB computed them internally with its bundled ONNX all-MiniLM-L6-v2; this calls the same class directly. Identical vectors, identical 384 dimensions, and vector_cosine_ops because the collections it replaces were created with {"hnsw:space": "cosine"}. Verified end to end against the production database: same four documents, same ranking, distances 0.2234 / 0.7867 / 1.0161 — re-verified unchanged after a later edit to query().

HNSW over IVFFlat, and not by preference: HNSW needs no training pass, so it is correct from an empty table. IVFFlat built on an empty table returns nothing until rebuilt — the exact failure this removes. Measured on this schema, 30 000 rows × 384 dims:

build                                20 s
top-5 filtered, no vector index    4738 ms
top-5 filtered, HNSW (warm)        0.92 ms

hnsw.iterative_scan was measured too and makes no difference at this selectivity; the 65 ms it first appeared to save was a cold cache, controlled for with three warm runs each way.

Decisions worth arguing with

• VECTOR_STORE_BACKEND still defaults to chroma. The flip is a decision taken on a verified deployment, not a side effect of merging.
• psycopg beside asyncpg. The interface is called synchronously from ten sites including the agent's hot path (context_loader.py:213,421, knowledge_catalog_service.py:480,539). Converting those to async is a larger, riskier change than a second driver.
• Postgres only. Dev and the whole suite run on SQLite, where the migration is a deliberate no-op and the model carries dialect variants; asking for pgvector on a SQLite URL fails at start-up saying so, rather than later on a missing table.
• This does not wait for the Supabase move. pgvector is available on both — 0.8.1 on Heroku Postgres, 0.8.2 on Supabase — so the urgent fix is decoupled from the database migration.

Also: three checks over the migration graph

b1c2d3e4f5a6 was already taken by b1c2d3e4f5a6_batch_started_at_claim.py. Alembic reports that as CycleDetected naming four unrelated revisions, and only when something walks the graph — so a migration with a duplicate id can be committed and merged before anything notices. Ids here are hand-picked rather than generated, which puts it within reach.

Read with ast, not a regex: the first version reported seven heads where alembic reports one, because merge migrations name their parents as a tuple. A fourth test fails if that understanding ever regresses — a test that misreads the graph is worse than none, because it fails on a healthy repo and gets deleted.

Negative control run: with a duplicate planted the check fails; removed, all four pass.

Verification

• pytest tests/ — 6987 passed, 4 skipped, 1 xfailed.
• ruff format --check, ruff check, mypy app/ — clean, 0 errors.
• Migration proven against production Postgres inside a transaction, then rolled back; alembic_version untouched, table dropped, prod unchanged.
• Suppression ceilings raised by one each with the reason in the same commit, as that ratchet asks.

Post-merge, per CB-KNOW2

Deploy → set VECTOR_STORE_BACKEND=pgvector → one full re-index reaching pipeline_end → then watch a deploy happen and the next nightly run stay incremental. The row is struck on that evidence, not on this merge.