perf(knowledge): the write batch was tied to the embed batch, and they want opposites

[sshlg]

The regression, and the wrong suspect

Moving the vector store to Postgres (#241) made code_symbol_embed go 38 min → 65 min on a full rebuild.

The obvious culprit is HNSW index maintenance on insert. Measured server-side, 8 000 rows of 384 dimensions:

without HNSW index    1 376 ms
with HNSW index      12 348 ms     9× per row

9× is real — and it comes to only ~44 s across our whole corpus. Nowhere near the tens of minutes observed. The index is not the cause, and batching the write would have been the wrong fix had it been.

The real one

One number governed both the embedding and the write. That cost nothing while ChromaDB's write was a local file, and a great deal once it was a statement over a network. The two constraints are opposed:

• Embedding is memory-bound. The ONNX model pads every document to 256 tokens, so activations are sized by the batch: 967 MiB at 200 against 415 MiB at 8. An unbounded batch is what SIGKILLed the worker (AUD-0819-01).
• Writing is round-trip-bound. Measured against the production database with the embedder stubbed out, 800 rows:

batch=  8   16.67 s   (100 round-trips, ~167 ms each)
batch=100    2.31 s   (  8 round-trips)
batch=400    1.92 s   (  2 round-trips)

A fixed ~167 ms per statement, paid 3 196 times over a full rebuild.

PGVECTOR_WRITE_BATCH_SIZE defaults to 100, the embed batch stays at 8, and the write batch can never be smaller than the embed batch. End to end on the same 800 rows: 13.89 s → 2.96 s, ranking unchanged to four decimal places (0.2234 / 0.7867 / 1.0161).

Ceiling follows the measurement

repo_index_job_timeout_seconds → 21600. The pgvector rebuild measured 15 051 s end to end (15:52 → 20:02 on 2026-08-28, chained code↔DB sync included) against 12 039 s on ChromaDB. The write fix should bring that down and has not been re-measured end to end, so the ceiling is sized from what was measured rather than from what is expected.

Verified in production, with the event that used to empty the store

before restart   34 038 vectors, visible to a fresh one-off dyno
restart both dynos
after restart    34 038 vectors — handle.count() = 34 038, force_full would not fire

Both halves matter. Surviving the restart is the fix; being visible to a different dyno is the half ChromaDB could never do, because web and worker hold separate filesystems.

The rebuild reached pipeline_end, the checkpoint was deleted (which only happens on success), and the chained sync ran unprompted — code_db_sync 20:01:51 → 20:02:51. Map after: 136 matched / 97 db_only / 23 code_only / 0 mismatch.

Verification

• pytest tests/ — 7003 passed, 4 skipped, 1 xfailed.
• ruff format --check, ruff check, mypy app/ — clean, 0 errors.
• New tests pin the split: the write batch has its own setting, it is floored at the embed batch, and both are read at the call site.
• # noqa ceiling 128 → 129 for the DocEmbedding re-export, which carries the convention the other forty-two imports in that file already carry. A second suppression was added and then deleted — annotating an inner function's argument cost one word, which is the answer the ratchet exists to provoke.