fix(graph): stop the embedding miner from failing silently

[Cipher208]

The bug

miner_embedding wrapped embedding, quantization and bit conversion in one try/except Exception that returned {"edges": 0} and logged nothing:

except Exception:
    return {"edges": 0}  # embedding backend unavailable (no numpy/model) — miner skipped

The comment is wrong twice. embed_texts does not raise when no model is available — it falls back to hash vectors (_compute_missing_embeddings) — and embed_to_binary has its own numpy-less path. So the only exceptions reaching that handler were real failures, an unreachable or slow embedding service above all.

What it cost

semantic_overlap stood still for six days while every other relation kept growing:

relation	30.09	04.10	delta
semantic_overlap	18294	18294	0
co_mentions	8840	9626	+786
topic_overlap	6664	6678	+14
узлов	1785	1863	+78

By edge created_at, the two passes of 03.10 and 05.10 produced co_mentions edges and exactly zero semantic_overlap — so the passes ran and this one miner was dead. memory.log said nothing, and a frozen number reads as saturation: the outage was first explained as the gate working as intended.

The fix that was already there

graph_enrich had known how to report a failed miner since an audit that found a silently failed miner:

except Exception as exc:
    logger.warning("miner %s failed: %s", name, exc)
    miners[name] = {"edges": -1, "error": str(exc)[:200]}

That branch was unreachable, because the swallow sits inside the miner, below it. The exception is now allowed to escape, which reaches it. Two tests pin both halves: the miner raises, and the pass records -1 with the reason.

Per-miner edge counts in the log

miner edges: layer=user tags=42 tokens=117 sessions=88 embedding=0 markers=3 ...

-1 still means failure, with the reason on the warning line above. The phase timings from fa6c3e5 say which miner ran and for how long; they could not say that a miner ran and produced nothing — which is exactly what the outage looked like, a 0 ms phase beside a 0 indistinguishable from "no work to do".

The 30 s ceiling

_remote_embed posted the whole text list in one request under a fixed timeout=30. A ~6k-text layer measured 24.4 s — about 20 % of headroom — so the timeout was a ceiling on layer size, and the swallowed timeout is the likeliest trigger.

It now asks in batches of 500 (_REMOTE_BATCH_SIZE, matching the existing _CACHE_LOOKUP_CHUNK convention), keeping a single request near ~2 s and giving room for much larger layers. Vectors still come back in input order, and a list smaller than one batch still costs exactly one POST.

Tests

Four on batching (split into bounded requests, order preserved across batches, one request when small, no request when empty), one that a backend failure escapes miner_embedding, one that a failed miner is reported as -1 with its reason, one that per-miner counts reach the log.

Full suite: 2085 passed.

Note on a test-isolation trap

The new test file initially broke test_shared.py::test_archived_memories in the full suite: its fixture left a connection in connection_manager._conns pointing at a deleted tmp_path, and that neighbouring test is not isolated — it takes its connection from the same cache. Fixed with a fixture that clears _conns on teardown, and documented in the file, because the trap is in the neighbour, not the new test.

The one pre-existing failure this exposed is unrelated and was confirmed by stashing: test_archived_memories fails on a clean master too when run alone, on no such table: archived_memories.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

• Configuration used: Repository: Cipher208/a-memory/.coderabbit.yaml
• Review profile: CHILL
• Plan: Advanced
• Run ID: 78be3158-e6d6-41b0-a5b3-e238a3dbfc0c

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}