Skip to content

Remove the Brain search reranker - #1768

Merged
mrubens merged 2 commits into
developfrom
chore/remove-brain-reranker
Aug 28, 2026
Merged

Remove the Brain search reranker#1768
mrubens merged 2 commits into
developfrom
chore/remove-brain-reranker

Conversation

@mrubens

@mrubens mrubens commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Why

While evaluating whether the reranker earns its keep, we found it has never actually run:

  1. gbrain's init (writeNewInstallRerankerDefault) writes search.reranker.enabled=false at Brain creation for installs keyed the way ours are (no VOYAGE_API_KEY). Our entrypoint set the reranker model after init but never flipped enabled — its reranker: boot log line was cosmetic. Verified in the config table of our deployments.
  2. Even force-enabled, openrouter:voyageai/rerank-2.5-lite is not on the OpenRouter recipe's strict rerank model allowlist (Cohere v3.5/4-fast/4-pro + one NVIDIA model), so every call throws pre-HTTP and the fail-open contract silently returns RRF order. Failures land only in an in-container audit JSONL.

A 72-query known-item A/B (gold page per query, reranker off vs force-enabled, config restored after) confirmed it: 0/72 result lists changed, and the pure hybrid-RRF baseline everyone has been using all along scores hit@1 68% / hit@5 90% / hit@20 97%, median gold rank 1.

So this PR makes reality the explicit contract instead of an accident. Search behavior is unchanged by construction; autocut already no-ops without rerank scores by design.

What

  • entrypoint: replaces the reranker gateway wiring with an explicit gbrain config set search.reranker.enabled false — converges pre-existing brains and is immune to upstream mode-bundle default flips (one is planned for September).
  • inference gateway: drops /v1/rerank from the Brain's allowed paths, the rerank upstream branch, and the OpenRouter-only guard; tests updated (rerank path now 403s like any unlisted path).
  • compose bundles: stop passing GBRAIN_RERANKER_MODEL/R_BRAIN_RERANKER_MODEL/R_BRAIN_RERANK_UPSTREAM_URL, and the local-inference Infinity service no longer loads BAAI/bge-reranker-v2-m3 (several GB of RAM for a model nothing calls).
  • env schema / example / docs: reranker variables and instructions removed; docs now state Memory retrieval is hybrid RRF with no cross-encoder stage.

If we ever want a real reranker, the path is an allowlisted model (e.g. openrouter:cohere/rerank-4-fast) or a local upstream — gated on an A/B showing it actually helps.

Testing

  • @roomote/api: 1910 passed (brain-inference suite updated).
  • @roomote/dev gbrain-image test, compose security contract validator, bash -n on the entrypoint: all pass.
  • Prod compose resolves; the self-host compose fragment parse quirk under bare docker compose config is pre-existing and unchanged.

The reranker never actually ran: gbrain's init writes
search.reranker.enabled=false for installs keyed the way ours are, and
even when force-enabled, the configured openrouter:voyageai model is not
on the OpenRouter recipe's rerank allowlist, so every call failed open
to RRF order before issuing any request. A 72-query known-item A/B
confirmed enabling it changes nothing (0/72 result lists differed;
hit@1 68%, hit@5 90%, hit@20 97% on pure hybrid RRF).

Make that the explicit contract: the entrypoint disables the reranker
per-brain, the inference gateway drops /v1/rerank, the compose bundle
stops loading a reranker model nothing calls, and the reranker env
plumbing and docs go away. Autocut already no-ops without rerank scores
by design.
@roomote-community

roomote-community Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

No code issues found. See task

  • Remove stale reranker variables and operator guidance from the Coolify, Railway, and Render deployment templates.

Reviewed 64c557b

Comment thread packages/env/src/index.ts
Coolify, the Railway template, and Render still exposed and forwarded
R_BRAIN_RERANKER_MODEL, and their READMEs told operators to set a knob
the entrypoint now ignores.
@mrubens
mrubens merged commit a59b564 into develop Aug 28, 2026
18 checks passed
@mrubens
mrubens deleted the chore/remove-brain-reranker branch August 28, 2026 04:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant