Background
The current semantic search uses pure vector (cosine similarity) retrieval. Pure vector search has a known weakness: short, specific, or keyword-heavy queries (e.g. ConfigureServices, UseAuthentication, IDbContextFactory) retrieve poorly because the embedding model compresses them into a sparse region of vector space.
Hybrid search combines two complementary signals:
- BM25 / full-text — precise term matching; excellent for exact identifiers, method names, config keys
- Vector / semantic — concept matching; excellent for natural language, paraphrases, and related terms
The spec explicitly deferred this as a non-goal but tracked it separately:
"Hybrid keyword + vector re-ranking — tracked separately."
Current retrieval quality
| Metric |
Value |
Threshold |
Status |
| Recall@1 |
0.4667 |
≥ 0.60 |
INVESTIGATE |
| Recall@3 |
0.8333 |
≥ 0.75 |
PASS |
| MRR |
0.6428 |
≥ 0.65 |
INVESTIGATE |
Recall@1 is 13 points below the pass threshold. Hybrid search is expected to improve short, specific queries that currently miss rank-1.
Proposed Approach
-
SQLite provider — SQLite supports FTS5 natively. Add a pages_fts virtual table mirroring the page_id, title, and content columns. At query time, run BM25 via MATCH and merge scores with the vector cosine result using a configurable weight (default: 0.7 vector + 0.3 BM25).
-
Postgres provider — PostgreSQL supports tsvector / tsquery full-text. Add a search_vector GIN-indexed column. Same weighted merge approach.
-
IVectorStore interface — add HybridSearchAsync(ReadOnlyMemory<float> queryVector, string queryText, int topN, float minScore, CancellationToken ct) or a SearchOptions parameter to the existing SearchAsync that carries the raw query text.
-
SemanticSearchToolHandler — pass the original query text through to the store so BM25 can use it; the current flow discards it after embedding.
-
Evaluation — re-run the golden-dataset evaluation after implementing hybrid search and record results in evaluation-report.md.
Acceptance Criteria
Background
The current semantic search uses pure vector (cosine similarity) retrieval. Pure vector search has a known weakness: short, specific, or keyword-heavy queries (e.g.
ConfigureServices,UseAuthentication,IDbContextFactory) retrieve poorly because the embedding model compresses them into a sparse region of vector space.Hybrid search combines two complementary signals:
The spec explicitly deferred this as a non-goal but tracked it separately:
Current retrieval quality
Recall@1 is 13 points below the pass threshold. Hybrid search is expected to improve short, specific queries that currently miss rank-1.
Proposed Approach
SQLite provider — SQLite supports FTS5 natively. Add a
pages_ftsvirtual table mirroring thepage_id,title, andcontentcolumns. At query time, run BM25 viaMATCHand merge scores with the vector cosine result using a configurable weight (default: 0.7 vector + 0.3 BM25).Postgres provider — PostgreSQL supports
tsvector/tsqueryfull-text. Add asearch_vectorGIN-indexed column. Same weighted merge approach.IVectorStoreinterface — addHybridSearchAsync(ReadOnlyMemory<float> queryVector, string queryText, int topN, float minScore, CancellationToken ct)or aSearchOptionsparameter to the existingSearchAsyncthat carries the raw query text.SemanticSearchToolHandler— pass the original query text through to the store so BM25 can use it; the current flow discards it after embedding.Evaluation — re-run the golden-dataset evaluation after implementing hybrid search and record results in
evaluation-report.md.Acceptance Criteria
bookstack_semantic_searchuses hybrid BM25+vector scoring when availableVectorSearch:Hybrid:Enabled, defaultfalse) so it can be enabled gradually