Repository navigation
fix(graphile-search): exclude rows with no embedding from vector search - #1978
Conversation
Rows whose parent embedding and every chunk embedding are NULL have no distance to rank. The pgvector adapter now reports a scorable predicate (embedding IS NOT NULL, or an embedded chunk exists on the chunk-aware path) that the filter ANDs into WHERE, and the chunk-aware distance is a plain LEAST(parent, chunk) instead of COALESCE(..., 'Infinity'). This stops Infinity reaching the GraphQL Float serializer, makes totalCount and first/after paging consistent on both paths, and keeps the HNSW index scan for parent-only ordering. The rank window used by searchScore is NULL for unscored rows instead of numbering them after the scored ones.
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
|
Review complete. 🟡 1 medium 💬 Inline comments (1)
The PR makes vector search exclude rows that cannot be scored instead of passing
Reviewed commit: 76782b3 |
There was a problem hiding this comment.
This PR fixes NULL-embedding handling in the graphile-search pgvector adapter by switching the combined parent/chunk distance to a NULL-skipping LEAST, adding a scorableClause exclusion predicate to FilterApplyResult, and making the rank NULL for unscored rows, with a new seed fixture and integration test.
Key findings
- 🟡 Duplicate correlated chunk EXISTS added per row — pgvector.ts:236
| const scorableClause = sql`(${parentColumnExpr} IS NOT NULL OR EXISTS ( | ||
| SELECT 1 | ||
| FROM ${chunksTableRef} AS ${chunksAlias} | ||
| WHERE ${chunksAlias}.${parentFk} = ${parentId} | ||
| AND ${chunksAlias}.${chunkEmbedding} IS NOT NULL | ||
| ))`; |
There was a problem hiding this comment.
🟡 performance · medium
Duplicate correlated chunk EXISTS added per row
The new scorableClause at pgvector.ts:236-241 adds a second correlated EXISTS (SELECT 1 FROM chunks WHERE parent_fk = ... AND embedding IS NOT NULL) evaluated for every candidate row in the WHERE clause, even though the adjacent chunkDistanceSubquery (lines 221-225) already scans the same chunks rows to compute MIN(distance). Postgres does not common-subquery-eliminate these, so chunk-table scanning work per parent row roughly doubles on the vector search path, worst for parents with many chunks.
📋 Prompt for AI Agents
In graphile/graphile-search/src/adapters/pgvector.ts around lines 235-241, remove the newly added EXISTS (SELECT 1 FROM chunks ...) used as scorableClause. Reuse the existing chunkDistanceSubquery variable (lines 221-225) and express the scorable predicate as (${parentColumnExpr} IS NOT NULL OR ${chunkDistanceSubquery} IS NOT NULL), avoiding a second scan of the chunks table per candidate row.
…scorable predicate
Summary
A row whose parent embedding is NULL and whose chunks (if any) all have NULL embeddings has no distance to rank. The pgvector adapter used to keep such rows anyway: the chunk-aware path scored them
LEAST(COALESCE(d, 'Infinity'), COALESCE(chunk_d, 'Infinity')), which GraphQL'sFloatserializer rejects (Float cannot represent non numeric value: Infinity), and the parent-only path scored them NULL, sototalCountcounted them andfirst/afterpaging over a NULL cursor lost rows depending on the plan (HNSW index scan skips NULLs; a seq scan does not). They now stay out of the result set on both paths.Mechanics:
SearchAdapter.applyFilterresult gains an optionalscorableClause: SQL. The pgvector adapter sets it toembedding IS NOT NULL(parent-only / no@hasChunks) orembedding IS NOT NULL OR (SELECT MIN(chunk_d) FROM chunks c WHERE c.fk = parent.pk) IS NOT NULL(chunk-aware; same subquery as the distance expression, andMINis NULL exactly when no chunk is embedded). The plugin ANDs it into the per-adapter filter's WHERE next towhereClause. The compositeunifiedSearchfilter does not apply it: a row with no embedding can still match via another adapter.LEAST(parent_d, (SELECT MIN(chunk_d) ...))with noCOALESCE(..., 'Infinity').LEASTignores NULL operands, and the predicate above means the only rows where both are NULL are gone.searchScorerank:ROW_NUMBER() OVER (ORDER BY score NULLS LAST)becameCASE WHEN score IS NULL THEN NULL ELSE ROW_NUMBER() ... END, so rows an adapter cannot score contribute no rank to the RRF fusion instead of being numbered right after the scored rows. The existing score field descriptions ("Returns null when no ... search filter is active") are still accurate and are unchanged..agents/skills/graphile-search/references/pgvector-adapter.mdupdated to the new SQL.Not in this PR (noted only): deep paging past the default
hnsw.ef_searchlimit (hnsw.iterative_scan), and the underlying@dataplan/pgbehaviour where a NULLORDER BYvalue yields a cursor that drops rows.Tests
New
src/__tests__/vector-unembedded.test.ts+vector-unembedded.sql: nullablevector(3)columns with HNSW indexes on a parent + chunks table, plus the same rows in a table with no index (forces a seq scan). Covers chunk-aware andincludeChunks: falserow sets,totalCount,first/afterpaging in both directions, the distance threshold, andsearchScoremonotonicity.main: 14 failed / 7 passed (Infinity serialization errors,totalCount7 instead of 3/4, seq-scan paging returning unembedded rows).tsc --noEmitclean;eslint srcreports only the pre-existing unusedscoreMetaKeywarning.HNSW plan, 200k rows (64-dim, 10% NULL parents, 50k chunks of which 10k NULL),
ORDER BY distance LIMIT 10, PG 18, warm cache, 3 runsincludeChunks: false)docs_embedding_idx, 0.28–0.31 msdocs_embedding_idx+Filter: (embedding IS NOT NULL), 0.11–0.12 msRows Removed by Filter: 20000) + top-N heapsort, 337–340 msThe parent-only path keeps the HNSW index scan; the NULL predicate is applied as a filter on top of it. The chunk-aware path never used the index (ordering by
LEAST(...)over a subquery is not indexable) and is unchanged in shape. AnEXISTS (... embedding IS NOT NULL)form of the chunk predicate was also measured (386–389 ms, planner hashes it into a full chunks scan) and rejected in favour of reusing theMINsubquery.Link to Devin session: https://app.devin.ai/sessions/2c8a6e5b1ad0423f8bb10849a019c44b
Open in Devin Desktop: https://app.devin.ai/desktop/session/2c8a6e5b1ad0423f8bb10849a019c44b?variant=devin
Requested by: @pyramation