Skip to content

feat(search): surface camelCase symbols via segment vocab (#1520) - #1530

Open
terrabeckhub wants to merge 2 commits into
colbymchenry:mainfrom
terrabeckhub:feat/fts-camelcase-segment-1520
Open

feat(search): surface camelCase symbols via segment vocab (#1520)#1530
terrabeckhub wants to merge 2 commits into
colbymchenry:mainfrom
terrabeckhub:feat/fts-camelcase-segment-1520

Conversation

@terrabeckhub

Copy link
Copy Markdown

Summary

Addresses the largest finding in #1520: camelCase / PascalCase identifiers are unreachable by sub-word FTS queries because nodes_fts uses the default unicode61 tokenizer (no case-boundary splits).

name_segment_vocab already materializes those sub-words at index time for the prompt hook. This PR merges that vocabulary into searchNodes candidate generation so queries like checkout surface getShippingMethodIdFromCheckout without a custom FTS5 tokenizer or schema migration.

Changes

  • After FTS, call supplementWithSegmentMatches() to pull symbol names from name_segment_vocab for each query term and join them into the candidate set (still subject to kind/language filters and the existing rescoring path).
  • Add a regression test that indexes a camelCase helper and asserts searchNodes('checkout') finds it.

Other #1520 items (persisted bm25 rank weights, dropping id from FTS, --path CLI, GROUP BY dedupe) are left for follow-ups; search already applies column-weighted bm25(...) in SQL and supports path: query filters.

Test plan

  • npx vitest run __tests__/search-segment-supplement.test.ts
  • CI

69fd998aiw33bc added 2 commits August 8, 2026 10:26
…ry#1520)

Signed-off-by: 69fd998aiw33bc <69fd998aiw33bc@users.noreply.github.com>
…ry#1520)

Signed-off-by: 69fd998aiw33bc <69fd998aiw33bc@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant