Skip to content

Refactor hybrid search pattern into oidm-common for reuse across packages #33

Description

@talkasab

Problem

Both findingmodel and anatomic-locations packages implement similar hybridsearch patterns with significant code duplication:

  1. Exact match lookup (by ID, name, slug, synonym)
  2. FTS search using DuckDB's BM25
  3. Semantic search using vector embeddings
  4. RRF fusion to combine FTS + semantic scores
  5. Result ranking and deduplication

The search() and search_batch() methods in findingmodel/index.py duplicate this pattern, and anatomic-locations has its own similar implementation. This leads to:

  • Duplicated logic that's hard to maintain
  • Inconsistent behavior between packages
  • Higher complexity scores (C901) from inlined fusion logic

Proposed Solution

Extract a reusable HybridSearcher or hybrid_search() utility into oidm-common that encapsulates:

# oidm-common/src/oidm_common/search.py
class HybridSearcher(Generic[T]):
    """Reusable hybrid search with RRF fusion."""
    def __init__(
        self,
        exact_fn: Callable[[str], list[T]],
        fts_fn: Callable[[str, int], list[tuple[T, float]]],
        semantic_fn: Callable[[list[float], int], list[tuple[T, float]]],
        id_extractor: Callable[[T], str],
    ): ...
    def search(
        self,
        query: str,
        embedding: list[float] | None,
        limit: int,
    ) -> list[T]: ...

Packages to Update

  1. oidm-common: Add HybridSearcher or equivalent
  2. findingmodel: Refactor DuckDBIndex.search() and search_batch() to use it
  3. anatomic-locations: Refactor AnatomicIndex search methods to use it

Acceptance Criteria

  • oidm-common exports reusable hybrid search utility
  • findingmodel search methods use the shared utility
  • anatomic-locations search methods use the shared utility
  • No C901 complexity suppressions needed
  • All existing tests pass
  • Search behavior unchanged (verify with existing test coverage)

References

  • packages/findingmodel/src/findingmodel/index.py - search(), search_batch()
  • packages/anatomic-locations/src/anatomic_locations/index.py - search implementation
  • packages/oidm-common/src/oidm_common/duckdb.py - existing rrf_fusion(),normalize_scores()

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions