feat(retrieval): Multi-Query and HyDE fusion retrievers with RRF (#215)

[shivamm-gupta]

Description

Implements MultiQueryRetriever and HyDERetriever as requested in #215 — two first-class query-expansion adapters that address vocabulary mismatch and underspecified queries in RAG applications.

This PR provides a comprehensive implementation that addresses gaps and limitations in PR #222, ensuring complete backwards compatibility with maintainer @Kaap10's specification in #215, full multi-passage HyDE support, true async LLM execution, and idiomatic Dynavec ergonomics.

Related issue

Closes #215

---

Key Advantages over PR #222

Area	PR #222	This PR	Why It Matters
Issue #215 API Spec Compatibility	Accepts only generate_queries, generate_hypothetical, include_original	Accepts both conventions: llm_generate_queries, llm_generate_hypothetical, include_original_query (plus shorthand aliases)	Code copied directly from the issue description written by maintainer @Kaap10 works without raising TypeError.
Multi-Passage HyDE	Drops all passages after the first (cleaned[0]); only takes Callable[[str], str]	Supports Callable[[str], str | Sequence[str]] with strategy="average" (Centroid HyDE) and strategy="fuse"	In the HyDE literature (Gao et al., 2022), generating 3–5 passages and averaging embeddings (Centroid) cancels out hallucinations and saves AWS query cost by doing 1 vector search instead of $N$.
Async LLM Generators	asearch wraps sync search in a thread; passing an async def function returns an unawaited coroutine that gets silently dropped	Introspects coroutines; awaits them natively in asearch() and runs safely in search()	Modern RAG systems (FastAPI, LiteLLM, LangChain ainvoke, Anthropic/OpenAI async clients) use async LLM calls.
Client & Namespace Ergonomics	Requires manual constructor calls	Exposes db.as_multiquery_retriever(), db.as_hyde_retriever(), ns.as_multiquery_retriever(), and ns.as_hyde_retriever()	Aligns with Dynavec's fluent interface pattern (as_langchain_tool, namespace()).
Import Locations	Exported only from dynavec and dynavec.retrievers	Exported from dynavec, dynavec.retrieval (as requested in #215), and dynavec.retrievers	from dynavec.retrieval import MultiQueryRetriever works out of the box.
Query Weighting	Only original_weight exposed; reformulations hardcoded to 1.0	Supports custom weights in search(..., weights=...)	Compatible with learned RRF weights from RRFWeightFitter (#49).
Unit Test Coverage	37 tests	48 tests (100% offline, 0 AWS needed)	Comprehensive coverage including centroid averaging, async LLM generators, parameter aliasing, and factory methods.

---

Changes

• src/dynavec/retrievers.py (new):
  • QueryExpansionRetriever: Shared base class handling client/namespace resolution, concurrent fan-out, submission-order tie-breaking, error fallback/raise policy, and RRF fusion.
  • MultiQueryRetriever: LLM reformulation fan-out, text sanitization and deduplication, original query prioritization, and custom weight support.
  • HyDERetriever: Document-side embedding (embed_documents), single- and multi-passage hypothetical generation with centroid averaging (strategy="average") and multi-search RRF (strategy="fuse").
• src/dynavec/client.py: Added as_multiquery_retriever and as_hyde_retriever factory methods.
• src/dynavec/namespace.py: Added as_multiquery_retriever and as_hyde_retriever factory methods pinned to the scoped namespace.
• src/dynavec/retrieval.py: Re-exported retrievers for backwards compatibility with the issue specification.
• src/dynavec/__init__.py: Exported MultiQueryRetriever, HyDERetriever, and QueryExpansionRetriever in __all__.
• src/dynavec/integrations/tools.py: Extended make_retriever_fn, as_langchain_tool, and as_crewai_tool to accept QueryExpansionRetriever.
• tests/test_retrievers.py (new): 48 comprehensive unit tests covering deduplication, error fallback, concurrency determinism, async LLMs, multi-passage centroid averaging, and tool integrations.
• examples/query_expansion.py (new): Runnable offline demonstration (no AWS, no LLM required) illustrating how Multi-Query and HyDE solve vocabulary mismatch.
• README.md & CHANGELOG.md: Added documentation, code examples, and changelog entry.

Testing

• All 48 new tests pass: uv run --no-sync pytest tests/test_retrievers.py -v
• Full regression suite passes: uv run --no-sync pytest -q (100% pass)
• Ruff lint checks pass: uv run ruff check src tests examples (0 errors)
• Offline example runs cleanly: uv run python examples/query_expansion.py

Checklist

• My changes are focused and relevant to this pull request.
• I have added or updated tests where appropriate.
• I have reviewed my changes for unrelated modifications.
• I have updated documentation where necessary.

[shivamm-gupta]

@Kaap10 please check this once

[codeforstartups]

Landed in development as 7d75e52 — thanks @shivamm-gupta! 🙌 Multi-Query + HyDE (with both average-centroid and fuse strategies) fused via RRF, plus the client/namespace helpers and tests. It had drifted into conflict with __init__.py/namespace.py from other merges this cycle, so I rebased it onto current development and resolved those (both were pure additions — kept everything). Verified: ruff clean, full suite 606 passed. Closing as merged-manually. This supersedes #222.