feat(search): implement normalized weighted score fusion for hybrid retrieval

ARCHITECTURE.md

@@ -1,4 +1,4 @@
-# Architecture: ContextCortex (v2.13.0)
+# Architecture: ContextCortex (v2.14.0)
 
 ContextCortex provides fast, local, syntax-aware semantic and hybrid search over codebases, git repositories, markdown notes, architecture documents, and system documentation. It is built natively on the **Model Context Protocol (MCP) SDK 2.0.0+** using `FastMCP`, with an integrated FastAPI web engine, real-time diagnostic logging, pluggable relational and vector store backends (PostgreSQL 16 with pgvector, Qdrant, ChromaDB, and SQLite), automatic polling daemons, multi-provider webhooks, interactive dependency topology graph explorer, RFC 9728 OAuth 2.1 Protected Resource Server, 3-tier API key RBAC, and a React 19 administrative dashboard.
 

DEVELOPER_DOCS.md

@@ -1,4 +1,4 @@
-# Developer Documentation: ContextCortex (v2.13.0)
+# Developer Documentation: ContextCortex (v2.14.0)
 
 This document provides instructions for developing, testing, configuring, and running ContextCortex locally.
 

README.md

@@ -1,4 +1,4 @@
-# ContextCortex (v2.13.0)
+# ContextCortex (v2.14.0)
 
 
 [![Build and Publish Docker Image](https://github.com/spelech/contextcortex/actions/workflows/docker-publish.yml/badge.svg)](https://github.com/spelech/contextcortex/actions/workflows/docker-publish.yml)

REQUIREMENTS.md

@@ -1,8 +1,8 @@
-# Software Requirements Specification: ContextCortex (v2.13.0)
+# Software Requirements Specification: ContextCortex (v2.14.0)
 
 > **Note:** This document is automatically generated and verified against the live test suite by `scripts/generate_requirements.py` and `tests/backend/test_requirements_sync.py`.
 
-**Test Verification Baseline:** **928 Automated Tests** (613 Pytest Backend + 269 Vitest Frontend + 46 Playwright E2E).
+**Test Verification Baseline:** **934 Automated Tests** (619 Pytest Backend + 269 Vitest Frontend + 46 Playwright E2E).
 
 ---
 
@@ -784,7 +784,7 @@ classDiagram
 - `test_test_connection_active_embedded` - _Verify test_connection succeeds on active embedded Qdrant store without file lock conflict._
 - `test_switch_same_embedded_directory` - _Verify switch_vector_store succeeds when switching collection on the same embedded Qdrant directory._
 
-#### `tests/backend/test_vector_store_qdrant.py` (26 tests)
+#### `tests/backend/test_vector_store_qdrant.py` (32 tests)
 - `TestQdrantVectorStoreInit::test_init_in_memory_or_embedded`
 - `TestQdrantVectorStoreInit::test_init_remote_success`
 - `TestQdrantVectorStoreInit::test_init_remote_fallback_to_embedded_on_connection_error`
@@ -795,6 +795,9 @@ classDiagram
 - `TestQdrantVectorStoreOperations::test_upsert_failure_handling_and_logging`
 - `TestQdrantVectorStoreOperations::test_upsert_dict_documents_auto_computes_vectors`
 - `TestQdrantVectorStoreOperations::test_search_dense_and_hybrid_rrf`
+- `TestQdrantVectorStoreOperations::test_search_weighted_score_fusion_range_and_boost`
+- `TestQdrantVectorStoreOperations::test_search_weighted_score_fusion_alpha_weighting`
+- `TestQdrantVectorStoreOperations::test_search_dense_fallback_without_sparse`
 - `TestQdrantVectorStoreOperations::test_delete_by_path`
 - `TestQdrantVectorStoreOperations::test_delete_by_repo`
 - `TestQdrantVectorStoreOperations::test_get_stats_and_health_check`
@@ -808,6 +811,9 @@ classDiagram
 - `test_upsert_failure_handling_and_logging`
 - `test_upsert_dict_documents_auto_computes_vectors`
 - `test_search_dense_and_hybrid_rrf`
+- `test_search_weighted_score_fusion_range_and_boost`
+- `test_search_weighted_score_fusion_alpha_weighting`
+- `test_search_dense_fallback_without_sparse`
 - `test_delete_by_path`
 - `test_delete_by_repo`
 - `test_get_stats_and_health_check`

app/mcp/handlers/search_handlers.py

@@ -46,7 +46,7 @@ async def handle_search_code(
             link_url = p.get("permalink_url") or p.get("github_url")
             if link_url:
                 header += f"\nSource Link: {link_url}"
-            header += f"\nRRF Score: {hit.score:.4f}\n"
+            header += f"\nRelevance Score: {hit.score:.4f} ({hit.score * 100:.1f}%)\n"
 
             lang = p.get("language", "")
             block = f"{header}```{lang}\n{p.get('content')}\n```"
@@ -89,7 +89,7 @@ async def handle_search_docs(
             link_url = p.get("permalink_url") or p.get("github_url")
             if link_url:
                 header += f"\nSource Link: {link_url}"
-            header += f"\nRRF Score: {hit.score:.4f}\n"
+            header += f"\nRelevance Score: {hit.score:.4f} ({hit.score * 100:.1f}%)\n"
 
             block = f"{header}---\n{p.get('content')}"
             formatted.append(block)

app/services/vector_store/qdrant_store.py

@@ -277,7 +277,7 @@ def search(
         tag: Optional[str] = None,
         limit: int = 5
     ) -> List[VectorSearchResult]:
-        """Performs vector search returning ranked results using Dense + Sparse RRF."""
+        """Performs vector search returning ranked results using Dense + Sparse normalized weighted fusion."""
         if not query_text or not query_text.strip():
             return []
 
@@ -303,45 +303,93 @@ def search(
 
             query_filter = qmodels.Filter(must=must_conditions) if must_conditions else None
 
+            try:
+                alpha = float(os.getenv("HYBRID_DENSE_WEIGHT", "0.7"))
+            except (ValueError, TypeError):
+                alpha = 0.7
+            alpha = max(0.0, min(1.0, alpha))
+
+            candidate_limit = max(limit * 5, 50)
+
             if sparse_vec is not None and len(sparse_vec.indices) > 0:
-                response = self.client.query_points(
+                batch_response = self.client.query_batch_points(
                     collection_name=self.collection_name,
-                    prefetch=[
-                        qmodels.Prefetch(
+                    requests=[
+                        qmodels.QueryRequest(
                             query=dense_vec,
                             using="dense",
-                            limit=limit * 2,
-                            filter=query_filter
+                            limit=candidate_limit,
+                            filter=query_filter,
+                            with_payload=True,
                         ),
-                        qmodels.Prefetch(
+                        qmodels.QueryRequest(
                             query=sparse_vec,
                             using="sparse",
-                            limit=limit * 2,
-                            filter=query_filter
-                        )
+                            limit=candidate_limit,
+                            filter=query_filter,
+                            with_payload=True,
+                        ),
                     ],
-                    query=qmodels.FusionQuery(fusion=qmodels.Fusion.RRF),
-                    limit=limit
                 )
+
+                dense_pts = {str(p.id): p for p in batch_response[0].points}
+                sparse_pts = {str(p.id): p for p in batch_response[1].points}
+
+                sparse_scores = [p.score for p in sparse_pts.values() if p.score is not None]
+                max_sparse = max(sparse_scores, default=1.0)
+                if max_sparse <= 0.0:
+                    max_sparse = 1.0
+
+                all_uids = set(dense_pts.keys()).union(sparse_pts.keys())
+                scored_results: List[VectorSearchResult] = []
+
+                for uid in all_uids:
+                    d_score = max(0.0, min(1.0, float(dense_pts[uid].score))) if uid in dense_pts and dense_pts[uid].score is not None else 0.0
+                    s_score = (float(sparse_pts[uid].score) / max_sparse) if uid in sparse_pts and sparse_pts[uid].score is not None and max_sparse > 0 else 0.0
+                    s_score = max(0.0, min(1.0, s_score))
+
+                    if uid in dense_pts and uid in sparse_pts:
+                        final_score = alpha * d_score + (1.0 - alpha) * s_score
+                    elif uid in dense_pts:
+                        final_score = alpha * d_score
+                    else:
+                        final_score = (1.0 - alpha) * s_score
+
+                    final_score = max(0.0, min(1.0, final_score))
+                    pt = dense_pts.get(uid) or sparse_pts.get(uid)
+                    payload = (pt.payload or {}) if pt else {}
+
+                    scored_results.append(
+                        VectorSearchResult(
+                            id=uid,
+                            score=round(final_score, 4),
+                            payload=payload,
+                        )
+                    )
+
+                scored_results.sort(key=lambda r: r.score, reverse=True)
+                return scored_results[:limit]
             else:
                 response = self.client.query_points(
                     collection_name=self.collection_name,
                     query=dense_vec,
                     using="dense",
                     query_filter=query_filter,
-                    limit=limit
+                    limit=limit,
+                    with_payload=True,
                 )
-
-            results: List[VectorSearchResult] = []
-            for pt in response.points:
-                results.append(
-                    VectorSearchResult(
-                        id=str(pt.id),
-                        score=float(pt.score) if pt.score is not None else 0.0,
-                        payload=pt.payload or {}
+                results: List[VectorSearchResult] = []
+                for pt in response.points:
+                    raw_score = float(pt.score) if pt.score is not None else 0.0
+                    clamped_score = max(0.0, min(1.0, raw_score))
+                    results.append(
+                        VectorSearchResult(
+                            id=str(pt.id),
+                            score=round(clamped_score, 4),
+                            payload=pt.payload or {},
+                        )
                     )
-                )
-            return results
+                return results
         except Exception as e:
             logger.error(f"Error searching Qdrant collection '{self.collection_name}': {e}")
             return []

docs/TEST_COVERAGE.md

@@ -1,4 +1,4 @@
-# Test Coverage Report: ContextCortex (v2.13.0)
+# Test Coverage Report: ContextCortex (v2.14.0)
 
 This document provides comprehensive test coverage metrics and verification baselines for ContextCortex following the modular architectural restructuring, Codebase Navigator implementation, and test suite expansion.
 

docs/guide/user-guide.md

@@ -79,7 +79,7 @@ Follow these steps to test search retrieval:
 ### Interpreting Search Results
 
 Each result card displays:
-- **RRF Score**: Combined score calculated from Dense Cosine similarity and BM25 rank.
+- **Relevance Score**: Combined score calculated from Dense Cosine similarity and BM25 keyword matching.
 - **Source Link**: Clickable permalink directly to the file and line range in the upstream Git provider.
 - **Syntax Preview**: Code block with syntax highlighting and line numbers.
 

docs/reference/rest-api.md

@@ -16,7 +16,7 @@ All administrative routes are prefixed with `/admin/api`.
   ```json
   {
     "status": "ok",
-    "version": "2.13.0",
+    "version": "2.14.0",
     "database": "connected",
     "vector_store": "healthy"
   }

docs/requirements/index.md

@@ -1,6 +1,6 @@
 # Software Requirements Specification (SRS)
 
-This document establishes the Software Requirements Specification for ContextCortex (version 2.13.0).
+This document establishes the Software Requirements Specification for ContextCortex (version 2.14.0).
 
 This specification is written in accordance with the **ASD-STE100 Simplified Technical English (Issue 9)** standard and ISO/IEC/IEEE 29148 requirements engineering standards.