Merge pull request #160 from rootcodelabs/llm-403

Get update from llm-403 into llm-408

Author: nuwangeek

constants.ini

@@ -12,4 +12,5 @@ DOMAIN=localhost
 DB_PASSWORD=dbadmin
 RAG_SEARCH_RUUTER_PUBLIC_INTERNAL_SERVICE=http://ruuter:8086/services
 SERVICE_DMAPPER_HBS=http://data-mapper:3000/hbs/rag-search
-SERVICE_PROJECT_LAYER=services
+SERVICE_PROJECT_LAYER=services
+RAG_SEARCH_LLM_SERVICE=http://llm-orchestration-service:8100

docs/API_TOOL_CALLING.md

@@ -8,12 +8,12 @@ API Tool Calling enables the LLM module to discover and invoke external API endp
 in response to user queries. endpoints are
 registered, semantically indexed in Qdrant, and retrieved at query time using hybrid search.
 
-The feature has two halves:
 
-| Half | What it does | Status |
+| component | What it does | Status |
 |---|---|---|
-| **Indexing pipeline** | Takes an endpoint definition → enriches it with LLM context → stores hybrid vectors in Qdrant |  Complete |
-| **Tool classifier** | At query time, routes to the best matching endpoint via hybrid search |  In progress |
+| **Indexing pipeline** | Takes an endpoint definition → enriches it with LLM context → stores hybrid vectors in Qdrant | ✅ Complete |
+| **Tool classifier** | At query time, routes to the best matching endpoint via hybrid search + LLM disambiguation | ✅ Complete |
+| **Workflow executor** | Surfaces the matched endpoint; full agentic loop (param collection → API call) planned | 🔧 Partial (Task 10) |
 
 ---
 
@@ -30,7 +30,11 @@ main_indexer.py  (indexing pipeline)
         ↓  upsert
 api_tool_collection  (Qdrant)
         ↑  query at runtime
-ToolClassifier  (src/tool_classifier/)
+APISemanticSearcher  (src/tool_classifier/api_semantic_searcher.py)
+        ↑  called by
+ToolClassifier._try_api_tool_classification()
+        ↓  ClassificationResult(workflow=API_TOOL_CALLING)
+APIToolWorkflowExecutor  (src/tool_classifier/workflows/api_tool_workflow.py)
 ```
 
 ---
@@ -97,7 +101,7 @@ Defined in [DSL/CronManager/script/api_tool_indexer.sh](../DSL/CronManager/scrip
 
 **What it does (in order):**
 
-1. Validates required env vars (`endpoint_id`, `name`, `description`)
+1. Validates required env vars (`endpoint_id`, `name`, `description`, `url`)
 2. Activates the pre-built Python venv at `/app/python_virtual_env`
 3. Installs required packages via `uv pip install` (`httpx`, `pydantic`, `qdrant-client`, `loguru`)
 4. Sets `PYTHONPATH` to include `/app/src`
@@ -235,4 +239,192 @@ loop can execute the API call without an additional database round-trip.
 | `required` | bool | Whether the caller must supply this param |
 | `description` | str | Human-readable description |
 
+---
+
+
+## Part 2 — Tool Classifier (Query-Time)
+
+### Overview
+
+At query time, `ToolClassifier` in [src/tool_classifier/classifier.py](../src/tool_classifier/classifier.py) the layer by layer execution happens 
+
+
+1. **Service search** → `intent_collections` (Qdrant) — existing Bürokratt services
+2. **API Tool search** → `api_tool_collection` (Qdrant) — registered API tool endpoints
+
+API tool search (`_try_api_tool_classification`) is triggered when:
+- `SERVICE_WORKFLOW_ENABLED=false` (service workflow disabled globally)
+- Dense service search returns no results
+- Service cosine score falls below `DENSE_MIN_THRESHOLD`
+
+It is **always** tried before falling back to Context/RAG.
+
+---
+
+### Component: `APISemanticSearcher`
+
+Defined in [src/tool_classifier/api_semantic_searcher.py](../src/tool_classifier/api_semantic_searcher.py).
+
+Instantiated once in `ToolClassifier.__init__()` and reuses the shared Qdrant `httpx.AsyncClient`.
+
+**Constructor:**
+
+```python
+APISemanticSearcher(
+    embedding_service=orchestration_service,   # generates dense embeddings
+    qdrant_client=self._qdrant_client,         # shared connection pool
+    disambiguator=None,                        # optional: inject for testing
+)
+```
+
+**Key constants** (from `constants.py`):
+
+| Constant | Value | Purpose |
+|---|---|---|
+| `API_TOOL_COLLECTION` | `api_tool_collection` | Qdrant collection name |
+| `API_TOOL_SEARCH_TOP_K` | `5` | Max hybrid results |
+| `API_TOOL_MIN_THRESHOLD` | cosine threshold | Below this → no match |
+| `API_TOOL_HIGH_CONFIDENCE_THRESHOLD` | cosine threshold | Above this → high confidence |
+| `API_TOOL_SCORE_GAP_THRESHOLD` | gap threshold | Minimum lead over runner-up |
+
+---
+
+### Search Flow: `APISemanticSearcher.search()`
+
+```
+User query
+    │
+    ├─ precomputed_embedding provided? → reuse it (no extra API call)
+    └─ otherwise → generate dense embedding via embedding_service
+    │
+    ▼
+Step 1: Dense search (api_tool_collection)
+    → Real cosine similarity scores per endpoint
+    │
+    ├─ No results → return []
+    ├─ top_cosine < API_TOOL_MIN_THRESHOLD → return []
+    └─ continue
+    │
+    ▼
+Step 2: Hybrid search (dense + sparse/BM25 + RRF)
+    → Best-ranked results by RRF fusion score
+    │  Falls back to dense results if hybrid returns nothing
+    │
+    ▼
+Step 3: Annotate confidence for each hybrid result
+    │
+    │  cosine lookup: dense_cosine_map[endpoint_id]
+    │             └─ fallback: point["cosine_score"] (sparse-driven result)
+    │             └─ skip if neither available
+    │
+    │  effective_gap = this_cosine − best_other_cosine_in_dense
+    │
+    ├─ i==0 AND cosine ≥ HIGH_THRESHOLD AND effective_gap ≥ GAP_THRESHOLD → "high"
+    ├─ cosine ≥ MIN_THRESHOLD → "medium"
+    └─ else → skip
+    │
+    ▼
+Step 4: Resolve to exactly one result
+    ├─ high-confidence result exists → return immediately
+    ├─ single medium + large gap → return directly
+    └─ multiple medium OR small gap → LLM disambiguation
+           │
+           └─ EndpointDisambiguatorModule (DSPy + asyncio.to_thread)
+                  → picks winner or returns None
+                  → None means no match → return []
+```
+
+---
+
+### Embedding Reuse
+
+When `ToolClassifier.classify()` already generated a dense embedding for the service
+search, it passes it as `precomputed_embedding` to `_try_api_tool_classification`:
+
+```python
+api_tool_result = await self._try_api_tool_classification(
+    query, request, precomputed_embedding=query_embedding
+)
+```
+
+`APISemanticSearcher.search()` skips the embedding step entirely when this is provided,
+saving one embedding API call per request.
+
+---
+
+### LLM Disambiguation: `EndpointDisambiguatorModule`
+
+Used when multiple medium-confidence endpoints score similarly and no clear winner
+can be determined from cosine scores alone.
+
+- DSPy `Predict` module with `EndpointDisambiguationSignature`
+- Inputs: `user_query` + `candidates` (JSON list of `{endpoint_id, name, description, cosine_score}`)
+- Output: `best_endpoint_id` — the winning `endpoint_id`, or `"none"` if no match
+- Run via `asyncio.to_thread()` to avoid blocking the async event loop
+- Understands Estonian, Russian, and English queries
+
+---
+
+### Feature Flag
+
+API tool calling is gated by `FeatureFlags.API_TOOL_CALLING_WORKFLOW_ENABLED`.
+When `false`, `_try_api_tool_classification` returns `None` immediately without
+touching Qdrant.
+
+---
+
+### Component: `APIToolWorkflowExecutor`
+
+Defined in [src/tool_classifier/workflows/api_tool_workflow.py](../src/tool_classifier/workflows/api_tool_workflow.py).
+
+Handles `WorkflowType.API_TOOL_CALLING` after `ToolClassifier.classify()` has set
+`matched_endpoint` in the context dict.
+
+**Current behaviour (Task 4.1):**
+
+Reads `context["matched_endpoint"]` and returns a simple confirmation response:
+
+```
+**{name}**: {description}
+
+URL: {url}
+```
+
+**Planned (Task 10):** Full agentic loop —
+session management → parameter collection dialog → external API call → response formatting.
+
+---
+
+### End-to-End Flow (Query Time)
+
+```
+User: "What are the public holidays in Estonia?"
+    │
+    ▼
+ToolClassifier.classify()
+    │
+    ├─ Dense search (intent_collections) → low cosine → below threshold
+    │
+    └─ _try_api_tool_classification()
+           │
+           └─ APISemanticSearcher.search()
+                  │
+                  ├─ Dense: get_national_holidays cosine=0.87
+                  ├─ Hybrid: get_national_holidays ranked #1 (RRF)
+                  ├─ effective_gap large → confidence="high"
+                  └─ return [APIToolSearchResult(name="get_national_holidays", ...)]
+           │
+           └─ ClassificationResult(
+                  workflow=API_TOOL_CALLING,
+                  metadata={"matched_endpoint": {...}}
+              )
+    │
+    ▼
+ToolClassifier._execute_with_fallback_async()
+    │
+    └─ APIToolWorkflowExecutor.execute_async(context={"matched_endpoint": {...}})
+           │
+           └─ OrchestrationResponse(content="**get_national_holidays**: ...")
+```
+
 ---

src/api_tool_indexer/constants.py

@@ -30,8 +30,19 @@ class ApiToolIndexerConstants:
     RETRY_DELAY_BASE = 2  # Exponential backoff base (2^attempt seconds)
     REQUEST_TIMEOUT = 60  # seconds
 
+    # Number of example queries generated per endpoint.
+    # Each example becomes its own Qdrant point so its vector sits in the exact
+    # language region of the embedding space, enabling short-query matching.
+    EXAMPLE_QUERY_COUNT = 5
+
     # Context Enrichment Template
-    # Used to generate a rich semantic context for each endpoint before embedding
+    # Full template goes in chunk_prompt; document_prompt is left empty.
+    #
+    # Multi-point indexing strategy:
+    #   - Each example query line is extracted and stored as its own Qdrant point,
+    #     embedded from that individual sentence alone.
+    #   - The prose + all examples combined become one summary point.
+    # All in the same language as the endpoint description — no bilingual duplication.
     CONTEXT_TEMPLATE = """<document>
 {full_endpoint_info}
 </document>
@@ -44,12 +55,23 @@ class ApiToolIndexerConstants:
 </endpoint>
 
 Please generate a rich, detailed context that describes this API endpoint comprehensively for semantic search.
-Include information about:
+Keep the prose context general and country-agnostic. Include information about:
 - What the user wants to accomplish by calling this endpoint
 - Key terms and synonyms for this action
 - Related concepts and use cases
 - Common ways users might ask for this functionality in natural language
 
-IMPORTANT: Generate the context in the SAME LANGUAGE as the endpoint description above. If the description is in Estonian, respond in Estonian. If in English, respond in English. If in Russian, respond in Russian.
+IMPORTANT: Generate the prose context and the example questions in the SAME LANGUAGE as the endpoint description above. However, always use the exact section header "Example queries:" in English regardless of language — this is a required machine-readable marker.
+
+IMPORTANT for example queries: This is a system built for Estonian government digital services (Bürokratt). Ground the examples in an Estonian context — use Estonian cities (Tallinn, Tartu, Pärnu, Narva), Estonian institutions, and Estonia-relevant scenarios. Only use non-Estonian locations if the endpoint is explicitly about comparing or fetching data for multiple countries.
+
+Then add a section with exactly {example_count} realistic and diverse example questions a real user might ask when they need this endpoint. Cover different phrasings, synonyms, and indirect ways of asking — do not just repeat the description verbatim.
+
+Example queries:
+- <example question 1>
+- <example question 2>
+- <example question 3>
+- <example question 4>
+- <example question 5>
 
-Answer only with the enriched context and nothing else."""
+Answer only with the enriched context and example queries — nothing else."""