Skip to content

feat(rag): opt-in experimental Hybrid ask with evidence gate - #84

Merged
Air000000 merged 1 commit into
mainfrom
codex/hybrid-ask-runtime
Oct 16, 2026
Merged

Air000000 merged 1 commit into
mainfrom
codex/hybrid-ask-runtime

Conversation

@Air000000

Copy link
Copy Markdown
Owner

Summary

  • Add an explicitly experimental Hybrid /rag/ask path. Dense remains default; Ticket Agent and the original Dense 0.9 distance policy are unchanged.
  • Require both server switches (RAG_HYBRID_SEARCH_ENABLED, new default-false RAG_HYBRID_ASK_ENABLED) plus request retrieval_mode=hybrid and refusal_policy=v5_3_experimental. Reject max_distance on Hybrid rather than silently applying or ignoring a distance threshold. Hybrid defaults to Top14; Dense remains default3/max10.
  • Reuse existing Hybrid retrieval and frozen v5.3 pure prompt/strict original-line coverage validators without modifying experiment sources or reading evaluation data. Dedicated Phase B credentials are mandatory for runtime classification, without credential fallback.
  • Insufficient/conflicting/conditional/ambiguous evidence returns a fixed refusal without generation. Admitted requests generate from the complete ordered scoped context; require canonical source references drawn from gate-supported IDs. This checks provenance/format, not semantic truth.
  • Recheck SQL visibility/content before classification/generation and before publishing. Treat provider, schema, truncation, empty-output, citation and visibility failures as sanitized 503 + failed audit, never normal refusal or silent Dense fallback.
  • Preserve nullable Dense distances and separate rerank scores; return/audit experimental policy and validated gate details on successful/normal-refusal outcomes. Add refusal-state filters and complete status-count metrics. No SQL migration.

Verification

  • 360 offline tests passed: focused CI suite + auth. New cases cover gating/config, exact frozen prompt and line presentation, tenant/category scope, null and high Dense distances without threshold reuse, unsupported/ambiguous gate outcomes, generation refusal, malformed/truncated/provider-error responses, citation ID restrictions, concurrent document changes and audit/status metrics.
  • Real OpenAI-compatible SDK with mocked transports; isolated databases and external network guard. No real model calls, no new fee authorization consumed, no DEV access or public deployment.
  • ruff check ., in-memory Python compilation and git diff --check passed. Primary checkout's existing changes left untouched.

Explicit boundaries

This is runtime wiring for an off-by-default experiment, NOT v5.3 promotion or independent semantic-quality acceptance. Historical NO_PROMOTION / FAIL conclusions are unchanged. Valid evidence locators/source IDs do not prove entailment, faithful generation or prompt-injection resistance. It does not replace default refusal or create tickets.

Actual provider execution of the newly added classifier/generator path and controlled answer/refusal acceptance remain pending separate data-egress/cost authorization. The previous 8 embedding + 3 rerank smoke authorization is not reused.

@Air000000
Air000000 merged commit 73174a0 into main Oct 16, 2026
1 check passed
@Air000000
Air000000 deleted the codex/hybrid-ask-runtime branch October 16, 2026 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant