Skip to content

[Global Ask] Verify KG, ontology, and semantic claims with public SearXNG evidence #272

Description

@seonghobae

Buyer problem

The Buyer-surface stack #258 → #260 → #261 → #262 → #263 → #264 already assembles authorized source text, persisted Knowledge Graph facts, project/role/Keyman semantic facts, contextual-orchestrator answers, TEPP run metadata, and fast-mlsirm calibrated reports. The integration is incomplete in four material ways:

  1. Global Ask loads persisted semantic/KG evidence only after lexical candidate selection, so a buyer term that exists only in project, responsibility, affiliation, Keyman, edge type, endpoint label, or ontology IRI can miss the source entirely.
  2. The citation-bearing post-chat path requests contextual-orchestrator mode="route" although the accepted orchestration ADR reserves mode="verify" for checked, citation-bearing judgment.
  3. SearXNG is wired only to an admin relation-presence check; Global Ask cannot return FEVER-style supported / refuted / not_enough_information judgments for public KG/ontology/semantic claims.
  4. TEPP and fast-mlsirm artifacts can be confused with externally verifiable facts. Current TEPP main publishes an accepted-run contract, not a completed-result DTO; fast-mlsirm scores are calibrated measurement evidence, not web truth.

Required product slice

  • Add bounded, index-backed candidate nomination from persisted project, role/responsibility/affiliation, Keyman, KG edge, endpoint label, and ontology evidence.
  • Keep candidate IDs non-authoritative and re-run the existing source-post visibility/ABAC gate before any body or semantic evidence is exposed.
  • Change citation-bearing Global Ask/post chat to contextual-orchestrator mode="verify" with an explicit reasoning effort.
  • Add an opt-in verify_external Global Ask contract. Only claims backed by public source posts may leave the trust boundary. Never send private source text, raw source hints, credentials, PII, TEPP payloads, or fast-mlsirm respondent/item data to SearXNG.
  • Search a bounded number of public claims through configured SearXNG, then use contextual-orchestrator mode="verify" to classify each claim as supported, refuted, or not_enough_information against the retrieved evidence.
  • Keep external URLs separate from internal post citations and preserve provenance/truth status. Web evidence may corroborate or dispute a real-world claim; it must not promote an inferred edge to authoritative fact.
  • Surface verification status, rationale, and external citations in the Buyer Ask Agent with a clear next action.
  • Preserve the TEPP boundary: accepted transport evidence is not a completed measurement. Preserve the fast-mlsirm boundary: calibrated scores expose model/version/provenance but are never labeled web-verified.

TDD acceptance

  • semantic-only and KG/ontology-only terms nominate the authorized source;
  • private-only semantic facts never produce an external query;
  • external result URLs are bounded and reject search pages, localhost, and literal private-network hosts;
  • conflicting evidence can produce refuted; no usable evidence produces not_enough_information;
  • contextual-orchestrator requests contain literal "mode": "verify" for both citation chat and claim adjudication;
  • external citation IDs cannot become cited_post_ids;
  • TEPP accepted envelopes cannot become completed measurement claims;
  • fast-mlsirm report evidence cannot become externally verified facts;
  • frontend tests cover opt-in, unavailable state, three-way status, external links, and next-action copy;
  • changed production statement/branch coverage and public docstrings remain 100%.

References

  • Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018). FEVER: A large-scale dataset for fact extraction and verification. Proceedings of NAACL-HLT 2018, 809–819.
  • World Wide Web Consortium. (2013). PROV-O: The PROV ontology.
  • SearXNG. (2026). Search API documentation.

Stack gate

Implement as a draft PR stacked on #264. Do not merge or mark Ready until #264’s current cutoff-navigation review finding is repaired, the stack is semantically restacked on the current #258 head, all exact-head checks are terminal-success, and an independent current-head approval exists.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions