docs(skill): prefer MCP server / CLI over grep for external-agent search - #265
Open
sebastianbraun25 wants to merge 10 commits into
Open
docs(skill): prefer MCP server / CLI over grep for external-agent search#265sebastianbraun25 wants to merge 10 commits into
sebastianbraun25 wants to merge 10 commits into
Conversation
added 10 commits
August 28, 2026 16:25
Adds a dependency-free BM25 full-text index (openkb/fulltext_index.py) over concepts/entities/summaries pages, exposed as a new search_wiki tool alongside index.md-driven navigation in build_query_agent. Additive hybrid retrieval: surfaces pages whose one-line index summary omits a buried detail, without replacing existing navigation. Resolves VectifyAI#233.
…axonomy accessors - fulltext_index.py: extract shared _BM25Scorer from WikiFullTextIndex (no behavior change), add Locator (line/page) on SearchHit, add TieredWikiSearch with three independent tiers over summaries/ (briefs + full body) and sources/ (whole-file .md + per-page PageIndex .json, never the whole long doc as one BM25 unit). - frontmatter.py: add resolve_description()/body_only() shared helpers (kept separate from agent.compiler._resolve_description, which is under active unrelated development). - agent/tools.py: add list_taxonomy_items()/get_taxonomy_item() for semantic browsing of persisted concepts/entities (pending candidates in PendingTopicsStore are structurally excluded). - No wiring into CLI/MCP/query-agent yet (follow-up PRs); WikiFullTextIndex and agent.tools.search_wiki keep their existing signature/behavior.
…o query/chat agent - cli.py: new 'openkb list-taxonomy [--kind concept|entity] [--json]' and 'openkb search <query> [--scope briefs,summaries,sources] [--top-k N] [--json]' commands. - agent/tools.py: search_wiki now searches the new tiered briefs/summaries/sources index instead of the old combined concepts+entities+summaries index (concepts/entities move to the new list_taxonomy tool - semantic browsing, not keyword search); new list_taxonomy() text-formatting wrapper over list_taxonomy_items(). - agent/query.py (+ chat.py via tool inheritance): wires list_taxonomy and the retiered search_wiki in as agent tools; search strategy instructions updated to browse taxonomy first, then use scope-restricted search_wiki as a keyword fallback. - README.md: updated hybrid-retrieval paragraph and command table. - Intentional behavior change to agent.tools.search_wiki (scope param, concepts/entities out of scope, output grouped by tier) - safe since VectifyAI#234/VectifyAI#259, which introduced it, are not yet merged upstream; existing tests updated to the new contract.
- New openkb/mcp_server.py: FastMCP server with list_taxonomy and search_wiki tools, thin wrappers over agent.tools.list_taxonomy_items and fulltext_index.TieredWikiSearch. No index cache (rebuilt fresh per call, same as the CLI/agent). find_kb_dir() mirrors cli.py's KB resolution as a lightweight standalone copy so starting the MCP server doesn't pull in cli.py's much heavier import chain. - pyproject.toml: add explicit 'mcp==1.27.1' pin (already resolved transitively via openai-agents/PR VectifyAI#207 for MCP client support; this formalizes the now-also-server-side usage) and a new 'openkb-mcp' console script entry point. uv.lock regenerated accordingly. - README.md: new 'Using with an MCP client' section with a sample mcpServers config. - tests/test_mcp_server.py: KB resolution (cwd walk, global default fallback, not-found), list_taxonomy/search_wiki tool behavior, scope validation.
- skills/openkb/SKILL.md: 'See what's available' now leads with list_taxonomy (MCP) / 'openkb list-taxonomy' (CLI) before falling back to reading the full index.md. - 'Read content' table adds search_wiki (MCP) / 'openkb search' (CLI) rows ahead of the existing grep fallback, with a note on why BM25 ranking beats raw grep occurrence count. - 'When the KB doesn't have the answer' and the openkb-query guidance updated to reference the new search options alongside grep. - Documentation-only change; no behavior change to the underlying tools/CLI/MCP server (VectifyAI#259/VectifyAI#261/VectifyAI#263).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This PR was created in collaboration between a human and AI: implementation, tests, and
PR text were created by an AI assistant under the guidance and review of the human author.
Problem
skills/openkb/SKILL.mdcurrently tells external agents (GitHub Copilot, Claude Code,Cursor, etc.) to fall back to
grep -rfor keyword search and to readwiki/index.mdinfull for taxonomy/document discovery. Neither is ideal at scale:
grephas no relevanceranking (a document that merely repeats a generic word often outranks one actually about
the topic), and reading the full
index.mdcosts an increasing number of tokens as the KBgrows, with no guarantee the agent's attention lands on the one-liner that actually matches.
Since #260/#261 (CLI) and #262/#263 (MCP server), the same taxonomy-browsing and
tiered-search capability the internal
openkb query/openkb chatagent uses is alsoreachable from outside the OpenKB process — but the skill didn't mention either yet.
Solution / Changes
skills/openkb/SKILL.mdupdated, in order of preference:list_taxonomythen scope-restrictedsearch_wiki— mirrors theinternal agent's own search strategy exactly.
openkb list-taxonomy/openkb search(same underlyingfunctions, CLI-exposed).
wiki/index.mdin full /grep -r, asbefore.
Specifically:
list_taxonomy(MCP) /openkb list-taxonomy(CLI)ahead of reading the full
index.md.search_wiki(MCP) /openkb search(CLI) aheadof the existing
greprow, plus a short note on why BM25 ranking beats raw grepoccurrence count (with a concrete "case" vs. "in case of error" example).
openkb queryLLM-cost guidance updated toreference the new search options alongside grep.
Documentation-only change — no behavior change to the underlying tools/CLI/MCP server
(#259/#261/#263), so no test updates needed (no existing test reads this bundled
SKILL.md's content).Testing
pytestfull suite green — 1283 passed (18 pre-existing, environment-specific Windowsfailures unrelated to this change are unaffected and unchanged in count).
ruff/mypyare no-ops for a Markdown-only diff but were run for completeness.
Dependencies
Depends on #261(CLI) and#263(MCP server) — this PR documents both, so it onlymakes sense once both exist. The diff below includes both branches' commits (and, via
them, feat(search): tiered BM25 search (briefs/summaries/sources) + taxonomy accessors #259/feat(agent): hybrid BM25 search_wiki tool for query/chat agent #234); once feat(cli,agent): expose tiered search + taxonomy via CLI and wire into query/chat agent #261 and feat(mcp): expose taxonomy browsing + tiered search as an MCP server #263 merge first, this PR's diff will shrink to just this
PR's own commit.
Issues
Resolves #264