feat(routing): unified cross-primitive identity & collision policy (#671)#700
Merged
Conversation
) Add contextweaver.routing.primitive_id as the single source of truth for identifying MCP tools, resources, and prompts in one shared Catalog — the foundation for routing resources/prompts through the gateway (#555). - Tools keep their bare canonical tool_id; resources/prompts get disjoint-by-construction ids via a reserved kind:: prefix, validated through the shared tool_id grammar helpers. - Per-kind stable shape hashes (resource over URI; prompt over name + sorted argument names) with domain separation. - Deterministic ~N collision policy (resolve_collisions). - Re-exported from routing/__init__; documented in gateway_spec.md §9; public-API manifest regenerated. https://claude.ai/code/session_01MLuDdqTjfyP3AuQ4WeeGum
…669, #670) Extend the gateway's bounded-choice + firewall treatment from tools to MCP resources and prompts (#555), on the shared identity substrate from #671. - PrimitiveGatewayRuntime + PrimitiveUpstream protocol (adapters/ gateway_primitives.py) own per-kind routing indices (adapters/ _primitive_index.py) and share the tool runtime's ContextManager so reads land in one artifact store / firewall / tool_view surface. - adapters/mcp_primitives.py converts resources/list + prompts/list entries to resource/prompt SelectableItems and wraps resources/read + prompts/get results as firewall-ready envelopes; prompt arguments become an args_schema. - SelectableItem/ChoiceCard kind sets gain 'resource' and 'prompt'; schemas and public-API manifest regenerated. https://claude.ai/code/session_01MLuDdqTjfyP3AuQ4WeeGum
…#670) Surface MCP resources and prompts through the bounded-choice gateway as four distinct meta-tools (resource_browse / resource_read / prompt_browse / prompt_get) in adapters/mcp_gateway_primitives.py, dispatching to PrimitiveGatewayRuntime and packaging responses via the shared envelope_call_result. McpGatewayServer gains an optional primitive_runtime= and advertises + routes the four meta-tools over stdio when supplied, sharing the tool runtime's ContextManager for a unified artifact/tool_view surface. New public symbols are imported from their submodules (mirroring McpGatewayServer) to keep adapters/__init__.py within the module-size convention. https://claude.ai/code/session_01MLuDdqTjfyP3AuQ4WeeGum
…673) - AGENTS.md: module-map rows + Key Types for primitive_id, mcp_primitives, gateway_primitives, _primitive_index, mcp_gateway_primitives, and the PrimitiveGatewayRuntime / PrimitiveUpstream types (#672; gateway_spec.md §9 added earlier in the series). - benchmarks/primitive_gateway_benchmark.py: non-gating mixed-primitive (resources+prompts) benchmark measuring bounded-surface token savings and recall@k, with a test that verifies the harness on a small catalog (#673). - examples/mcp_primitives_demo.py: runnable end-to-end demo of the four resource/prompt meta-tools, wired into the make example target. - llms.txt regenerated. https://claude.ai/code/session_01MLuDdqTjfyP3AuQ4WeeGum
There was a problem hiding this comment.
Pull request overview
This PR extends the MCP gateway beyond tools to also support resources and prompts, introducing a unified cross-primitive ID scheme (routing/primitive_id.py) and a new PrimitiveGatewayRuntime that provides bounded browse + firewalled read/get meta-tools for these primitives.
Changes:
- Add
contextweaver.routing.primitive_idas the shared identity + hashing + collision-policy surface for tools/resources/prompts, and re-export it fromrouting/__init__.py. - Introduce a resource/prompt gateway runtime (
PrimitiveGatewayRuntime) + MCP meta-tools (resource_browse/read,prompt_browse/get) and wire them intoMcpGatewayServerbehind an optionalprimitive_runtime=. - Expand
kindenums and schemas to includeresourceandprompt, plus tests/benchmarks/docs/examples for the new capability.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_primitive_id.py | Adds unit coverage for primitive_id formatting/parsing/hashing/collision policy. |
| tests/test_primitive_gateway_benchmark.py | Validates the benchmark harness reports positive per-kind savings and expected recall. |
| tests/test_mcp_gateway_primitives.py | Exercises resource/prompt meta-tools end-to-end and server wiring (list_tools). |
| tests/test_gateway_primitives.py | Covers PrimitiveGatewayRuntime behaviors: browse/read/get, firewalled persistence, and error codes. |
| src/contextweaver/types.py | Extends SelectableItem.kind to include resource and prompt. |
| src/contextweaver/routing/primitive_id.py | Implements cross-primitive ID grammar + per-kind hash8 + collision policy utilities. |
| src/contextweaver/routing/init.py | Re-exports primitive_id public API from the routing package surface. |
| src/contextweaver/envelope.py | Extends ChoiceCard.kind and CHOICE_CARD_KINDS to include resource/prompt. |
| src/contextweaver/adapters/mcp_primitives.py | Adds MCP resource/prompt → SelectableItem converters and read/get → ResultEnvelope wrappers. |
| src/contextweaver/adapters/mcp_gateway_server.py | Optionally advertises/dispatches primitive meta-tools when primitive_runtime is provided. |
| src/contextweaver/adapters/mcp_gateway_primitives.py | Defines/dispatches the four resource/prompt gateway meta-tools. |
| src/contextweaver/adapters/gateway_primitives.py | Implements PrimitiveGatewayRuntime (bounded browse + firewalled read/get + artifact persistence). |
| src/contextweaver/adapters/gateway_error.py | Extends GatewayErrorCode with RESOURCE_NOT_FOUND / PROMPT_NOT_FOUND. |
| src/contextweaver/adapters/_primitive_index.py | Adds a per-kind Catalog/Graph/Router wrapper used by PrimitiveGatewayRuntime. |
| schemas/choice_card.schema.json | Adds resource/prompt to ChoiceCard schema enum. |
| schemas/catalog.schema.json | Adds resource/prompt to catalog item-kind enum. |
| Makefile | Includes the new primitives demo in make example. |
| examples/mcp_primitives_demo.py | Demonstrates resource/prompt meta-tools end-to-end via an in-process upstream stub. |
| docs/schemas/v0/choice_card.schema.json | Updates published v0 schema enum for ChoiceCard kinds. |
| docs/schemas/v0/catalog.schema.json | Updates published v0 schema enum for catalog item kinds. |
| docs/gateway_spec.md | Documents §9 cross-primitive identity + hashing + collision handling. |
| CHANGELOG.md | Announces the new primitives gateway runtime and identity module. |
| benchmarks/primitive_gateway_benchmark.py | Adds a mixed-primitive benchmark harness (non-gating) for surface savings/recall@k. |
| api/public_api.txt | Regenerates public API manifest for new exports and updated kind literals. |
| AGENTS.md | Updates module map and adapter/gateway documentation entries for the new modules. |
…licy docs Address Copilot review comments on #700 (#671 cross-primitive identity): - mcp_resource_read_to_envelope: base64-decode resource blob parts back to their original bytes before persisting, so tool_view drilldown on binary resources stays byte-accurate; malformed blobs fall back to raw bytes. - PrimitiveIndex.browse: reject non-integer / non-positive top_k with a structured GatewayError(ARGS_INVALID) instead of crashing with TypeError across the meta-tool boundary. - resolve_collisions docs + gateway_spec.md §9: state determinism is for a given catalog order (index-based), not order-independent; document the ~N form as an opaque catalog key outside the §1.1 grammar (does not round-trip). - Collision tests use canonical 8-hex-char ids; add regression tests for the base64 decode and top_k guard. https://claude.ai/code/session_012vcCws2x39jWHSB8sHxqHa
…primitive_id mypy >= 2.0 narrows the kind prefix after the guard, making the `# type: ignore[assignment]` an unused-ignore error under the CI mypy (2.1.0) even though local mypy 1.x accepted it — this was failing the test job's type-check step on the PR. Use an explicit cast(PrimitiveKind, ...) instead, which is correct under both mypy versions (warn_redundant_casts is off) and keeps the assignment type-safe. https://claude.ai/code/session_012vcCws2x39jWHSB8sHxqHa
Benchmark delta (vs
|
| size | recall@k (head Δ vs base) | MRR (head Δ vs base) | p99 (ms) |
|---|---|---|---|
| 50 | ✅ 0.5649 (+0.0000) | ✅ 0.4978 (+0.0000) | ✅ 0.672 (base 0.759) |
| 83 | ✅ 0.3825 (+0.0000) | ✅ 0.3242 (+0.0000) | ✅ 0.728 (base 1.134) |
| 1000 | ✅ 0.1475 (+0.0000) | ✅ 0.1456 (+0.0000) | ✅ 34.805 (base 41.711) |
Per-backend × per-size matrix
| backend | size | recall@k (Δ) | MRR (Δ) | p99 (ms) |
|---|---|---|---|---|
| bm25 | 100 | ✅ 0.3825 (+0.0000) | ✅ 0.3399 (+0.0000) | ✅ 5.823 (base 8.140) |
| bm25 | 500 | ✅ 0.2250 (+0.0000) | ✅ 0.2165 (+0.0000) | ✅ 28.160 (base 38.989) |
| bm25 | 1000 | ✅ 0.1575 (+0.0000) | ✅ 0.1525 (+0.0000) | ✅ 85.022 (base 111.716) |
| embedding_hashing | 100 | ✅ 0.5175 (+0.0000) | ✅ 0.4360 (+0.0000) | ✅ 8.217 (base 7.225) |
| embedding_hashing | 500 | ✅ 0.2700 (+0.0000) | ✅ 0.2674 (+0.0000) | ✅ 41.523 (base 44.182) |
| embedding_hashing | 1000 | ✅ 0.2000 (+0.0000) | ✅ 0.1931 (+0.0000) | ✅ 102.781 (base 98.277) |
| embedding_st | 100 | skipped (skipped: missing sentence-transformers) | — | — |
| embedding_st | 500 | skipped (skipped: missing sentence-transformers) | — | — |
| embedding_st | 1000 | skipped (skipped: missing sentence-transformers) | — | — |
| fuzzy | 100 | skipped (skipped: missing rapidfuzz) | — | — |
| fuzzy | 500 | skipped (skipped: missing rapidfuzz) | — | — |
| fuzzy | 1000 | skipped (skipped: missing rapidfuzz) | — | — |
| tfidf | 100 | ✅ 0.3825 (+0.0000) | ✅ 0.3220 (+0.0000) | ✅ 0.979 (base 1.102) |
| tfidf | 500 | ✅ 0.2325 (+0.0000) | ✅ 0.2314 (+0.0000) | ✅ 11.989 (base 11.492) |
| tfidf | 1000 | ✅ 0.1475 (+0.0000) | ✅ 0.1456 (+0.0000) | ✅ 36.155 (base 50.755) |
Context pipeline (per scenario)
| scenario | tokens | dropped | dedup |
|---|---|---|---|
| large_catalog | 1480 (base 1514, Δ-34) | 0 (base 0, Δ+0) | 0 (base 0, Δ+0) |
| long_conversation | 2500 (base 2548, Δ-48) | 0 (base 0, Δ+0) | 0 (base 0, Δ+0) |
| mixed_payload | 488 (base 497, Δ-9) | 0 (base 0, Δ+0) | 0 (base 0, Δ+0) |
| short_conversation | 487 (base 496, Δ-9) | 0 (base 0, Δ+0) | 0 (base 0, Δ+0) |
| stress_conversation | 6590 (base 6651, Δ-61) | 11 (base 7, Δ+4) | 4 (base 4, Δ+0) |
| tiny_payload | 256 (base 267, Δ-11) | 0 (base 0, Δ+0) | 0 (base 0, Δ+0) |
Numbers come from make benchmark / make benchmark-matrix.
Latency is hardware-dependent — treat the markers as a rough guide.
See benchmarks/scorecard.md for the full picture.
Phase 3 audit follow-ups (no behavior change): - Parenthesize the _validate() schema-empty check in gateway_primitives for unambiguous and/or precedence. - Drop the unused reserved `runtime` param from make_primitive_meta_tools() (definitions are static; dispatch binds the runtime separately) and update the server, demo, and test call sites. https://claude.ai/code/session_012vcCws2x39jWHSB8sHxqHa
This was referenced Jun 15, 2026
dgenio
added a commit
that referenced
this pull request
Jun 15, 2026
#672, #673) (#701) * feat(gateway): make resources/prompts reachable end-to-end (#669, #670, #672, #673) PR #700 landed the resource/prompt gateway runtime, meta-tools, converters, and benchmark, but the feature was library-only: there was no shipped PrimitiveUpstream adapter and `contextweaver mcp serve` never wired a primitive runtime. This closes those gaps. - adapters/mcp_primitive_upstream.py: ship StubPrimitiveUpstream, McpClientPrimitiveUpstream, and MultiplexPrimitiveUpstream, mirroring the tool mcp_upstream trio. Per the PrimitiveUpstream contract these raise transport errors so the runtime classifies them via classify_upstream_exception. - _mcp_cli.py: `mcp serve --gateway` now builds a PrimitiveGatewayRuntime from a snapshot catalog's optional `resources` / `prompts` lists (sharing the tool runtime's ContextManager) and passes it to McpGatewayServer; tools-only catalogs are unchanged. Factored a shared `_parse_catalog_file` helper. The serve summary reports primitive counts. - gateway_primitives.py: add resource_ids() / prompt_ids() accessors mirroring ProxyRuntime.list_tool_ids(). - Makefile: add `benchmark-primitives` target for the mixed-primitive benchmark. - docs/gateway_spec.md: add §9.4 request flows and §9.5 serve/catalog wiring. - Tests: test_mcp_primitive_upstream.py + serve-CLI wiring tests. https://claude.ai/code/session_01FSXGXXiPxXckah5iFwa7ng * fix(gateway): make multiplex primitive listings idempotent and unwrap dict-shaped results Address Copilot review on #701: - MultiplexPrimitiveUpstream.list_resources/list_prompts now clear their ownership index at the start of each build, so repeated listings (e.g. successive PrimitiveGatewayRuntime.refresh() calls) are idempotent rather than returning an empty union on the second call and erasing the catalog. - McpClientPrimitiveUpstream.list_resources/list_prompts route through a new _unwrap_listing helper that handles pydantic-result, dict-shaped ({"resources": [...]}), and bare-list payloads, so a dict listing no longer iterates string keys into _model_to_dict. - Tests: multiplex idempotent-listing + repeated-refresh, and client dict-shaped listing unwrap. https://claude.ai/code/session_01FSXGXXiPxXckah5iFwa7ng * fix(gateway): warn when malformed snapshot-catalog primitive entries are skipped _load_primitive_defs_from_catalog silently filtered out non-dict resources/prompts entries, so a mistyped catalog entry vanished without a trace. Factor the per-kind filtering into _collect_primitive_defs, which now also drops dict entries lacking the required identity field (uri for resources, name for prompts) and logs a warning for every skipped entry. Adds test_load_primitive_defs_skips_malformed_entries covering non-dict and identity-less entries plus the warning count. https://claude.ai/code/session_01J3qykQ9umrpbdy4n5gKq6c --------- Co-authored-by: Claude <noreply@anthropic.com>
This was referenced Jun 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add contextweaver.routing.primitive_id as the single source of truth for
identifying MCP tools, resources, and prompts in one shared Catalog — the
foundation for routing resources/prompts through the gateway (#555).
disjoint-by-construction ids via a reserved kind:: prefix, validated
through the shared tool_id grammar helpers.
sorted argument names) with domain separation.
public-API manifest regenerated.
https://claude.ai/code/session_01MLuDdqTjfyP3AuQ4WeeGum