feat(exceptions): stable error codes, remediation hints, and error reference (#635, #637)#699
Merged
Merged
Conversation
…ference (#635, #637) Implements the "stable library error surface" group as one change. #635 — stable error codes + remediation hints: - Every ContextWeaverError subclass now carries a frozen class-level `code` (e.g. `CW_CONFIG`) so callers branch on failures without string-matching, and an optional instance `hint` (with a class-level `default_hint` fallback). - `__str__` renders `[code] message (hint: …)`, so existing CLI error rendering (`__main__.main`) and `typer.BadParameter(str(exc))` surface the code + hint automatically. - Six high-traffic errors ship default hints that link into the error reference (ConfigError, CatalogError, ItemNotFoundError, DeterminismError, PathInvalidError, BudgetOverflowError). - `tests/test_exceptions.py` freezes the codes against a golden list and asserts uniqueness, exhaustiveness, and str()/hint rendering — a rename or a new code-less exception now fails CI. #637 — error-reference page: - New `docs/errors.md`: a code index table plus a per-exception section (stable code, raising modules from a raise-site audit, common causes, fix), added to the mkdocs nav and cross-linked from the troubleshooting guide. - Registered in `scripts/gen_llms.py` and regenerated `llms.txt` / `llms-full.txt` so the page lands in the agent-facing bundle. Other: - Regenerated `api/public_api.txt` for the new `hint` constructor parameter. - AGENTS.md documents the code/hint convention and the new-exception rule. Out of scope (follow-up): threading library codes into the gateway `GatewayError` mapping (touches recently-hardened dispatch; not required by the acceptance criteria). https://claude.ai/code/session_01BeQyyFrL1PUhuDxa99sNuN
There was a problem hiding this comment.
Pull request overview
This PR stabilizes the library’s public error surface by adding machine-readable error codes and remediation hints to the ContextWeaverError hierarchy, and publishing a dedicated error-reference documentation page that those hints can link to.
Changes:
- Added stable per-exception
codevalues plus optionalhint/default_hintplumbing and updated__str__to render[code] message (hint: …). - Added tests that golden-list codes, enforce uniqueness/exhaustiveness via module introspection, and validate
str()/hint rendering. - Added and registered a new
docs/errors.mdreference page (plus nav + LLM bundle wiring) documenting codes, causes, and fixes.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
src/contextweaver/exceptions.py |
Introduces code/hint/default_hint on the exception hierarchy and updates __str__ formatting. |
tests/test_exceptions.py |
Adds golden-list and introspection-based tests to enforce stability and rendering of codes/hints. |
docs/errors.md |
New error-reference page indexing codes and documenting each exception with causes/fixes. |
docs/troubleshooting.md |
Cross-links troubleshooting guidance to the new error reference. |
mkdocs.yml |
Adds the error reference page to the documentation navigation. |
scripts/gen_llms.py |
Includes docs/errors.md in the generated LLM documentation bundle and index. |
llms.txt |
Updates the LLM doc index to mention the error reference page. |
llms-full.txt |
Regenerated bundle content to include the full docs/errors.md page. |
api/public_api.txt |
Regenerates the public API manifest to reflect new hint parameters/signatures. |
AGENTS.md |
Documents the new code/hint convention and the “new exception must add code+golden+docs” rule. |
CHANGELOG.md |
Notes the new stable error codes/hints and the new error reference page under Unreleased. |
Benchmark delta (vs
|
| size | recall@k (head Δ vs base) | MRR (head Δ vs base) | p99 (ms) |
|---|---|---|---|
| 50 | ✅ 0.5649 (+0.0000) | ✅ 0.4978 (+0.0000) | ✅ 0.412 (base 0.759) |
| 83 | ✅ 0.3825 (+0.0000) | ✅ 0.3242 (+0.0000) | ✅ 0.531 (base 1.134) |
| 1000 | ✅ 0.1475 (+0.0000) | ✅ 0.1456 (+0.0000) | ✅ 26.322 (base 41.711) |
Per-backend × per-size matrix
| backend | size | recall@k (Δ) | MRR (Δ) | p99 (ms) |
|---|---|---|---|---|
| bm25 | 100 | ✅ 0.3825 (+0.0000) | ✅ 0.3399 (+0.0000) | ✅ 4.570 (base 8.140) |
| bm25 | 500 | ✅ 0.2250 (+0.0000) | ✅ 0.2165 (+0.0000) | ✅ 22.293 (base 38.989) |
| bm25 | 1000 | ✅ 0.1575 (+0.0000) | ✅ 0.1525 (+0.0000) | ✅ 66.596 (base 111.716) |
| embedding_hashing | 100 | ✅ 0.5175 (+0.0000) | ✅ 0.4360 (+0.0000) | ✅ 6.017 (base 7.225) |
| embedding_hashing | 500 | ✅ 0.2700 (+0.0000) | ✅ 0.2674 (+0.0000) | ✅ 32.564 (base 44.182) |
| embedding_hashing | 1000 | ✅ 0.2000 (+0.0000) | ✅ 0.1931 (+0.0000) | ✅ 74.328 (base 98.277) |
| embedding_st | 100 | skipped (skipped: missing sentence-transformers) | — | — |
| embedding_st | 500 | skipped (skipped: missing sentence-transformers) | — | — |
| embedding_st | 1000 | skipped (skipped: missing sentence-transformers) | — | — |
| fuzzy | 100 | skipped (skipped: missing rapidfuzz) | — | — |
| fuzzy | 500 | skipped (skipped: missing rapidfuzz) | — | — |
| fuzzy | 1000 | skipped (skipped: missing rapidfuzz) | — | — |
| tfidf | 100 | ✅ 0.3825 (+0.0000) | ✅ 0.3220 (+0.0000) | ✅ 0.762 (base 1.102) |
| tfidf | 500 | ✅ 0.2325 (+0.0000) | ✅ 0.2314 (+0.0000) | ✅ 7.136 (base 11.492) |
| tfidf | 1000 | ✅ 0.1475 (+0.0000) | ✅ 0.1456 (+0.0000) | ✅ 26.333 (base 50.755) |
Context pipeline (per scenario)
| scenario | tokens | dropped | dedup |
|---|---|---|---|
| large_catalog | 1480 (base 1514, Δ-34) | 0 (base 0, Δ+0) | 0 (base 0, Δ+0) |
| long_conversation | 2500 (base 2548, Δ-48) | 0 (base 0, Δ+0) | 0 (base 0, Δ+0) |
| mixed_payload | 488 (base 497, Δ-9) | 0 (base 0, Δ+0) | 0 (base 0, Δ+0) |
| short_conversation | 487 (base 496, Δ-9) | 0 (base 0, Δ+0) | 0 (base 0, Δ+0) |
| stress_conversation | 6590 (base 6651, Δ-61) | 11 (base 7, Δ+4) | 4 (base 4, Δ+0) |
| tiny_payload | 256 (base 267, Δ-11) | 0 (base 0, Δ+0) | 0 (base 0, Δ+0) |
Numbers come from make benchmark / make benchmark-matrix.
Latency is hardware-dependent — treat the markers as a rough guide.
See benchmarks/scorecard.md for the full picture.
…hint with the safe remediation Audit follow-up on #635 hints: - DeterminismError.default_hint now leads with providing a deterministic summarizer/extractor (the data-protective path) and qualifies disabling deterministic mode, matching docs/errors.md. DeterminismError is a fail-closed privacy guard, so the hint should not lead with removing it. - PathNotFoundError gains its own default_hint anchored to #pathnotfounderror instead of inheriting CatalogError's #catalogerror anchor.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the tightly-coupled "stable library error surface" pair in one PR: machine-readable error codes + remediation hints on the exception hierarchy (#635), and the error-reference page that documents them (#637). The two share one module (
src/contextweaver/exceptions.py) — the page documents the codes the hints anchor to, so they are cleaner together than apart.Fixes #635
Fixes #637
Changes
#635 — stable error codes + remediation hints
src/contextweaver/exceptions.py: everyContextWeaverErrorsubclass gains a frozen class-levelcode(e.g.CW_CONFIG) and an optional instancehintwith a class-leveldefault_hintfallback.__str__renders[code] message (hint: …).f"Error: {exc}"(__main__.main) and the gateway CLI usestyper.BadParameter(str(exc)), code + hint surface in CLI output automatically — no CLI changes needed.ConfigError,CatalogError,ItemNotFoundError,DeterminismError,PathInvalidError,BudgetOverflowError.tests/test_exceptions.py: frozenGOLDEN_CODESgolden list + tests for uniqueness, exhaustiveness (introspects the module), andstr()/hint rendering. A rename or a new code-less exception now fails CI.#637 — error-reference page
docs/errors.md: code-index table + per-exception section (stable code, raising modules from a raise-site audit, common causes, fix). Anchors are the lower-cased class name so the hints in Give library exceptions stable error codes and remediation hints #635 link straight to the right section.mkdocs.ymlnav entry; cross-linked fromdocs/troubleshooting.md.scripts/gen_llms.py; regeneratedllms.txt/llms-full.txtso the page lands in the agent-facing bundle.Supporting
api/public_api.txtregenerated for the newhintconstructor parameter (diff is exclusively the exception classes).AGENTS.mddocuments the code/hint convention and the new-exception rule;CHANGELOG.mdupdated under## [Unreleased].How verified
All commands run in a clean
.venvwithpip install -e ".[dev]":ruff check src/ tests/ examples/ scripts/— All checks passedruff format --check …— 370 files already formattedmypy src/ scripts/— Success: no issues found in 177 source filespython -m pytest— 2594 passed, 31 skipped, 1 xfailed; the only failure istest_serve_dry_run_writes_catalog_diagnostic_event, which is environmental:contextweaver mcp serve --dry-runexits 0 but emits a tiktoken403 Forbiddenwarning to stderr because this sandbox can't fetch thecl100k_baseencoding (unrelated to error handling; fails identically onmain).python -m pytest tests/test_exceptions.py— 21 passedscripts/gen_api_manifest.py --check,gen_llms.py --check,check_module_size.py,check_doc_snippets.py— all OK (exceptions.pyis 285 lines, under the 300 ceiling)Checklist
make citargets pass locally (one environmental, network-only tiktoken failure noted above;make docs/example/demonot runnable here —mkdocs/extras absent — CI covers)CHANGELOG.mdupdated under## [Unreleased]api/public_api.txtwithmake apiexceptions.py= 285)docs/errors.md)Notes for reviewers
str(exc)now carries the[code]prefix. Blast radius was measured — only one test asserted exact equality (updated);pytest.raises(match=…)usesre.search, so substring matches are unaffected.GatewayErrormapping. That touches the recently-hardenedtool_executedispatch and is not part of either issue's acceptance criteria, so it's deferred to keep this PR focused.Generated by Claude Code