diff --git a/AGENTS.md b/AGENTS.md index 7094442b..199ca597 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -105,4 +105,4 @@ See [`skills/workflow/creating-skills/SKILL.md`](./skills/workflow/creating-skil - [`mcp/README.md`](./mcp/README.md) — MCP server structure and available servers - [`mcp/servers/witan/README.md`](./mcp/servers/witan/README.md) — witan graph-memory server - [`custom-agents/README.md`](./custom-agents/README.md) — agent definitions for Claude/Copilot -- [`docs/`](./docs/) — the **witan-context** documentation site (https://witan-context.readthedocs.io). `docs/reference/` is GENERATED and `docs/guides/` is mostly MIRRORED from the packages — do not hand-edit either; run `just docs-gen` and commit. `just docs-check` gates this in CI, and `just docs-serve` previews locally. Historical specs live in `docs/internals/`. +- [`docs/`](./docs/) — the **witan-context** documentation site (set up to publish on Read the Docs once that project is registered). `docs/reference/` is GENERATED and `docs/guides/` is mostly MIRRORED from the packages — do not hand-edit either; run `just docs-gen` and commit. `just docs-check` gates this in CI, and `just docs-serve` previews locally. Historical specs live in `docs/internals/`. diff --git a/README.md b/README.md index 77891730..0a29f6bc 100644 --- a/README.md +++ b/README.md @@ -1,18 +1,35 @@ # agent-kit -A shared toolkit of AI agent utilities for the team, including reusable skills, -custom agent definitions, MCP server install helpers, and sample configurations. +A shared toolkit of AI agent utilities for the team. It holds three published +Python packages — [`witan-council`](./mcp/servers/witan/README.md), +[`witan-code`](./mcp/servers/witan-code/README.md), and +[`witan-core`](./packages/witan-core/README.md), the shared memory, +work-coordination, and code-graph layer for coding agents — alongside reusable +skills, custom agent definitions, MCP server install helpers, and sample +configurations. + +Documentation for the witan packages lives in [`docs/`](./docs/) — tutorials, +guides, a generated reference for every MCP tool, CLI command and environment +variable, and the architecture notes. Build it locally with `just docs-serve`. +It is set up to publish on Read the Docs as **witan-context** once that project +is registered. ## Repository Structure ``` . +├── mcp/servers/witan/ # witan-council — memory, tasks, workflow; the `witan` umbrella CLI +├── mcp/servers/witan-code/ # witan-code — tree-sitter code graph + cross-repo bridge +├── packages/witan-core/ # witan-core — shared internals for the two servers above +├── packages/agent-config-kit/ # agent-kit — the CLI that installs the skills/MCP servers declared below +├── packages/agent-kit/ # PyPI meta-package (ol-agent-kit): agent-config-kit[cli] + witan + witan-code ├── skills/ # Reusable skills, installed via agent-config-kit ├── custom-agents/ # Custom agent definitions for GitHub Copilot and Claude Code -├── mcp/ # Install helpers and configuration for common MCP servers +├── mcp/ # Install helpers and configuration for other MCP servers ├── configs/ # Sample / reference agent configurations -├── packages/agent-config-kit/ # agent-kit — the CLI that installs the skills/MCP servers declared below -├── packages/agent-kit/ # PyPI meta-package (ol-agent-kit): agent-config-kit[cli] + witan + witan-code +├── docker/ # Deployment images: the witan MCP tier and the omnigraph data tier +├── docs/ # The witan-context documentation site (Zensical → Read the Docs) +├── bin/ # Repo maintenance scripts (version/pin checks, docs generation) └── agent-config.toml # This repo's own manifest for agent-kit ``` diff --git a/docs/getting-started/index.md b/docs/getting-started/index.md index 71763a6e..fa35e6ac 100644 --- a/docs/getting-started/index.md +++ b/docs/getting-started/index.md @@ -44,9 +44,18 @@ You will need: You do **not** need a server, a database, or any credentials. The default store is a single file at `~/.local/share/witan/graph.omni`, and everything in this -tutorial runs against it locally. Pointing witan at a shared, deployed service -is a later, separate step — see [Using a deployed -witan](../guides/deployed-witan.md). +tutorial runs against it locally. + +Moving to a shared, deployed witan later is two separate jobs, and it is worth +knowing both exist before you accumulate a graph worth keeping: + +- **Point your client at it** — [Using a deployed + witan](../guides/deployed-witan.md). Register the target, log in, switch your + agent over. +- **Bring your data with you** — [Migration + runbook](../guides/migration-runbook.md). A store cannot be moved by copying + it, so this is a real procedure rather than a file copy; the runbook's + "Local → shared: the cutover" section is the one you want. ## A note on where things run diff --git a/docs/guides/index.md b/docs/guides/index.md index 3c3d1096..68ae8f64 100644 --- a/docs/guides/index.md +++ b/docs/guides/index.md @@ -57,6 +57,8 @@ started](../getting-started/index.md) instead. | I want to… | Where | | --- | --- | | Change where the graph is stored | [`WITAN_MEMORY_URI`](../reference/environment.md#store-and-attribution) | +| Move my local graph onto the team's shared store | [Migration runbook](migration-runbook.md) — never by copying the directory | +| Point my CLI and agent at a deployed witan | [Using a deployed witan](deployed-witan.md) | | Route work repos and personal repos at different stores | [Named targets](witan-user-guide.md) | | Stop a detector flagging a false positive | [Write-path scanning](write-path-scanning.md) | | Run the code indexer in CI | [`WITAN_CODE_CI_REPOS`](../reference/environment.md#ci-code-graph-indexer) | diff --git a/mcp/servers/witan-code/README.md b/mcp/servers/witan-code/README.md index 885a7dbd..9cf8e1eb 100644 --- a/mcp/servers/witan-code/README.md +++ b/mcp/servers/witan-code/README.md @@ -8,9 +8,9 @@ repository's symbols (functions, methods, classes, modules) and their relationships, then exposes definition / reference / caller / impact queries to the agent. -It is a self-contained sibling of `witan` (Layer 1) and shares its -subprocess/CLI conventions, but stores a **separate, per-repo, local-only** -graph. +It is a sibling of `witan` (Layer 1) — it never imports it, though both build on +`witan-core` — and shares its subprocess/CLI conventions, but stores a +**separate, per-repo** graph rather than writing into Layer 1's. > **Wiring it into your agents locally** (MCP server, the `PostToolUse` reindex > hook, and the indexer CLI, run straight from your checkout): see @@ -37,7 +37,14 @@ resolution, not a true call graph) — see | Layer | Server | Stores | Scope | Synced | |------|--------|--------|-------|--------| | 1 | `witan` | patterns, project facts, lessons, workflow traces | team-wide | yes (S3) | -| 2 | `witan-code` | code symbols + edges | per-repo | **no — local only** | +| 2 | `witan-code` | code symbols + edges | per-repo | local by default; shareable | + +Layer 2 was local-only originally and is no longer. A code graph can live on a +shared `omnigraph-server` (`WITAN_CODE_SERVER` in-cluster, or through the +deployed witan MCP tier from outside it), where a CI indexer owns each repo's +default `main` view and every other writer gets its own per-actor branch view. +On a local store there is one user, who is its writer, and none of that +arbitration applies. See [`docs/BRANCH_INDEXING.md`](docs/BRANCH_INDEXING.md). The two layers compose through **soft symbol-ID references**. A Layer-1 node (e.g. a `lesson` or `agent_context`) can record symbol ids of the form: @@ -82,7 +89,7 @@ The per-repo graph stops at a repo boundary, but service-oriented architectures couple repos through **shared contracts**: an env var that infra sets and an app reads, an HTTP endpoint one service serves and another calls, a package one repo publishes and others import. The bridge records these as **interface bindings** -in a single shared, local-only store (`_bridge.omni`, a sibling of the per-repo +in a single shared store (`_bridge.omni` locally, a sibling of the per-repo stores) so linkages can be queried across every indexed repo. It is **zero-config**: every `index`/reindex of a repo also extracts that repo's diff --git a/mcp/servers/witan/README.md b/mcp/servers/witan/README.md index 86de4f9e..2a590fa6 100644 --- a/mcp/servers/witan/README.md +++ b/mcp/servers/witan/README.md @@ -170,8 +170,10 @@ Links a git branch (repo + raw branch name, e.g. `feature/new-api` — never witan-code's sanitized omnigraph branch name) to the task/project it's carrying, so "which branch carries task X" and "which tasks are in flight on branch B" are one-hop graph queries. Coordination state that lives in witan -(shared, durable), not witan-code's per-repo/bridge omnigraph stores (local, -re-derivable caches `branches --prune` may destroy at any time). +(shared, durable), not witan-code's per-repo/bridge omnigraph stores +(re-derivable caches `branches --prune` may destroy at any time — shareable +now, but still rebuildable from the source tree, which is the distinction that +matters here). Wired in automatically, best-effort — no dedicated tool call needed: diff --git a/mcp/servers/witan/policy/README.md b/mcp/servers/witan/policy/README.md index adc17075..ef43eabb 100644 --- a/mcp/servers/witan/policy/README.md +++ b/mcp/servers/witan/policy/README.md @@ -262,5 +262,5 @@ The mapping the renderer applies: its rules grant nobody — deliberate, and logged as a warning on every boot so it stays visible. Adding a group to a bundle without adding it to the renderer's `KNOWN_GROUPS` fails the boot rather than rendering an empty group that silently -denies its members; `tests/../test_render_groups.py` pins that against the real +denies its members; `../tests/test_render_groups.py` pins that against the real committed bundles. diff --git a/packages/agent-kit/README.md b/packages/agent-kit/README.md index 27b4c630..fd1d0ee2 100644 --- a/packages/agent-kit/README.md +++ b/packages/agent-kit/README.md @@ -27,7 +27,9 @@ witan-code --help Version bumps go through [`bump-my-version`](https://github.com/callowayproject/bump-my-version) (config in `[tool.bumpversion]`), same as `agent-config-kit`, `witan`, and `witan-code`. `dependencies` versions in `pyproject.toml` are open-ended -floors (`agent-config-kit[cli]>=0.3.7`, `witan-council>=0.2.0`, -`witan-code>=0.2.0` — no upper bound), so a new release of any of the three -is picked up by a fresh install automatically without needing a matching -`ol-agent-kit` release. +floors with no upper bound, so a new release of any of the three is picked up +by a fresh install automatically without needing a matching `ol-agent-kit` +release. The floors themselves move as the meta-package comes to depend on +newer behaviour — currently the two servers are floored at the releases that +speak MCP 2026-07-28 — so read them from `pyproject.toml` rather than from +here. diff --git a/packages/witan-core/README.md b/packages/witan-core/README.md index 714332e9..c95d0a2a 100644 --- a/packages/witan-core/README.md +++ b/packages/witan-core/README.md @@ -30,8 +30,21 @@ both, preserving the one-directional `witan` → `witan_code` optional-mount DAG The base package is stdlib-only. Heavier concerns are gated behind extras so neither server pulls weight it doesn't use: -- `witan-core[cli]` → `cyclopts`, `rich` (CLI scaffolding, styled installer output) +- `witan-core[cli]` → `cyclopts`, `rich`, `agent-config-kit` (CLI scaffolding, + styled installer output) - `witan-core[mcp]` → `fastmcp` (MCP elicitation primitives) +- `witan-core[remote]` → `httpx2`, `fastmcp` (the ADR-0005 client stack: OIDC + device-auth + token cache, and the MCP-client proxy) +- `witan-core[observability]` → `structlog`, OpenTelemetry (structured logs and + traces; the OTel halves are imported defensively so an install without an + exporter still works) +- `witan-core[sentry]` → `sentry-sdk` + +`sentry` is **additive to `observability`, not an alternative to it**: +`telemetry.py` imports `observability.logging`, which imports `structlog` at +module scope, so `sentry` on its own is an ImportError rather than a lighter +build. The split lets a deployment take logs and traces *without* shipping +errors to Sentry, not the reverse. Both servers request both. ## What's here @@ -60,6 +73,28 @@ Extracted so far (each deletes the duplicated copies from both servers): shared matcher, which is structurally typed over just the four `match_*` lists. -Still local to each server (intentionally): the CLI scaffolding — its -extraction coordinates with the in-flight multi-user deployment work (see the -spec). +Later additions, past the original extraction list: + +- `cli` — shared CLI scaffolding (`make_app`, `resolve_author`, + `report_install`), used by both servers' `setup` commands. Needs the `cli` + extra. This is no longer local to each server; what stays local is each + server's own commands and setup behaviour. +- `identity` — Keycloak `sub` → omnigraph actor id (ADR-0004). witan maps the + claim server-side off a validated JWT; witan-code maps the same claim + client-side off its cached token to name the branch views it owns. One + derivation, so the two agree. +- `remote/` — the client-side remote-access layer behind `witan login` (ADR-0005 + path a): `config` (`RemoteConfig`), `oidc` (device-auth grant + shared token + cache), `proxy` (MCP-client proxy). Needs the `remote` extra. +- `observability/` — structlog configuration plus OpenTelemetry, patterned after + `mitol-django-observability` so witan reports the way the rest of the estate + does. Includes the ASGI and MCP middleware and `telemetry.configure_sentry`. +- `omnigraph_http` — pooled HTTP transport for a deployed omnigraph-server, so a + remote read need not pay for a CLI subprocess. Gated by + `WITAN_OMNIGRAPH_HTTP`; the CLI path beneath it stays maintained and is still + the only route to `load`, `branch`, and `optimize`. +- `chunking` — splits a bulk load into batches omnigraph-server will accept, + rather than dying on a `413` part-way through a repo-scale index. +- `caching` — server-declared cache directives for `tools/list` and friends + (MCP 2026-07-28, SEP-2549), so a client stops re-fetching a surface that only + changes on deploy.