feat(mcp): ship the manual in the package and serve it as resources

CHANGELOG.md

@@ -4,6 +4,17 @@
 
 ### Features
 
+- **#610**: The manual ships in the package and is served over MCP. The 21 section-3
+  pages (one per MCP tool -- `search-notes(3)`, `write-note(3)`, ...) now live in
+  `src/basic_memory/man/man3/` as canonical, portable notes. The MCP server exposes
+  them as resources: `memory://man` is the index and `memory://man/<page>` is a page,
+  with every page also listed as a concrete resource so clients that browse
+  `resources/list` see each one with its summary. Page references parse the way people
+  and models actually write them -- `search-notes(3)`, `3/search-notes`,
+  `search_notes`, `man3/search-notes.md` -- and the server instructions point agents at
+  a tool's page before first use. `bm man <topic>` prints a page as Markdown and
+  `bm man list` is apropos; `bm man install` is unchanged.
+
 - **#1259**: `write_note` now tells the caller when a freshly created note looks like
   notes that already exist. On a server with semantic search enabled, a create probes
   the vector index with the new note's title and opening content and appends a

docs/manual-pages.md

@@ -9,9 +9,14 @@ documents the tools; the tools verify the manual.
 
 ## Where it lives
 
-The canonical manual is the **`manual` project in the Basic Memory team
-workspace** (cloud, shared). Anyone can build their own: the schema ships as
-an opt-in seed at `plugins/claude-code/schemas/manpage.md` — copy it into any
+Section 3 — one page per MCP tool — is canonical **in the package**, at
+`src/basic_memory/man/man3/`, so every install ships the same pages: local,
+cloud, or offline. The MCP server serves them as resources (`memory://man`
+is the index, `memory://man/search-notes(3)` a page) and `bm man <topic>`
+prints one in a shell. The `manual` project in the Basic Memory team
+workspace (cloud, shared) holds the full manual — sections 5 and 7 are
+canonical there — and anyone can build their own: the schema ships as an
+opt-in seed at `plugins/claude-code/schemas/manpage.md` — copy it into any
 project's folder and start writing pages against it.
 
 Layout:
@@ -102,7 +107,17 @@ bm tool search-notes --project manual          # then filter, or via MCP:
 #   build_context(url="man3/write-note-3", project="manual")
 ```
 
-A future `bm man <topic>` command is thin sugar over exactly these calls.
+From an MCP client, the same pages are resources — no project required:
+
+```
+memory://man                      # the index (apropos)
+memory://man/search-notes(3)      # one page
+memory://man/3/search-notes       # any common spelling resolves,
+memory://man/search_notes         # including the tool name itself
+```
+
+And in a shell, `bm man search-notes` prints the page as Markdown and
+`bm man list` lists every page with its summary.
 
 And for the real thing — `man bm` in an actual terminal:
 
@@ -175,9 +190,12 @@ GOTCHAS, SEE ALSO, observations) survives — that ownership split is what the
   MCP tool registry (docstrings + pydantic schemas), section-1 from Typer
   help; the hand-written corpus is the template spec. Regenerate-and-diff in
   CI becomes the drift gate.
-- **`bm man <topic>`** — CLI sugar over `read_note` + metadata search.
-  (`bm man install` + a hand-written `bm.1` already ship — the first slice
-  of [#610](https://github.com/basicmachines-co/basic-memory/issues/610);
-  the generator will produce per-command pages from the same extraction.)
+- **Projects as consumers** — `bm man install --project <name>` copies the
+  bundled pages into a project as notes, so `SEE ALSO` becomes traversable
+  relations and the pages join search. (`bm man <topic>`, `bm man list`, the
+  `memory://man` resources, and the bundled section 3 already ship — the
+  second slice of [#610](https://github.com/basicmachines-co/basic-memory/issues/610).)
+- **Groff for section 3** — render the bundled pages to roff so
+  `man search-notes` works after `bm man install`, alongside `bm.1`.
 - **Docs site** — the notes remain canonical for sections 5 and 7, code is
   canonical for 1 and 3; both render to the hosted docs site.

src/basic_memory/cli/commands/man.py

@@ -1,24 +1,74 @@
-"""Install the bundled man pages so `man bm` works."""
+"""`bm man`: read the bundled manual, and install the groff pages so `man bm` works."""
 
 import shutil
 import subprocess
+import sys
 from pathlib import Path
-from typing import Annotated, Optional
+from typing import Annotated, Optional, override
 
 import typer
 from rich.console import Console
+from typer.core import TyperGroup
+
+# Typer vendors its own click; an override must be typed with the base class's
+# types, and these are the ones TyperGroup.resolve_command is declared with.
+from typer._click.core import Command, Context
 
 from basic_memory.cli.app import app
+from basic_memory.man import bundled_pages, find_page, parse_page_ref
 
 console = Console()
 
-man_app = typer.Typer(help="Manage the bm man pages.")
+
+class ManGroup(TyperGroup):
+    """Let `bm man <topic>` read like man(1).
+
+    A first argument that is not a subcommand is a page name, so `bm man search-notes`
+    is `bm man show search-notes` without the ceremony. Real subcommands (`install`,
+    `list`, `show`) and options keep their meaning.
+    """
+
+    @override
+    def resolve_command(
+        self, ctx: Context, args: list[str]
+    ) -> tuple[str | None, Command | None, list[str]]:
+        if args and not args[0].startswith("-") and self.get_command(ctx, args[0]) is None:
+            args = ["show", *args]
+        return super().resolve_command(ctx, args)
+
+
+man_app = typer.Typer(help="Read the Basic Memory manual, or install the man pages.", cls=ManGroup)
 app.add_typer(man_app, name="man")
 
 # Bundled groff sources ship inside the package (src/basic_memory/man).
 _MAN_SOURCE_DIR = Path(__file__).parent.parent.parent / "man"
 
 
+@man_app.command()
+def show(
+    topic: Annotated[str, typer.Argument(help="Page name, e.g. search-notes or search-notes(3)")],
+) -> None:
+    """Print a manual page as Markdown."""
+    try:
+        page = find_page(parse_page_ref(topic))
+    except ValueError as error:
+        console.print(f"[red]{error}[/red]")
+        raise typer.Exit(1) from error
+    if page is None:
+        console.print(f"[red]No manual entry for {topic}[/red]  (try: bm man list)")
+        raise typer.Exit(1)
+    # Raw Markdown, unwrapped: agents and pagers read this as often as eyes do.
+    sys.stdout.write(page.body())
+    sys.stdout.write("\n")
+
+
+@man_app.command(name="list")
+def list_pages() -> None:
+    """List every manual page with its one-line summary (apropos)."""
+    for page in bundled_pages():
+        sys.stdout.write(f"{page.title:<28} {page.summary}\n")
+
+
 def _default_man_root() -> Path:
     # Why ~/.local/share/man: manpath(1) derives man directories from PATH
     # entries on both man-db (Linux) and BSD man (macOS), so ~/.local/bin on

src/basic_memory/man/__init__.py

@@ -0,0 +1,173 @@
+"""The bundled Basic Memory manual.
+
+Pages are Markdown notes in Unix man-page form, kept in numbered section
+directories (``man1/``, ``man3/``, ...). Section 3 — one page per MCP tool — is
+canonical here in the package, so every install ships the same pages whether it
+is local, cloud, or offline. Three consumers read them: the MCP server serves
+them as ``memory://man`` resources, ``bm man <topic>`` renders them in a
+terminal, and projects can take copies as ordinary notes.
+
+See ``docs/manual-pages.md`` for the page anatomy and the verification rules.
+"""
+
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass
+from functools import cache
+from pathlib import Path
+from urllib.parse import unquote
+
+from basic_memory.file_utils import parse_frontmatter, remove_frontmatter
+
+MAN_DIR = Path(__file__).resolve().parent
+
+
+@dataclass(frozen=True)
+class PageRef:
+    """A reference to a page, as the caller wrote it: a name and maybe a section."""
+
+    name: str
+    section: int | None
+
+    @property
+    def display(self) -> str:
+        return f"{self.name}({self.section})" if self.section is not None else self.name
+
+
+_SECTION_DIR_RE = re.compile(r"(?:man)?([1-9])")
+_NAME_WITH_SECTION_RE = re.compile(
+    r"(?P<name>.+?)(?:\((?P<paren>[1-9])\)|\.(?P<dot>[1-9])|-(?P<dash>[1-9]))"
+)
+
+
+def parse_page_ref(text: str) -> PageRef:
+    """Read a page reference in any of the forms people and models actually write.
+
+    Parse, don't validate: the reference is whatever the caller reached for first,
+    so every common spelling of the same page is accepted —
+
+        search-notes(3)    search-notes.3    search-notes-3    3/search-notes
+        man3/search-notes  search_notes      man3/search-notes(3).md
+
+    — plus percent-encoded variants of the above. The section is optional. Tool
+    names with underscores map to the hyphenated page name.
+
+    Raises ValueError for a reference that cannot name a page at all (empty, or
+    a path whose directory is not a section).
+    """
+    ref = unquote(text).strip().strip("/").removesuffix(".md")
+    section: int | None = None
+
+    if "/" in ref:
+        directory, _, ref = ref.rpartition("/")
+        match = _SECTION_DIR_RE.fullmatch(directory)
+        if match is None:
+            raise ValueError(f"{text!r} is not a manual page reference")
+        section = int(match.group(1))
+
+    match = _NAME_WITH_SECTION_RE.fullmatch(ref)
+    if match is not None:
+        ref = match.group("name")
+        suffix = match.group("paren") or match.group("dot") or match.group("dash")
+        section = int(suffix)
+
+    name = ref.lower().replace("_", "-")
+    if not name:
+        raise ValueError(f"{text!r} is not a manual page reference")
+    return PageRef(name=name, section=section)
+
+
+@dataclass(frozen=True)
+class ManPage:
+    """One bundled page and the frontmatter fields the manual schema guarantees."""
+
+    section: int
+    name: str
+    summary: str
+    tool: str | None
+    path: Path
+
+    @property
+    def title(self) -> str:
+        return f"{self.name}({self.section})"
+
+    @property
+    def uri(self) -> str:
+        return f"memory://man/{self.title}"
+
+    def read(self) -> str:
+        """The page as shipped: frontmatter and body."""
+        return self.path.read_text(encoding="utf-8")
+
+    def body(self) -> str:
+        """The page without its frontmatter, for rendering."""
+        return remove_frontmatter(self.read())
+
+
+@cache
+def bundled_pages() -> tuple[ManPage, ...]:
+    """Every page in the package, ordered by section then name."""
+    pages: list[ManPage] = []
+    for path in sorted(MAN_DIR.glob("man[1-9]/*.md")):
+        frontmatter = parse_frontmatter(path.read_text(encoding="utf-8"))
+        tool = frontmatter.get("tool")
+        pages.append(
+            ManPage(
+                section=int(frontmatter["section"]),
+                name=str(frontmatter["name"]),
+                summary=str(frontmatter["summary"]),
+                tool=str(tool) if tool is not None else None,
+                path=path,
+            )
+        )
+    return tuple(sorted(pages, key=lambda page: (page.section, page.name)))
+
+
+def find_page(ref: PageRef) -> ManPage | None:
+    """Resolve a reference the way man(1) does: the named section, else the lowest.
+
+    A page may be named differently from the tool it documents (chatgpt-search(3)
+    documents `search`), so the tool name is an alias for the page name. An exact
+    page name wins over an alias.
+    """
+    candidates = [
+        page for page in bundled_pages() if ref.section is None or page.section == ref.section
+    ]
+    for page in candidates:
+        if page.name == ref.name:
+            return page
+    for page in candidates:
+        if page.tool is not None and page.tool.replace("_", "-") == ref.name:
+            return page
+    return None
+
+
+def render_index(pages: tuple[ManPage, ...], registered_tools: frozenset[str] | None = None) -> str:
+    """The apropos view: every page, grouped by section, one line each.
+
+    The same corpus serves the local and the hosted server, whose tool sets differ,
+    so when the caller knows which tools this server registers, pages for the
+    others are marked rather than presented as callable.
+    """
+    section_titles = {1: "User commands", 3: "MCP tools", 5: "File formats", 7: "Concepts"}
+    lines = [
+        "# Basic Memory manual",
+        "",
+        "Read a page with its `memory://man/...` URI, or `bm man <name>` in a shell.",
+    ]
+    current_section: int | None = None
+    for page in pages:
+        if page.section != current_section:
+            current_section = page.section
+            heading = section_titles.get(page.section, f"Section {page.section}")
+            lines.extend(["", f"## Section {page.section} — {heading}", ""])
+        line = f"- [{page.title}]({page.uri}) — {page.summary}"
+        if (
+            registered_tools is not None
+            and page.tool is not None
+            and page.tool not in registered_tools
+        ):
+            line += " *(tool not registered on this server)*"
+        lines.append(line)
+    return "\n".join(lines) + "\n"