Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@

### Features

- **#610**: The manual's SYNOPSIS blocks are now generated from the tool registry.
`just man-regen` renders the MCP call on every section-3 page from the schema
clients actually receive (required parameters first, then defaults, in schema
order) and a test holds the shipped blocks byte-equal to the rendering -- change
a tool signature without regenerating and CI points at the fix. Those pages
declare `generated: registry`; curated sections stay hand-owned. The manual also
gains `basic-memory-diagnostics(3)`, closing the one gap between the section-3
corpus and the tool registry.

- **#610**: The manual ships in the package and is served over MCP. The 21 section-3
pages (one per MCP tool -- `search-notes(3)`, `write-note(3)`, ...) now live in
`src/basic_memory/man/man3/` as canonical, portable notes. The MCP server exposes
Expand Down
10 changes: 6 additions & 4 deletions docs/manual-pages.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,10 +186,12 @@ GOTCHAS, SEE ALSO, observations) survives — that ownership split is what the

## Roadmap

- **Registry generator** — section-3 SYNOPSIS/PARAMETERS generated from the
MCP tool registry (docstrings + pydantic schemas), section-1 from Typer
help; the hand-written corpus is the template spec. Regenerate-and-diff in
CI becomes the drift gate.
- **Registry generator (SYNOPSIS: shipped)** — `just man-regen` renders every
section-3 MCP SYNOPSIS block from the live tool registry and a test holds
the shipped blocks byte-equal to the rendering, so a tool change without a
regenerate fails CI. Those pages declare `generated: registry`. Still to
come: PARAMETERS from the schema descriptions, and section-1 from Typer
help — the hand-written corpus remains the template spec.
- **Projects as consumers** — `bm man install --project <name>` copies the
bundled pages into a project as notes, so `SEE ALSO` becomes traversable
relations and the pages join search. (`bm man <topic>`, `bm man list`, the
Expand Down
4 changes: 4 additions & 0 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,10 @@ clean:
format:
uv run ruff format .

# Regenerate the manual's registry-owned SYNOPSIS blocks from the MCP tool registry
man-regen:
uv run python scripts/update_man_pages.py

# Run MCP inspector tool
run-inspector:
npx @modelcontextprotocol/inspector
Expand Down
55 changes: 55 additions & 0 deletions scripts/update_man_pages.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
"""Regenerate the registry-owned sections of the bundled manual.

The MCP SYNOPSIS block on every section-3 page whose tool this build registers is
mechanical: it must show exactly the call the tool schema advertises. This script
renders those blocks from the live registry (``mcp.list_tools()``) and rewrites
them in place, flipping the page's ``generated:`` field to ``registry`` so the
ownership split is declared. Curated sections — DESCRIPTION, PARAMETERS,
EXAMPLES, GOTCHAS, SEE ALSO — are never touched.

Run after changing any MCP tool signature:

just man-regen (or: uv run python scripts/update_man_pages.py)

A test (tests/test_man_pages.py) holds every shipped block byte-equal to the
rendering, so a forgotten run fails CI with a pointer here.
"""

from __future__ import annotations

import asyncio

from basic_memory.man import (
bundled_pages,
declare_registry_ownership,
render_synopsis,
replace_mcp_synopsis,
)
from basic_memory.mcp.server import mcp
import basic_memory.mcp.tools # noqa: F401 (importing registers the tools)


async def main() -> None:
tools = {tool.name: tool for tool in await mcp.list_tools(run_middleware=False)}
changed: list[str] = []
for page in bundled_pages():
# Pages for tools this build does not register (hosted-only ones like
# cloud_info) stay hand-owned: there is no schema here to render from.
if page.section != 3 or page.tool not in tools:
continue
text = page.read()
updated = replace_mcp_synopsis(
text, render_synopsis(page.tool, tools[page.tool].parameters)
)
updated = declare_registry_ownership(updated)
if updated != text:
page.path.write_text(updated, encoding="utf-8")
changed.append(page.title)
if changed:
print(f"updated {len(changed)} page(s): {', '.join(changed)}")
else:
print("all pages already match the registry")


if __name__ == "__main__":
asyncio.run(main())
95 changes: 95 additions & 0 deletions src/basic_memory/man/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,13 @@

from __future__ import annotations

import json
import re
from collections.abc import Mapping
from dataclasses import dataclass
from functools import cache
from pathlib import Path
from typing import Any
from urllib.parse import unquote

from basic_memory.file_utils import parse_frontmatter, remove_frontmatter
Expand Down Expand Up @@ -85,6 +88,7 @@ class ManPage:
section: int
name: str
summary: str
generated: str
tool: str | None
path: Path

Expand Down Expand Up @@ -117,6 +121,7 @@ def bundled_pages() -> tuple[ManPage, ...]:
section=int(frontmatter["section"]),
name=str(frontmatter["name"]),
summary=str(frontmatter["summary"]),
generated=str(frontmatter["generated"]),
tool=str(tool) if tool is not None else None,
path=path,
)
Expand All @@ -143,6 +148,96 @@ def find_page(ref: PageRef) -> ManPage | None:
return None


# --- Registry-generated SYNOPSIS ---
# The MCP SYNOPSIS block on a section-3 page is a mechanical section: it must show
# exactly the call the tool schema advertises. These helpers render it from the
# schema and splice it into a page; scripts/update_man_pages.py runs them over the
# corpus and a test holds every shipped block byte-equal to the rendering.

SYNOPSIS_WIDTH = 76

# Matches the MCP call block under ## SYNOPSIS. Pages that also show a CLI form
# label the MCP block "MCP:"; MCP-only pages have a single unlabelled block.
_MCP_SYNOPSIS_RE = re.compile(r"(## SYNOPSIS\n\n(?:MCP:\n\n)?```\n)(.*?)(\n```)", re.S)


def _default_literal(value: object) -> str:
"""Render a schema default the way the call would be written in Python."""
if isinstance(value, str):
# json.dumps escapes quotes, backslashes, and control characters, and its
# double-quoted output is also a valid Python string literal.
return json.dumps(value)
# None, booleans, and numbers all repr() to their Python spelling.
return repr(value)


def render_synopsis(tool_name: str, parameters: Mapping[str, Any]) -> str:
"""Render a tool's MCP SYNOPSIS call from the JSON schema clients receive.

Required parameters come first as bare names, then optional ones as
``name=default``, each group in schema order — the order clients see. Lines
wrap at the code block's width with continuations aligned under the first
argument.
"""
required: list[str] = parameters.get("required") or []
properties: Mapping[str, Any] = parameters.get("properties") or {}

ordered = [name for name in properties if name in required]
for name, prop in properties.items():
if name in required:
continue
# A default factory leaves no `default` in the schema; render `name=...` so
# the parameter still reads as optional, not as a bare required name.
ordered.append(
f"{name}={_default_literal(prop['default'])}" if "default" in prop else f"{name}=..."
)

indent = " " * (len(tool_name) + 1)
current = f"{tool_name}("
lines: list[str] = []
for position, argument in enumerate(ordered):
piece = argument + ("," if position < len(ordered) - 1 else ")")
trial = current + piece if current.endswith("(") else f"{current} {piece}"
if len(trial) > SYNOPSIS_WIDTH and not current.endswith("("):
lines.append(current)
current = indent + piece
else:
current = trial
if not ordered:
current += ")"
lines.append(current)
return "\n".join(lines)


def extract_mcp_synopsis(page_text: str) -> str:
"""The MCP call block a page currently shows under ## SYNOPSIS."""
match = _MCP_SYNOPSIS_RE.search(page_text)
if match is None:
raise ValueError("page has no MCP SYNOPSIS block")
Comment thread
phernandez marked this conversation as resolved.
return match.group(2)


def replace_mcp_synopsis(page_text: str, synopsis: str) -> str:
"""Return the page with its MCP SYNOPSIS block replaced; other blocks untouched."""
match = _MCP_SYNOPSIS_RE.search(page_text)
if match is None:
raise ValueError("page has no MCP SYNOPSIS block")
return f"{page_text[: match.start()]}{match.group(1)}{synopsis}{match.group(3)}{page_text[match.end() :]}"


def declare_registry_ownership(page_text: str) -> str:
"""Flip ``generated: hand`` to ``registry`` — in the frontmatter only.

A curated body may legally contain a literal ``generated: hand`` line (a YAML
example, say); only the opening frontmatter block is the generator's to rewrite.
"""
frontmatter, fence, body = page_text.partition("\n---\n")
frontmatter = re.sub(
r"^generated: hand$", "generated: registry", frontmatter, count=1, flags=re.M
)
return frontmatter + fence + body


def render_index(pages: tuple[ManPage, ...], registered_tools: frozenset[str] | None = None) -> str:
"""The apropos view: every page, grouped by section, one line each.

Expand Down
68 changes: 68 additions & 0 deletions src/basic_memory/man/man3/basic-memory-diagnostics(3).md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
title: basic-memory-diagnostics(3)
type: manpage
section: 3
name: basic-memory-diagnostics
summary: report version, system, and redacted configuration for troubleshooting
generated: registry
tool: basic_memory_diagnostics
verified: 0.23.2 local
---

# basic-memory-diagnostics(3)

## NAME

**basic-memory-diagnostics** — report version, system, and redacted configuration for troubleshooting

## SYNOPSIS

```
basic_memory_diagnostics()
```

## DESCRIPTION

Returns a markdown report for support requests and install debugging: the
basic-memory package version and API version, the Python version, platform,
and architecture, and the config file path with its contents as a JSON block.
Secrets and API keys are redacted before anything is emitted.

Read-only by contract: the tool only computes the config path — it never
creates the data directory or touches the database, so it works on a broken
or half-installed setup, which is exactly when it is needed.

## MCP USAGE

Verified locally (0.23.2):

```
basic_memory_diagnostics()
# → "# Basic Memory Diagnostics
#
# ## Version
# - basic-memory: 0.23.2
# - API: v2
#
# ## System
# - Python: 3.14.5 (...)
# - Platform: macOS-26.6-arm64-arm-64bit-Mach-O
# - Architecture: arm64
#
# ## Configuration
# - Config path: ~/.basic-memory/config.json
# - Config exists: True
# ```json
# { redacted config ... }
# ```"
```

## GOTCHAS

- [gotcha] The config JSON block lists every configured project with its path and mode — redaction removes secrets, not project names, so treat the report as private when sharing #privacy
- [gotcha] A missing or unreadable config file is reported inline (`<config file not found>` / `<error reading config: ...>`) instead of failing — no result is still a diagnostic #resilience

## SEE ALSO

- see_also [[list-memory-projects(3)]]
- see_also [[cloud-info(3)]]
7 changes: 3 additions & 4 deletions src/basic_memory/man/man3/build-context(3).md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: manpage
section: 3
name: build-context
summary: traverse the knowledge graph outward from a memory:// URL
generated: hand
generated: registry
tool: build_context
verified: 0.21.6 mcp
---
Expand All @@ -20,9 +20,8 @@ verified: 0.21.6 mcp
MCP:

```
build_context(url, depth=1, timeframe="7d", max_related=10,
project=None, project_id=None,
page=1, page_size=10, output_format="json")
build_context(url, project=None, project_id=None, depth=1, timeframe="7d",
page=1, page_size=10, max_related=10, output_format="json")
```

CLI:
Expand Down
2 changes: 1 addition & 1 deletion src/basic_memory/man/man3/chatgpt-fetch(3).md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: manpage
section: 3
name: chatgpt-fetch
summary: OpenAI-actions-compatible document fetch adapter
generated: hand
generated: registry
tool: fetch
verified: 0.21.6 mcp
---
Expand Down
2 changes: 1 addition & 1 deletion src/basic_memory/man/man3/chatgpt-search(3).md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: manpage
section: 3
name: chatgpt-search
summary: OpenAI-actions-compatible search adapter
generated: hand
generated: registry
tool: search
verified: 0.21.6 mcp
---
Expand Down
7 changes: 3 additions & 4 deletions src/basic_memory/man/man3/create-memory-project(3).md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: manpage
section: 3
name: create-memory-project
summary: create a new project, locally or in a cloud workspace
generated: hand
generated: registry
tool: create_memory_project
verified: 0.21.6 mcp+cli
---
Expand All @@ -20,9 +20,8 @@ verified: 0.21.6 mcp+cli
MCP:

```
create_memory_project(project_name, project_path,
set_default=False, workspace=None,
output_format="text")
create_memory_project(project_name, project_path, set_default=False,
workspace=None, output_format="text")
```

CLI:
Expand Down
6 changes: 3 additions & 3 deletions src/basic_memory/man/man3/delete-note(3).md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: manpage
section: 3
name: delete-note
summary: delete a note or directory from the knowledge base
generated: hand
generated: registry
tool: delete_note
verified: 0.21.6 mcp+cli
---
Expand All @@ -20,8 +20,8 @@ verified: 0.21.6 mcp+cli
MCP:

```
delete_note(identifier, is_directory=False,
project=None, project_id=None, output_format="text")
delete_note(identifier, is_directory=False, project=None, project_id=None,
output_format="text")
```

CLI:
Expand Down
2 changes: 1 addition & 1 deletion src/basic_memory/man/man3/delete-project(3).md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: manpage
section: 3
name: delete-project
summary: remove a project from configuration and index (files survive by default)
generated: hand
generated: registry
tool: delete_project
verified: 0.21.6 mcp
---
Expand Down
Loading
Loading