From 17b6eb9d8619f9b3f1acb12c81d0badf4bf24480 Mon Sep 17 00:00:00 2001 From: Peter Tos <38345014+bmxcode@users.noreply.github.com> Date: Thu, 6 Aug 2026 23:19:33 +1000 Subject: [PATCH 1/3] Go public: a README that reads like a project, plus CONTRIBUTING and SECURITY MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The README's first screen was a title, a one-line description and a status blockquote that was flatly untrue — M1 and M2 are both done. It now opens with what the thing is, a three-command quickstart, and an honest status line, so a visitor with thirty seconds gets the shape of it before any prose. Seven claims were wrong or missing, found by running the commands rather than reading them: "One card per session" predates ADR-0015 (a day of a session is a card); "Four things underneath it are mechanical" sat above six commands; Codex capture, `crate day`, `crate lint` and per-card token usage were absent entirely; and "Three layers:" headed a four-row table, CONVENTIONS.md being Layer 3's second owner rather than a fourth layer. Adds a "Why not just…" section, because chat history, RAG and Obsidian-alone are the comparisons a reader arrives with and the README never answered them; and a commands table, which is where the invisible commands stop being invisible. CONTRIBUTING.md and SECURITY.md say what is actually true: no PRs during the deliverable sequence, issues welcome, and a private route for a security report. SECURITY.md's advisory link needs private vulnerability reporting enabled, so that command joins topics in the flip runbook. Refs #27 Co-Authored-By: Claude Opus 5 --- CONTRIBUTING.md | 39 ++++++++++++++ README.md | 121 ++++++++++++++++++++++++++++++------------- SECURITY.md | 28 ++++++++++ docs/going-public.md | 19 +++++++ 4 files changed, 171 insertions(+), 36 deletions(-) create mode 100644 CONTRIBUTING.md create mode 100644 SECURITY.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..316f17d --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,39 @@ +# Contributing + +Thanks for looking. Here's the honest state of things. + +## Pull requests + +**I'm not taking pull requests right now.** This is a solo project working through a numbered set of deliverables, one per issue, and each issue is the spec for the change that closes it. A PR arriving outside that sequence is one I'd have to decline, and I'd rather say so here than waste your afternoon. + +That will change once the roadmap in [Milestone 3](https://github.com/bmxcode/crate-wiki/milestone/3) is done. Until then, the useful contribution is an issue. + +## Issues + +Genuinely welcome, and the more specific the better: + +- **A bug** — what you ran, what happened, what you expected. `crate --version`, your OS, and your Python version help. **Don't paste session transcripts, vault content, or anything from a work context** — a redacted description of the shape of the problem is enough, and this repo's history is public and permanent. +- **A command that lies** — the README and `--help` make promises, and promises rot. If one of them doesn't match what the code does, that's a real bug and I want it. +- **A design decision that looks wrong** — read the [ADR](docs/adr/) first if there is one, since it will name the alternative that was already rejected and why. If the reasoning doesn't hold, say so; that's worth more than a patch. + +## Running the checks + +```bash +uv run ruff check . +uv run ruff format --check . +uv run pytest -q +uv tool install --editable . && crate --version +``` + +CI runs all of these plus a check that the *installed* command works, which is where a broken entrypoint hides. System Python on macOS is 3.9 and this project needs ≥3.11 — go through `uv`, never bare `python3`. + +Secrets are scanned on both sides. Install the local hook once per clone: + +```bash +brew install gitleaks +pre-commit install +``` + +## Test data + +**Fixtures are synthetic, always.** The temptation is to copy a real session out of `~/.claude/projects/` to test the parser. Don't — this repo's git history is exposed retroactively, and one such commit is permanent. diff --git a/README.md b/README.md index 82ba8c6..24ad2a5 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,21 @@ An LLM wiki that compounds what you learn and what you've done. -> **Status: early.** Milestone 1 is in progress — see the [issues](https://github.com/bmxcode/crate-wiki/issues) for the roadmap. +[![CI](https://github.com/bmxcode/crate-wiki/actions/workflows/ci.yml/badge.svg)](https://github.com/bmxcode/crate-wiki/actions/workflows/ci.yml) ![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue) -## The problem +Since I started building with AI assistants, my working memory of my own work collapsed from about six weeks to about three days. crate-wiki fixes the part of that a tool can fix. Every Claude Code session is parsed to disk as it runs — pure Python, zero tokens, no assistant involved (Codex too, on a one-command sweep). Then, only when you ask, an LLM folds one into a cross-referenced wiki you can still read in three months. + +```bash +uv tool install git+https://github.com/bmxcode/crate-wiki +crate init ~/crate-personal --scope personal # an Obsidian vault, schema included +crate install-hook --vault ~/crate-personal # sessions now capture themselves +``` + +> **Status: early.** Milestones 1 and 2 are done — capture, `/ingest`, `/ask`, `/daily`, `/lint` all work and have tests. [Milestone 3](https://github.com/bmxcode/crate-wiki/milestone/3) is open. There's no tagged release, the vault format can still change under `crate upgrade`, and as far as I know I'm the only person running it. Every contested decision has a record in [docs/adr/](docs/adr/), including the ones later reversed. + +Everything stays on your machine: capture reads local transcript files and writes local Markdown, and nothing is uploaded anywhere. The only thing that costs tokens is a synthesis you ran yourself. -Since I started building with AI assistants, my working memory of my own work collapsed from about six weeks to about three days. +## The problem The work still happens. It just stops sticking, because the context now lives in chat sessions that scroll away instead of in my head. And chat history is a poor substitute for memory: it's append-only, unstructured, and re-read from scratch every time. Nothing accumulates. @@ -18,52 +28,57 @@ Three layers: | Layer | Contents | Written by | |---|---|---| -| **Raw sources** | Sessions, clips, transcripts, pastes | You and the capture hook — immutable | -| **The wiki** | Summaries, entities, concepts, syntheses | The LLM | -| **The schema** | `CLAUDE.md` / `AGENTS.md` | The engine — it's what makes the LLM a knowledge manager rather than a chatbot | -| **Your conventions** | `CONVENTIONS.md` | You — what this particular vault has decided | +| **1 · Raw sources** | Sessions, clips, transcripts, pastes | You and the capture hook — immutable | +| **2 · The wiki** | Summaries, entities, concepts, syntheses | The LLM | +| **3 · The schema** | `CLAUDE.md` / `AGENTS.md` | The engine — it's what makes the LLM a knowledge manager rather than a chatbot | + +Layer 3 has a second file with a different owner: `CONVENTIONS.md` sits beside the schema and holds what *this* vault has decided. You write it, and the engine never touches it after creating it ([ADR-0010](docs/adr/0010-conventions-file-and-upgrade-baseline.md)). Four operations: **ingest** a source, **ask** the wiki a question, write up a **day**, and **lint** it for contradictions and gaps. *On the name:* DJs call it crate-digging — going through records to find what's worth playing. The raw sources are the crate. +## Why not just… + +**Chat history?** It's already there, and it's already failing. Scrolling back through sessions is re-reading, not remembering: nothing is condensed, nothing is cross-referenced, and the same conclusion gets re-derived every time you go looking for it. + +**RAG over your notes?** A fair alternative, and for question-answering over a fixed corpus it's the better one. The difference is what happens after the answer. Retrieval hands you fragments and re-derives the conclusion on every query; a wiki writes the conclusion down once, links it to everything it touches, and gets denser as you use it. The honest cost is maintenance — a vector index needs none, and a wiki needs `/lint` and a human who reads it. + +**Obsidian by itself?** The vault *is* an Obsidian vault — that's deliberate, and the graph view works from the first `crate init`. What Obsidian doesn't do is get your sessions into it for free or keep the cross-references honest. crate-wiki is the pipe and the discipline, not a replacement. + ## Two ideas shape the design **Code does mechanics; the LLM does judgment.** Anything with a single right answer should be code. Checking that every wikilink resolves is a `for` loop, not a reasoning task — so `crate lint` does it in Python, and the LLM is only asked the thing it's uniquely good at: *do these two pages contradict each other?* This keeps the wiki reliable rather than vibes-based. See [ADR-0004](docs/adr/0004-deterministic-cli.md) and [ADR-0020](docs/adr/0020-the-linter-reports-and-never-repairs.md), which splits the linter along that line and has it report without ever repairing. -**Capture is free; synthesis is paid.** A Stop hook parses each session to disk in pure Python at zero token cost, so capture is never the thing you forget. Synthesis only runs when you ask for it, so it never surprises your token budget. See [ADR-0002](docs/adr/0002-free-capture-paid-synthesis.md). +**Capture is free; synthesis is paid.** A Stop hook parses each session to disk in pure Python at zero token cost, so capture is never the thing you forget. Synthesis only runs when you ask for it, so it never surprises your token budget. Nothing that costs tokens is ever on an automatic trigger. See [ADR-0002](docs/adr/0002-free-capture-paid-synthesis.md). ## How it works ``` - session ends - │ - ▼ - Stop hook ──► crate capture ──► raw/sessions/… free, deterministic - │ - ▼ - you run /ingest paid, deliberate - │ - ┌──────────────────┼──────────────────┐ - ▼ ▼ ▼ - wiki/sources/ wiki/entities/ wiki/concepts/ - └──────────────────┼──────────────────┘ - ▼ - index.md + log.md + a session ends + │ + ▼ + crate capture ──────────► raw/sessions/… free · deterministic · automatic + Claude Code: a Stop hook fires pure Python, zero tokens + Codex: you run one sweep + │ + ├── /ingest ──► wiki/sources/ · entities/ · concepts/ paid · only when you ask + │ │ + │ └──► index.md + log.md + │ + └── /daily ───► wiki/daily/2026-07-24.md + reads raw/ directly — no ingest required + + then, over the wiki itself: + + /ask ──► an answer, promoted to wiki/syntheses/ when it's worth keeping + /lint ──► dead links, orphans, an index that's drifted — then the four + questions code can't answer. It reports, and never repairs. ``` -Then `/ask` queries the wiki, and answers worth keeping are promoted to `wiki/syntheses/` — so exploring compounds instead of evaporating into chat. - -And `/daily` answers the question this started from. It reads a day's session cards straight out of `raw/` — no ingest required — and writes `wiki/daily/2026-07-24.md`: an account of the day you can read in three months and have the day back. +`/ingest` is the one that builds the wiki. `/ask` is why you built it — answers worth keeping are promoted to `wiki/syntheses/`, so exploring compounds instead of evaporating into chat. `/daily` answers the question this started from: it reads a day's session cards straight out of `raw/` and writes an account of the day you can read in three months and have the day back. `/lint` is the one operation that writes nothing at all. -## Install - -```bash -uv tool install git+https://github.com/bmxcode/crate-wiki -crate --version -``` - -Then scaffold a vault: +## Set up a vault ```bash crate init ~/crate-personal --scope personal @@ -92,12 +107,20 @@ That merges a Stop hook into `~/.claude/settings.json`. It's idempotent and non- What to expect: - **Zero tokens, and it never blocks session exit.** Capture is pure Python ([ADR-0002](docs/adr/0002-free-capture-paid-synthesis.md)). If anything goes wrong — a broken vault, a missing transcript — it fails quietly and the session still ends normally. -- **One card per session.** The hook fires as the session runs and rewrites the same card in place, so a session is one file that stays current, not one per turn. +- **A day of a session is a card.** The hook fires as the session runs and rewrites the same card in place, so a session is one file that stays current, not one per turn — and a session you resume across three days is three cards, each dated to its own day and carrying that day's branch, duration and token usage ([ADR-0015](docs/adr/0015-a-day-of-a-thread-is-a-card.md), [ADR-0018](docs/adr/0018-tokens-are-captured-cost-is-external.md)). - **Failures are logged, not shown.** Every outcome, good or bad, is one line in `~/.claude/crate-capture.log` — `tail` it if a card doesn't appear. - **One machine, one vault.** Personal and work stay isolated by living on separate machines; the work machine uses `--vault ~/crate-work`. Capture keeps *everything* — deciding which sessions are worth folding into the wiki is a judgment call that happens later, when you run `/ingest`. +**Codex** works too, and produces the same cards from the same core ([ADR-0014](docs/adr/0014-shared-card-core-per-source-adapters.md)). It has no Stop hook to hang capture on — its `notify` slot fires per turn rather than on session exit, and is usually already taken — so it's swept on demand instead, either from the shell or with `/fetch-codex` inside the vault: + +```bash +crate capture codex --vault ~/crate-personal +``` + +That walks `~/.codex/sessions/` and captures every new or changed rollout in one idempotent pass. + ## Ingest, when you're ready `/ingest` is the first paid operation: it reads a raw source, works out what mattered, and folds it into the wiki. It's a slash command inside the vault, so run it from a Claude Code session there. @@ -111,7 +134,7 @@ Capture keeps *everything* — deciding which sessions are worth folding into th Then it writes the pages, regenerates `index.md`, and appends one line to `log.md`. -Four things underneath it are mechanical, so they're commands rather than judgment calls ([ADR-0008](docs/adr/0008-code-and-prompt-inside-an-operation.md)) — useful on their own: +The mechanical steps underneath it are commands rather than judgment calls ([ADR-0008](docs/adr/0008-code-and-prompt-inside-an-operation.md)) — useful on their own: ```bash crate pending --vault . # raw sources not yet in the wiki @@ -127,6 +150,26 @@ Two consequences worth knowing: - **`index.md` is generated — don't edit it.** A page's one-line index entry lives in its own `summary:` frontmatter, and `crate index` reads it from there. Anything you type into `index.md` below the header is discarded on the next regeneration. - **Re-running `/ingest` can't duplicate a page.** Whether a source is already ingested is read off the `sources:` frontmatter of `wiki/sources/` pages, so it's committed with the vault and travels with it. Delete a source page and its raw file becomes pending again, which is what you want. +## The commands + +| Command | What it does | +|---|---| +| `crate init --scope work\|personal` | Scaffold a vault: the schema, the tree, and a git repo to hold them | +| `crate install-hook --vault ` | Wire `crate capture claude` into `~/.claude/settings.json` | +| `crate upgrade ` | Refresh the engine-owned files in an existing vault | +| `crate capture claude` | Capture the current Claude Code session — what the Stop hook calls | +| `crate capture codex` | Sweep `~/.codex/sessions/` for new or changed rollouts | +| `crate pending` | Raw sources the wiki hasn't folded in yet, and ones that have gone stale | +| `crate day [yesterday]` | One day's session cards, oldest first — what `/daily` reads | +| `crate new ` | Scaffold a wiki page from the vault's template | +| `crate extend <title>` | Record that a page absorbed new material: bump `updated:`, add to `sources:` | +| `crate index` | Regenerate `index.md` from every page's `summary:` frontmatter | +| `crate log <op> --title <t>` | Append one entry to `log.md` | +| `crate fmt` | Put every page's paragraphs back on one line each | +| `crate lint` | The five checks with a single right answer; prints nothing when clean | + +Everything that operates on a vault takes `--vault`, defaulting to the current directory. `crate <command> --help` is the reference — each one carries the reasoning for why it exists at all. + ## Upgrading a vault Most of the engine lives in the installed package, so `uv tool upgrade crate-wiki` is usually all there is to it. Some files have to sit inside the vault — the schema, slash commands, page templates — so a release that changes those needs one command per vault: @@ -143,7 +186,13 @@ Overwriting the schema is safe because the vault records a hash of what the engi ## Design - [Architecture](docs/architecture.md) — the layers, the tiers, and the data flow -- [ADRs](docs/adr/) — decisions where an alternative was genuinely rejected +- [ADRs](docs/adr/) — one record per decision where a real alternative was rejected + +The ADRs are the honest part of this repo. Each one names the alternative that lost and why, and some of them record the engine changing its mind: [ADR-0010](docs/adr/0010-conventions-file-and-upgrade-baseline.md) reverses [ADR-0009](docs/adr/0009-engine-owned-vault-files.md) one deliverable after it was accepted, and [ADR-0012](docs/adr/0012-daily-reads-raw-and-earns-a-command.md) reverses the outcome of [ADR-0011](docs/adr/0011-ask-and-the-promoted-synthesis.md) on the same test. If you want to know whether the design holds up, read those rather than this file. + +## Contributing + +I'm not taking pull requests yet, but issues are welcome — see [CONTRIBUTING.md](CONTRIBUTING.md). Security reports go through [SECURITY.md](SECURITY.md). ## License diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..b2474de --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,28 @@ +# Security policy + +## Supported versions + +There's no tagged release yet. `main` is what's supported — if you're running crate-wiki, `uv tool upgrade crate-wiki` puts you on it. + +## Reporting a vulnerability + +Report it privately through GitHub: **[open a draft security advisory](https://github.com/bmxcode/crate-wiki/security/advisories/new)**. That reaches me without the report being public first. + +If that form isn't available to you, open a normal issue saying only that you have a security report and how to reach you — no details in the issue — and I'll follow up privately. + +I'm one person doing this alongside other work, so I can't promise a response time. I will acknowledge what I receive and say plainly whether I'm going to fix it. + +## What's in scope + +crate-wiki reads local AI-assistant session transcripts and writes local Markdown. The engine makes no network calls of its own and has no server and no telemetry — the one operation that reaches outward is `/lint`, which may ask *your* assistant to run a web search. So the interesting failures are all about **data staying where it should**: + +- Anything that causes vault content, session transcripts, or file paths to leave the machine. +- Anything that writes content from a **private** raw section into `wiki/`, which is committed ([ADR-0006](docs/adr/0006-private-sections-are-context-only.md) is why `crate lint` checks for exactly that). +- Anything that lets a crafted transcript file cause code execution, or a write outside the vault, when it's parsed. +- Anything that breaks the isolation between a work vault and a personal one ([ADR-0001](docs/adr/0001-local-only-work-vault.md)). + +**Out of scope:** the security of the AI assistants whose transcripts this reads, and anything you deliberately put in a vault and then pushed to a remote you chose. + +## When you report + +Please don't include real transcript content, vault pages, client names, or internal paths in the report — describe the shape of the problem, or build a synthetic file that reproduces it. This repo's history is public and permanent, and so is anything that ends up in an advisory. diff --git a/docs/going-public.md b/docs/going-public.md index 1b5c712..cd1c9bc 100644 --- a/docs/going-public.md +++ b/docs/going-public.md @@ -12,6 +12,7 @@ This repo is built to be public — the engine holds no vault content, and `raw/ ``` - **README and repo description** describe the current state, not a half-built one. +- **[CONTRIBUTING.md](../CONTRIBUTING.md) and [SECURITY.md](../SECURITY.md) exist and are true.** Both say what is actually the case — no PRs for now, issues welcome, and a private route for a security report. A file that describes a process nobody is running is worse than no file. ## At the flip @@ -50,6 +51,24 @@ JSON gh api --method PATCH "repos/$REPO" \ -f 'security_and_analysis[secret_scanning][status]=enabled' \ -f 'security_and_analysis[secret_scanning_push_protection][status]=enabled' + +# Private vulnerability reporting — the route SECURITY.md sends people to. Without this, the +# advisory form 404s and the only way to reach me is a public issue, which is the wrong shape +# for a security report. +gh api --method PUT "repos/$REPO/private-vulnerability-reporting" + +# Topics. The description is already set; these are what someone actually searches for. +gh repo edit "$REPO" \ + --add-topic llm \ + --add-topic knowledge-management \ + --add-topic personal-knowledge-management \ + --add-topic second-brain \ + --add-topic claude-code \ + --add-topic codex \ + --add-topic obsidian \ + --add-topic agent-memory \ + --add-topic python \ + --add-topic cli ``` ## Branching, once public From 95f0fbc54b6ebdc24799c7b16ae3edba91df49b4 Mon Sep 17 00:00:00 2001 From: Peter Tos <38345014+bmxcode@users.noreply.github.com> Date: Thu, 6 Aug 2026 23:26:05 +1000 Subject: [PATCH 2/3] README: expand the hand-wired hook JSON MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The snippet's job is to show the nesting — hooks -> Stop -> array -> object -> hooks -> array — and the one-line form collapsed exactly that. It now matches the shape `crate install-hook` writes, so the two can be compared side by side. Refs #27 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --- README.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 24ad2a5..97141d9 100644 --- a/README.md +++ b/README.md @@ -99,9 +99,20 @@ crate install-hook --vault ~/crate-personal That merges a Stop hook into `~/.claude/settings.json`. It's idempotent and non-destructive — re-running updates its own entry and leaves any other Stop hooks alone; point it at a new vault to move the target. Prefer to wire it by hand? Add this instead (use the absolute path from `which crate` if your hook environment doesn't have it on `PATH`): ```json -{ "hooks": { "Stop": [ { "hooks": [ - { "type": "command", "command": "crate capture claude --vault \"$HOME/crate-personal\"" } -] } ] } } +{ + "hooks": { + "Stop": [ + { + "hooks": [ + { + "type": "command", + "command": "crate capture claude --vault \"$HOME/crate-personal\"" + } + ] + } + ] + } +} ``` What to expect: From 43980520c4a87b951bda0fa641fac8ae47c8ec07 Mon Sep 17 00:00:00 2001 From: Peter Tos <38345014+bmxcode@users.noreply.github.com> Date: Fri, 28 Aug 2026 20:49:08 +1000 Subject: [PATCH 3/3] Bring the repo furniture up to samplerdisc's live standard MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit samplerdisc went public with three badges, tagged releases, a changelog, a code of conduct, issue and PR templates, and a README that says plainly what does not work yet. crate-wiki had the first half of that and not the second. Adds CHANGELOG.md with a 0.1.0 entry covering M1 and M2, CODE_OF_CONDUCT.md (Contributor Covenant 2.1, same text as samplerdisc), an issue form whose first instruction is not to paste transcripts, and a PR template carrying the verify checklist from CLAUDE.md. README gains a licence badge, an explicit install heading with the pip alternative, and two sections the samplerdisc README earns its credibility with: "What doesn't work yet" — no source ingesters, no MCP server, Codex swept by hand, the open forked-session double-count (#41), capture silent by contract, one person's scale — and "If something doesn't work", which is where the capture log stops being buried in a bullet. The status line now claims the dogfooding rather than hedging it. The runbook gains a fuller repo description, three more topics, and the commands to cut v0.1.0 once main carries this. Refs #27 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --- .github/ISSUE_TEMPLATE/bug.yml | 51 +++++++++++++++++++++++++++++++ .github/ISSUE_TEMPLATE/config.yml | 1 + .github/pull_request_template.md | 24 +++++++++++++++ CHANGELOG.md | 35 +++++++++++++++++++++ CODE_OF_CONDUCT.md | 27 ++++++++++++++++ README.md | 48 +++++++++++++++++++++++++++-- docs/going-public.md | 17 +++++++++-- 7 files changed, 198 insertions(+), 5 deletions(-) create mode 100644 .github/ISSUE_TEMPLATE/bug.yml create mode 100644 .github/ISSUE_TEMPLATE/config.yml create mode 100644 .github/pull_request_template.md create mode 100644 CHANGELOG.md create mode 100644 CODE_OF_CONDUCT.md diff --git a/.github/ISSUE_TEMPLATE/bug.yml b/.github/ISSUE_TEMPLATE/bug.yml new file mode 100644 index 0000000..9118ae9 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug.yml @@ -0,0 +1,51 @@ +name: Something's broken +description: A command fails, does nothing, or does something other than what the docs promise +labels: ["bug"] +body: + - type: markdown + attributes: + value: | + Thanks for the report. **Don't paste session transcripts, wiki pages, or anything from a work context** — this repo is public and its history is permanent. Describe the shape of the problem, or build a synthetic file that reproduces it. + - type: input + id: version + attributes: + label: Version + description: Output of `crate --version`, plus your OS and Python version. + placeholder: crate 0.1.0 · macOS 15.5 · Python 3.13 + validations: + required: true + - type: textarea + id: command + attributes: + label: Command and full output + description: The exact command you ran and everything it printed. Paste text, not a screenshot. + render: shell + validations: + required: true + - type: textarea + id: expected + attributes: + label: What you expected instead + description: If the README, `--help` or a slash command promised something different, quote the line. A promise that no longer matches the code is a real bug and worth its own report. + validations: + required: true + - type: dropdown + id: area + attributes: + label: Which part + options: + - Capture (the Stop hook, crate capture) + - A vault command (init, upgrade, index, lint, …) + - A slash command (/ingest, /ask, /daily, /lint) + - The docs are wrong + - Not sure + validations: + required: true + - type: textarea + id: log + attributes: + label: Capture log + description: If a session card didn't appear, the last few lines of `~/.claude/crate-capture.log`. Capture fails quietly by design, so this is usually the only place the reason shows up. Redact any paths you'd rather not publish. + render: shell + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000..0086358 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1 @@ +blank_issues_enabled: true diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000..626f0f1 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,24 @@ +<!-- One deliverable per branch, one issue per deliverable. The issue is the spec. --> + +Closes # + +## What this changes + + + +## Where the boundary falls + +<!-- Anything with a single right answer is code; judgment is a prompt (ADR-0004, ADR-0008). Tick what this touches. --> + +- [ ] CLI — a deterministic primitive +- [ ] A parser or the card core +- [ ] A slash command / the vault schema +- [ ] `docs/` only + +## Checklist + +- [ ] No real session data, vault content, client names or internal paths — fixtures are synthetic +- [ ] `uv run ruff check .` and `uv run ruff format --check .` pass +- [ ] `uv run pytest -q` passes +- [ ] `uv tool install --editable . && crate --version` works +- [ ] An ADR if a real alternative was rejected — and none if one wasn't diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..94f97ad --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,35 @@ +# Changelog + +Notable changes to `crate-wiki`. The design and its rejected alternatives live in [docs/adr/](docs/adr/); this file records what changed for someone using the tool. + +## Unreleased + +Nothing yet. + +## 0.1.0 — 2026-08-28 + +First public release. The loop closes: a session captures itself for free, and four operations turn captured sessions into a wiki that gets denser as you use it. + +### Capture — free, deterministic, automatic + +- **Claude Code sessions capture themselves** through a Stop hook (`crate install-hook`), in pure Python at zero token cost. It fails quietly and never blocks session exit, logging every outcome to `~/.claude/crate-capture.log` ([ADR-0002](docs/adr/0002-free-capture-paid-synthesis.md)). +- **The parser discards rather than converts.** It walks Claude Code's `parentUuid` tree to the live leaf so rewinds and abandoned branches never replay as work you did, and drops `tool_result` bodies and `thinking` blocks — roughly a tenth the size, carrying nearly all the signal, which is what makes synthesis affordable. +- **Codex sessions too**, through a second adapter over a shared card core ([ADR-0014](docs/adr/0014-shared-card-core-per-source-adapters.md)). Codex has no session-exit hook, so it's swept on demand with `crate capture codex` or `/fetch-codex`. +- **A day of a session is a card** ([ADR-0015](docs/adr/0015-a-day-of-a-thread-is-a-card.md)). A thread resumed across three days is three files, each dated to its own day and carrying that day's branch, duration and tool version. A rewind re-renders the day it changed, in place ([ADR-0016](docs/adr/0016-a-rewind-re-renders-the-day-it-changed.md)). +- **Cards are dated in local wall-clock time** ([ADR-0013](docs/adr/0013-local-session-timestamps.md)), so work after midnight lands on the day you did it. +- **Each card records the day's token usage** — input, output and cache read/write kept separate, with the model. The dollar cost is deliberately left outside the engine, where the rate table can age without the card going stale ([ADR-0018](docs/adr/0018-tokens-are-captured-cost-is-external.md)). + +### The four operations — paid, and only when you ask + +- **`/ingest`** folds a raw source into the wiki, stopping to show you its takeaways and a numbered page plan before it writes anything. There's no flag to skip that step. +- **`/ask`** queries the wiki and promotes answers worth keeping to `wiki/syntheses/`, adding no command of its own ([ADR-0011](docs/adr/0011-ask-and-the-promoted-synthesis.md)). +- **`/daily`** writes an account of a day, reading session cards straight out of `raw/` with no ingest required ([ADR-0012](docs/adr/0012-daily-reads-raw-and-earns-a-command.md)). +- **`/lint`** reports and never repairs ([ADR-0020](docs/adr/0020-the-linter-reports-and-never-repairs.md)). `crate lint` answers the five questions with a single right answer — dead wikilinks, orphans, a drifted `index.md`, a page citing a private or missing source — and the prompt asks only the four that need judgment. + +### The vault + +- **`crate init --scope work|personal`** scaffolds an Obsidian vault: the schema, the tree, the templates and a git repo. A work vault has no journal and refuses to push anywhere ([ADR-0001](docs/adr/0001-local-only-work-vault.md)). +- **Layer 3 is split.** `CLAUDE.md`/`AGENTS.md` are the engine's and `crate upgrade` refreshes them; `CONVENTIONS.md` is yours and the engine never writes it again ([ADR-0010](docs/adr/0010-conventions-file-and-upgrade-baseline.md)). +- **`crate upgrade` knows what it wrote.** A baseline of content hashes tells "the template moved" apart from "you edited this", so an edited file is reported rather than clobbered. `--adopt` claims a vault that predates the baseline; `--dry-run` shows what would change. +- **Staleness is a content comparison, not a timestamp** ([ADR-0017](docs/adr/0017-staleness-is-a-content-comparison.md)) — `git checkout` rewrites every mtime in a vault, which would report a fresh clone as entirely stale. +- **Deterministic primitives** the operations call at the points where the answer is fixed: `crate pending`, `day`, `new`, `extend`, `index`, `log`, `fmt`, `lint`. diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 0000000..57c30bc --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,27 @@ +# Contributor Covenant Code of Conduct + +## Our Pledge + +We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation. We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community. + +## Our Standards + +Examples of behavior that contributes to a positive environment include demonstrating empathy and kindness toward other people, being respectful of differing opinions and experiences, giving and gracefully accepting constructive feedback, accepting responsibility and apologizing to those affected by our mistakes, and focusing on what is best for the overall community. + +Examples of unacceptable behavior include the use of sexualized language or imagery and unwelcome sexual attention or advances, trolling, insulting or derogatory comments, personal or political attacks, public or private harassment, publishing others' private information without explicit permission, and other conduct which could reasonably be considered inappropriate in a professional setting. + +## Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful. + +## Scope + +This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement by opening an issue or contacting the maintainer through GitHub. All complaints will be reviewed and investigated promptly and fairly. All community leaders are obligated to respect the privacy and security of the reporter of any incident. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org), version 2.1, available at https://www.contributor-covenant.org/version/2/1/code_of_conduct.html. diff --git a/README.md b/README.md index 97141d9..9ecbb97 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,9 @@ An LLM wiki that compounds what you learn and what you've done. -[![CI](https://github.com/bmxcode/crate-wiki/actions/workflows/ci.yml/badge.svg)](https://github.com/bmxcode/crate-wiki/actions/workflows/ci.yml) ![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue) +[![CI](https://github.com/bmxcode/crate-wiki/actions/workflows/ci.yml/badge.svg)](https://github.com/bmxcode/crate-wiki/actions/workflows/ci.yml) +[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) +![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue) Since I started building with AI assistants, my working memory of my own work collapsed from about six weeks to about three days. crate-wiki fixes the part of that a tool can fix. Every Claude Code session is parsed to disk as it runs — pure Python, zero tokens, no assistant involved (Codex too, on a one-command sweep). Then, only when you ask, an LLM folds one into a cross-referenced wiki you can still read in three months. @@ -12,7 +14,7 @@ crate init ~/crate-personal --scope personal # an Obsidian vault, schema incl crate install-hook --vault ~/crate-personal # sessions now capture themselves ``` -> **Status: early.** Milestones 1 and 2 are done — capture, `/ingest`, `/ask`, `/daily`, `/lint` all work and have tests. [Milestone 3](https://github.com/bmxcode/crate-wiki/milestone/3) is open. There's no tagged release, the vault format can still change under `crate upgrade`, and as far as I know I'm the only person running it. Every contested decision has a record in [docs/adr/](docs/adr/), including the ones later reversed. +> **Status: early.** Milestones 1 and 2 are done — capture, `/ingest`, `/ask`, `/daily`, `/lint` all work and have tests. [Milestone 3](https://github.com/bmxcode/crate-wiki/milestone/3) is open. I run it daily on my personal and work machines. The vault format can still change under `crate upgrade`, and what doesn't work yet is listed below. Every contested decision has a record in [docs/adr/](docs/adr/), including the ones later reversed. Everything stays on your machine: capture reads local transcript files and writes local Markdown, and nothing is uploaded anywhere. The only thing that costs tokens is a synthesis you ran yourself. @@ -78,7 +80,15 @@ Four operations: **ingest** a source, **ask** the wiki a question, write up a ** `/ingest` is the one that builds the wiki. `/ask` is why you built it — answers worth keeping are promoted to `wiki/syntheses/`, so exploring compounds instead of evaporating into chat. `/daily` answers the question this started from: it reads a day's session cards straight out of `raw/` and writes an account of the day you can read in three months and have the day back. `/lint` is the one operation that writes nothing at all. -## Set up a vault +## Install, and set up a vault + +Python 3.11 or newer. The quickstart above uses [uv](https://docs.astral.sh/uv/); pip works too: + +```bash +pip install git+https://github.com/bmxcode/crate-wiki +``` + +Then scaffold a vault: ```bash crate init ~/crate-personal --scope personal @@ -194,6 +204,34 @@ It refreshes what the engine owns: `CLAUDE.md`, `AGENTS.md`, `.claude/commands/` Overwriting the schema is safe because the vault records a hash of what the engine last wrote it, in `.crate/baseline.json`. That's what separates "the template moved" from "you edited this" — a file you've changed is reported and left alone rather than clobbered, and so is one the engine has no record of writing. `crate upgrade --adopt` takes the shipped versions anyway, which is the one-time step for a vault created before that record existed. See [ADR-0010](docs/adr/0010-conventions-file-and-upgrade-baseline.md). +## What doesn't work yet + +- **Only sessions are ingested.** `raw/` scaffolds `clips/`, `youtube/` and `pastes/`, and nothing fills them — the Obsidian Clipper target, YouTube transcripts and pasted messages need normalising into raw sources first ([#10](https://github.com/bmxcode/crate-wiki/issues/10)). +- **There's no MCP search server.** [docs/architecture.md](docs/architecture.md) names one as part of the engine; it isn't built ([#11](https://github.com/bmxcode/crate-wiki/issues/11)). Today the wiki is searched by the assistant reading `index.md` and opening the pages it can name a reason for. +- **Codex captures only when you ask it to.** Its `notify` slot fires per turn rather than on session exit, and is usually already taken, so there's nothing to hang an automatic hook on. `crate capture codex` or `/fetch-codex` before an `/ingest` is the substitute — and forgetting is the failure mode. +- **A forked Claude Code session double-counts.** Rarely, Claude Code starts a new transcript beginning with a verbatim copy of a prior conversation's records under a new session id, so the same work lands on two cards ([#41](https://github.com/bmxcode/crate-wiki/issues/41), open). A per-day split doesn't close it. +- **Capture never tells you it failed.** That's the contract — it must not block session exit ([ADR-0002](docs/adr/0002-free-capture-paid-synthesis.md)) — but it means a missing card is silent until you look at the log. +- **It has only ever run at one person's scale.** Nothing here has met a large vault, a shared vault, or a corpus built by someone whose conventions differ from mine. + +## If something doesn't work + +**A session card didn't appear.** Capture fails quietly by design, so the reason is in the log and nowhere else: + +```bash +tail ~/.claude/crate-capture.log +``` + +**The wiki looks wrong.** Run the mechanical checks before reasoning about it — they're free and they're exhaustive for the questions they answer: + +```bash +crate lint --vault . # dead links, orphans, a drifted index, private or missing sources +crate pending --vault . # raw sources not folded in, and pages their source has outrun +``` + +**Something disagrees with the docs.** That's a real bug and worth an issue on its own. The README and `--help` make promises the same way a wiki page does, and promises rot. + +When you file one, **don't paste session transcripts, wiki pages, or anything from a work context** — this repo is public and its history is permanent. Describe the shape of the problem, or build a synthetic file that reproduces it. + ## Design - [Architecture](docs/architecture.md) — the layers, the tiers, and the data flow @@ -205,6 +243,10 @@ The ADRs are the honest part of this repo. Each one names the alternative that l I'm not taking pull requests yet, but issues are welcome — see [CONTRIBUTING.md](CONTRIBUTING.md). Security reports go through [SECURITY.md](SECURITY.md). +## Changelog + +[CHANGELOG.md](CHANGELOG.md) records what changed between releases and why. + ## License [MIT](LICENSE) diff --git a/docs/going-public.md b/docs/going-public.md index cd1c9bc..849218d 100644 --- a/docs/going-public.md +++ b/docs/going-public.md @@ -57,8 +57,10 @@ gh api --method PATCH "repos/$REPO" \ # for a security report. gh api --method PUT "repos/$REPO/private-vulnerability-reporting" -# Topics. The description is already set; these are what someone actually searches for. +# Description and topics — what a search result and the About sidebar show. The description +# is set but thin; this one says what it does, not just what it is. gh repo edit "$REPO" \ + --description "An LLM wiki that compounds what you learn and what you've done. Captures every Claude Code and Codex session to Markdown for free, then folds them into a cross-referenced Obsidian vault when you ask. Pure Python." \ --add-topic llm \ --add-topic knowledge-management \ --add-topic personal-knowledge-management \ @@ -68,7 +70,18 @@ gh repo edit "$REPO" \ --add-topic obsidian \ --add-topic agent-memory \ --add-topic python \ - --add-topic cli + --add-topic cli \ + --add-topic markdown \ + --add-topic note-taking \ + --add-topic developer-tools +``` + +Then cut the first release, so `uv tool install` is reproducible rather than pinned to whatever `main` happens to be: + +```bash +git tag -a v0.1.0 -m "v0.1.0 — first public release" +git push origin v0.1.0 +gh release create v0.1.0 --title "v0.1.0 — first public release" --notes-from-tag ``` ## Branching, once public