mdq: standalone markdown query and edit package, with a jq-like CLI

[DavertMik]

Extracts src/utils/markdown-query.ts into src/utils/mdq/ as a standalone npm package that can query and edit markdown, plus a jq-like CLI runnable with npx mdq.

Design: docs/superpowers/specs/2026-09-14-mdq-package-design.md · Plan: docs/superpowers/plans/2026-09-15-mdq-package.md

Shape

One class, two files. query.ts holds MarkdownQuery — the document plus the set of blocks currently selected — with the query engine as private methods below the public API. edit.ts holds pure functions over (source, ranges); it imports types from query.ts type-only and never touches a class value, so the split is acyclic. Reads narrow the selection, writes return a fresh query over the edited document.

mdq(source)
  .query('section("API")').append('## Notes\n')
  .query('blockquote[0]').remove()
  .toString();

toString() is always the whole document; text() is the current selection.

New beyond the move

• Write verbs — remove insertBefore insertAfter prepend append addRow addItem setEntry, each returning a MarkdownQuery so edits chain.
• comment selector — matches block comments on their inner body, so comment(/^test/) reaches the multi-line <!-- test … --> blocks that test-plan-markdown.ts:122+ hand-parses today. Plus html.
• Frontmatter — parsed with yaml's Document API, kept out of the token index, exposed via frontmatter()/setFrontmatter(); comments survive a write. Without this, marked reads a knowledge file's --- block as a setext heading titled url: /login.
• Value matchers — string (exact), RegExp (own flags), or predicate, removing the hand-escaping at researcher/focus.ts:77.
• Sugar — section heading table comment … each exactly query(sel, matcher), plus at/slice/exists.
• Loud failures — an unknown selector throws MdqSelectorError with a position instead of silently matching nothing.
• CLI + package — npx mdq 'section("API") table' --json README.md. Exit codes compose like grep: 0 match, 1 no match, 2 error.

Fixes to existing behaviour

• Regex matchers honored their flags nowhere: /x/ was always case-insensitive while "x" and ~"x" were case-sensitive. Now /x/i is insensitive and /x/ is not. One production call site depended on the old behaviour (researcher.ts:316) and is now explicit.
• Table text-matching only ever saw headers, so table(~"GET") could never match a cell.
• Reading a file no longer waits on stdin. readFileSync(0, 'utf8') throws EAGAIN on Node when stdin is a pipe with no data ready — Bun tolerates it, Node dies with a stack trace. Stdin is now read asynchronously, and only when no file argument is given.

Migration

All 54 existing call sites keep working through a re-export shim at src/utils/markdown-query.ts; deprecated aliases (get toJson keyValue setKeyValue meta before after) cover every read rename.

Writes could not be shielded — they return MarkdownQuery now — so 17 call sites were migrated. The design doc predicted eleven; type-checking by type name repo-wide rather than by file path found six more. One deserves attention: research-result.ts did if (updated === this.text) return;, which a class instance makes always-false — a guard that silently stops firing rather than crashing. Another fed the result to marked.lexer, which does not coerce and threw at runtime.

This repo runs tsc with --noCheck, so none of that surfaces in a green CI. The gate used here was bunx tsc -p tsconfig.json --noEmit | grep -E "MarkdownQuery|Markdown", which returns nothing.

Whitespace

marked separators are uneven — a heading raw is "# A\n\n", a mid-document paragraph raw is "para" with a sibling space token, and a document-final paragraph raw is "last\n" with none. The invariant is never zero blank lines between blocks, never more than one, and two rules follow, both derived from real token output:

• remove() collapses only at end-of-document. The obvious alternative ("no trailing space → trim backwards") turns '# A\n\n## B\n\ntext\n' into '# A\ntext\n'.
• insertAt normalizes only the seam. A global \n{3,} collapse would rewrite blank lines inside fenced code blocks, whose raw genuinely contains them.

Packaging

Mirrors how prima-cli ships. src/utils/mdq/package.json is the standalone manifest; scripts/build-mdq-npm.ts bundles both entries with bun build --target=node (marked, yaml, commander stay external) and emits declarations; .github/workflows/publish-mdq.yml publishes on a release tagged mdq-*, gated on that prefix so an explorbot release cannot publish it by accident.

Result is 11.5 kB, 6 files, 3 dependencies. The workflow's smoke test installs the tarball into a clean project and exercises the CLI, stdin, the EAGAIN case, exit codes and a library import on Node — the same sequence was run locally. The mdq bin is removed from explorbot's package.json so the two packages cannot collide on the binary name.

To publish: cut a release tagged mdq-v0.1.0.

On the grammar

The selector grammar is bespoke — it is not a standard and has no upstream parser. The README now says so, names the real alternative (unist-util-select, CSS selectors over mdast via css-selector-parser), and states the three things that do not fall out of that stack: mdast is flat so there is no section; remark-sectionize adds the nesting but its nodes carry no position, which splice-based editing needs; and mdast headings have no flat text field while CSS dropped :contains() and unist-util-select ignores the attribute i flag.

A spike compiled the terse grammar to that CSS and got 23/23 identical results against this engine, so a migration is viable — at the cost of 5 packages / 2.0 MB becoming 79 / 7.9 MB, and rebuilding edit.ts on derived section ranges. Not done here; recorded so the option is understood.

Tests

1547 unit + 138 prima, zero failures. The original 110-test suite moved to tests/unit/mdq/query.test.ts intact as the regression net; six new suites cover frontmatter, selectors, sugar, edits, structural inserts and the CLI. Every README example was executed before committing.

Regression

This touches 17 call sites across researcher, planner and experience-tracker. A regression run looks worthwhile, but the label is the maintainer's call — none was applied.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FhJNfxLquFknSLJ6V8J9iD

[DenysKuchma]

The npm mdq binary keeps a Bun shebang, unlike the other CLIs rewritten to Node. Be better add it to the build-npm.sh rewrite