diff --git a/README.md b/README.md index 5978e43..34a8094 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ **Cuts shell-output tokens in opencode.** -Wraps only what [snip](https://github.com/edouard-claude/snip) can filter. Everything else runs byte-identical. +Routes supported shell commands through [snip](https://github.com/edouard-claude/snip). Uncertain commands run unchanged. [![npm version](https://img.shields.io/npm/v/opencode-smartsnip?style=flat-square&logo=npm&color=cb3837)](https://www.npmjs.com/package/opencode-smartsnip) [![npm downloads](https://img.shields.io/npm/dm/opencode-smartsnip?style=flat-square&color=cb3837)](https://www.npmjs.com/package/opencode-smartsnip) @@ -96,9 +96,16 @@ flowchart LR E -- yes --> W([prefix with snip]) ``` -The allowlist comes from snip's own filters (131 built-in, plus anything you drop in -`~/.config/snip/filters/`). `exclude_flags` and `require_flags` are honored, so -`git log --format=...` stays raw. `#nosnip` anywhere in a command skips the whole thing. +The allowlist is generated from snip's own filters, pinned to the v0.25.2 release (132 +files, 173 command/subcommand rules), plus anything you drop in `~/.config/snip/filters/`. +`exclude_flags` and `require_flags` are matched the way snip matches them — against every +argument, flag or not — so `git log --format=...`, `git diff -p` and `gh pr diff` all stay +raw, and `npm install` wraps while `npm view` does not. `#nosnip` anywhere in a command +skips the whole thing. + +Filters are read once at startup. A new file in `~/.config/snip/filters/` needs an +opencode restart before smartsnip routes to it. `smartsnip doctor` reports your installed +snip version and warns when it differs from the release the table was generated from. A wrong passthrough costs a few tokens. A wrong wrap breaks a command. The bias follows. @@ -131,7 +138,7 @@ any of them back. ## Getting raw output back snip can tee the original to a local file and append `[full output: /path.log]` to the -filtered result. The agent already has a Read tool, so nothing is ever lost. In +filtered result, which the agent can then Read instead of re-running the command. In `~/.config/snip/config.toml`: ```toml @@ -139,7 +146,16 @@ filtered result. The agent already has a Read tool, so nothing is ever lost. In mode = "always" ``` -`smartsnip doctor` checks this, along with the rest of your setup. +tee recovers most output, not all of it. On snip 0.25.2 the defaults are: + +- outputs under 500 bytes are not saved at all +- files are capped at 1 MiB; anything past that is cut +- only the 20 most recent files are kept, so a busy session drops its own older ones + +So treat tee as a good chance of recovery, not a guarantee. When the full output actually +matters, `#nosnip` is the reliable answer. + +`smartsnip doctor` checks the tee mode, along with the rest of your setup. One line in `AGENTS.md` makes agents use it well: @@ -157,20 +173,27 @@ bunx opencode-smartsnip discover --days 30 Replays your real opencode bash history (read-only, local) through the router: ``` -2106 commands, ~893.5k tokens of raw output +2106 commands, ~893.5k est. tokens of stored output. -FILTERED by snip: +RAN UNDER SNIP (stored output is post-filter): git 418 calls 182.4k est. tokens + +WRAP-ELIGIBLE, RAN RAW (routing would wrap these today): pnpm 184 calls 164.7k est. tokens -NO FILTER (biggest missed savings first): +NO FILTER IN SNIP (largest stored output first): python3 49 calls 73.2k est. tokens - agent-browser 83 calls 33.2k est. tokens ``` -A snip filter is ~10 lines of YAML. `bunx opencode-smartsnip install-command` adds a -`/snip-filter` command that writes and tests one for you — say `/snip-filter python3` -and the new filter is picked up automatically. +Every number is what opencode stored, so entries that already ran under snip are counted +after filtering. Nothing here is a measured saving, and a large number is a place to look, +not a filter worth writing. Pipelines, `#nosnip` calls and unparseable commands are +reported on their own lines because their stored output says nothing about the command's +own filter. + +For an actual measurement, `bun scripts/measure-savings.ts` replays stored output through +the real snip filters. `bunx opencode-smartsnip install-command` adds a `/snip-filter` +command that writes and tests a new filter for you. ## Why not wrap everything? @@ -186,8 +209,9 @@ it's where this project started. The failure modes are all known issues there: | heredocs | [#6](https://github.com/VincentHardouin/opencode-snip/issues/6) | detected, passthrough | | permission rules see rewritten commands | [#7](https://github.com/VincentHardouin/opencode-snip/issues/7) | only filterable commands change | -The router is validated against 23k+ real bash commands from actual opencode sessions — -65% of calls still get filtered, with none of the breakage. +The router is validated against a fixed corpus of 23k+ real bash commands from actual +opencode sessions. Its wrap decisions were also checked against `snip check` on the pinned +release; runtime verification remains the authority for commands outside that corpus. ¹ One feedback loop is unavoidable at this layer: opencode stores the *rewritten* command, so agents start typing `snip` themselves — sometimes on things snip can't @@ -202,7 +226,7 @@ the rare command too complex to parse, set `quiet_no_filter = true` under `[disp bun install bun test # includes a replay of 656 sanitized real-world commands bun run typecheck -bun run generate:filters # re-sync allowlist from upstream snip filters +bun run generate:filters # re-sync allowlist from the pinned upstream snip release bun run measure --days 7 # replay your real bash history through snip (the Numbers) ``` diff --git a/bin/smartsnip.ts b/bin/smartsnip.ts index 0008eb5..c54043d 100644 --- a/bin/smartsnip.ts +++ b/bin/smartsnip.ts @@ -11,8 +11,9 @@ import { homedir } from "node:os" import { join } from "node:path" import { loadConfig, DEFAULT_DENY } from "../src/config" import { buildMatchTable } from "../src/filters" -import { BUILTINS, shouldWrap } from "../src/router" +import { BUILTINS, isSnipHead, OPT_OUT_RE, shouldWrap, stripSnipPrefix } from "../src/router" import { splitTopLevel, analyzeSegment } from "../src/parser" +import { PINNED_SNIP_VERSION, resolveSnip, snipVersion } from "../src/snip-cli" import { formatTokens } from "../src/stats" const dataRoot = process.env["XDG_DATA_HOME"] ?? join(homedir(), ".local", "share") @@ -25,18 +26,32 @@ function opendb(path: string) { interface Agg { calls: number - outChars: number + /** + * Characters of tool output as opencode stored them. For a segment that ran + * under snip this is already the filtered text, so it is never a saving and + * never the raw size — only what the model actually read. + */ + storedChars: number } -/** True when a filter rule matches command+subcommand regardless of flag exclusions. */ function matchesFilterIgnoringFlags( head: string, - sub: string | null, + matcherKey: string, table: ReturnType, ): boolean { const entry = table.get(head) if (!entry) return false - return entry.subcommands.has(null) || (sub !== null && entry.subcommands.has(sub)) + return entry.subcommands.has(null) || entry.subcommands.has(matcherKey) +} + +function compareVersions(a: string, b: string): number { + const left = a.split(".").map(Number) + const right = b.split(".").map(Number) + for (let i = 0; i < 3; i++) { + const difference = (left[i] ?? 0) - (right[i] ?? 0) + if (difference !== 0) return difference + } + return 0 } function discover(days: number): void { @@ -62,68 +77,89 @@ function discover(days: number): void { .all(since) as { cmd: string | null; len: number | null }[] db.close() - const wrapped = new Map() + const alreadyFiltered = new Map() + const wouldWrap = new Map() const denied = new Map() const noFilter = new Map() - const formatExcluded: Agg = { calls: 0, outChars: 0 } - const unparseable: Agg = { calls: 0, outChars: 0 } + const formatExcluded: Agg = { calls: 0, storedChars: 0 } + const piped: Agg = { calls: 0, storedChars: 0 } + const optedOut: Agg = { calls: 0, storedChars: 0 } + const unparseable: Agg = { calls: 0, storedChars: 0 } let total = 0 let totalChars = 0 const bump = (m: Map, key: string, chars: number) => { - const a = m.get(key) ?? { calls: 0, outChars: 0 } + const a = m.get(key) ?? { calls: 0, storedChars: 0 } a.calls++ - a.outChars += chars + a.storedChars += chars m.set(key, a) } + const bumpOne = (a: Agg, chars: number) => { + a.calls++ + a.storedChars += chars + } for (const r of rows) { if (!r.cmd) continue total++ const chars = r.len ?? 0 totalChars += chars + + // an explicit opt-out is a decision, not a gap + if (OPT_OUT_RE.test(r.cmd)) { + bumpOne(optedOut, chars) + continue + } const pieces = splitTopLevel(r.cmd) if (!pieces) { - unparseable.calls++ - unparseable.outChars += chars + bumpOne(unparseable, chars) continue } - // classify every interesting top-level segment; share output chars evenly - const segs: { head: string; sub: string | null; text: string }[] = [] - let prevOp: string | null = null + // In a pipeline the stored output belongs to the last stage, so nothing + // about the head's filter can be read off these characters. + if (pieces.some((p) => p.kind === "op" && (p.text === "|" || p.text === "|&"))) { + bumpOne(piped, chars) + continue + } + + // classify every interesting top-level segment; split stored chars evenly + const segs: { head: string; sub: string | null; matcherKey: string; text: string; wasWrapped: boolean }[] = [] for (const p of pieces) { - if (p.kind === "op") { - prevOp = p.text - continue - } - const downstreamOfPipe = prevOp === "|" || prevOp === "|&" - prevOp = null - if (downstreamOfPipe || !p.text.trim()) continue // pipe consumers are never wrappable - const info = analyzeSegment(p.text) + if (p.kind === "op" || !p.text.trim()) continue + // history stores the rewritten command, so unwrap it and judge the real one + const text = stripSnipPrefix(p.text, config.snipPath) + const info = analyzeSegment(text) if (!info) continue if (BUILTINS.has(info.head)) continue // builtins are noise, not opportunity - if (info.head === "snip") continue // already-wrapped historical commands - segs.push({ head: info.head, sub: info.subcommand, text: p.text }) + // what survives stripping with a snip head is snip's own CLI (`snip gain`) + if (isSnipHead(info.head, config.snipPath)) continue + segs.push({ + head: info.head, + sub: info.subcommand, + matcherKey: info.tokens[1] ?? "", + text, + wasWrapped: text !== p.text, + }) } if (segs.length === 0) { - unparseable.calls++ - unparseable.outChars += chars + bumpOne(unparseable, chars) continue } const share = chars / segs.length for (const s of segs) { - if (shouldWrap(s.text, table, config)) { - bump(wrapped, s.head, share) + if (s.wasWrapped) { + bump(alreadyFiltered, s.head, share) + } else if (shouldWrap(s.text, table, config)) { + bump(wouldWrap, s.head, share) } else if ( table.has(s.head) && (config.deny.includes(s.head) || (s.sub && config.deny.includes(`${s.head} ${s.sub}`))) ) { bump(denied, s.head, share) - } else if (matchesFilterIgnoringFlags(s.head, s.sub, table)) { + } else if (matchesFilterIgnoringFlags(s.head, s.matcherKey, table)) { // a filter exists but the agent asked for a specific format (exclude_flags) - // — intentional decline, not a missed saving - formatExcluded.calls++ - formatExcluded.outChars += share + // — an intentional decline + bumpOne(formatExcluded, share) } else { // subcommand granularity for commands snip partially covers (e.g. "git checkout") const sub = s.sub && /^[a-z0-9:_-]+$/i.test(s.sub) ? s.sub : null @@ -134,15 +170,26 @@ function discover(days: number): void { } const top = (m: Map, n: number) => - [...m.entries()].sort((a, b) => b[1].outChars - a[1].outChars).slice(0, n) + [...m.entries()].sort((a, b) => b[1].storedChars - a[1].storedChars).slice(0, n) const line = (k: string, a: Agg) => - ` ${k.padEnd(24)} ${String(a.calls).padStart(6)} calls ${formatTokens(Math.round(a.outChars / 4)).padStart(8)} est. tokens` + ` ${k.padEnd(24)} ${String(a.calls).padStart(6)} calls ${formatTokens(Math.round(a.storedChars / 4)).padStart(8)} est. tokens` + const one = (label: string, a: Agg) => + console.log(`${label}: ${a.calls} calls, ~${formatTokens(Math.round(a.storedChars / 4))} est. tokens`) console.log(`\nsmartsnip discover — last ${days} days of opencode bash history`) - console.log(`${total} commands, ~${formatTokens(Math.round(totalChars / 4))} tokens of raw output\n`) + console.log( + `${total} commands, ~${formatTokens(Math.round(totalChars / 4))} est. tokens of stored output.`, + ) + console.log( + "Counts below are what opencode stored, not raw command output and not savings:\n" + + "snip-filtered entries were already condensed before they were stored.\n", + ) + + console.log("RAN UNDER SNIP (stored output is post-filter):") + for (const [k, a] of top(alreadyFiltered, 10)) console.log(line(k, a)) - console.log("FILTERED by snip (working for you):") - for (const [k, a] of top(wrapped, 10)) console.log(line(k, a)) + console.log("\nWRAP-ELIGIBLE, RAN RAW (routing would wrap these today):") + for (const [k, a] of top(wouldWrap, 10)) console.log(line(k, a)) const deniedTop = top(denied, 5) if (deniedTop.length) { @@ -150,30 +197,14 @@ function discover(days: number): void { for (const [k, a] of deniedTop) console.log(line(k, a)) } - console.log("\nNO FILTER (biggest missed savings first):") + console.log("\nNO FILTER IN SNIP (largest stored output first):") for (const [k, a] of top(noFilter, 10)) console.log(line(k, a)) - if (formatExcluded.calls > 0) - console.log( - `\nDECLINED — agent asked for a specific format (exclude_flags): ${formatExcluded.calls} calls, ~${formatTokens(Math.round(formatExcluded.outChars / 4))} est. tokens`, - ) - console.log( - `\nUNWRAPPABLE (heredocs/control flow/pipes-only): ${unparseable.calls} calls, ~${formatTokens(Math.round(unparseable.outChars / 4))} est. tokens`, - ) - - const best = top(noFilter, 3).filter(([, a]) => a.outChars > 100_000) - if (best.length) { - console.log("\nSuggestions:") - for (const [k] of best) { - console.log( - ` - write a snip filter for '${k}' (~5 min of YAML): https://github.com/edouard-claude/snip/blob/master/SKILL.md`, - ) - } - console.log(` then it is auto-detected — no plugin config needed (scanUserFilters).`) - console.log( - ` tip: \`smartsnip install-command\` adds a /snip-filter slash command that automates this.`, - ) - } + console.log() + if (formatExcluded.calls > 0) one("DECLINED — a specific format was requested (exclude_flags)", formatExcluded) + if (optedOut.calls > 0) one("OPTED OUT — #nosnip", optedOut) + one("PIPED — stored output is the last stage's, so the head's filter tells nothing", piped) + one("UNPARSEABLE — heredocs, control flow, subshells", unparseable) console.log() } @@ -184,25 +215,44 @@ async function doctor(): Promise { console.log("\nsmartsnip doctor\n") - // snip binary - const which = Bun.spawnSync(["sh", "-c", 'command -v "$1"', "smartsnip-doctor", config.snipPath]) - const snipAvailable = which.exitCode === 0 - if (snipAvailable) ok(`snip binary: ${which.stdout.toString().trim()}`) - else warn(`'${config.snipPath}' not on PATH — plugin will disable itself`) + const resolved = resolveSnip(config.snipPath) + if (!resolved) warn(`'${config.snipPath}' not found — plugin will disable itself`) + else { + ok(`snip binary: ${resolved}`) + const version = snipVersion(resolved) + if (!version) warn("could not read snip's version") + else if (version === PINNED_SNIP_VERSION) + ok(`snip ${version} (routing table is generated from this release)`) + else if (compareVersions(version, PINNED_SNIP_VERSION) < 0) + warn( + `snip ${version} installed; routing is generated from ${PINNED_SNIP_VERSION}.\n` + + ` Filters differ between releases, so upgrade to ${PINNED_SNIP_VERSION} or newer:\n` + + " brew upgrade snip", + ) + else + warn( + `snip ${version} installed; embedded routing rules target ${PINNED_SNIP_VERSION}.\n` + + " Check for a SmartSnip update with rules for this Snip release.", + ) + } // snip config / tee mode (reversibility) let cfgText = "" - if (snipAvailable) { - const snipCfg = Bun.spawnSync([config.snipPath, "config"], { stderr: "ignore" }) + if (resolved) { + const snipCfg = Bun.spawnSync([resolved, "config"], { stderr: "ignore" }) cfgText = snipCfg.stdout.toString() } const teeMode = cfgText.match(/tee\.mode:\s*(\S+)/)?.[1] if (teeMode === "always") - ok("tee.mode=always — every filtered output is recoverable ([full output: …] markers)") + ok( + "tee.mode=always — raw output is saved and linked as [full output: …].\n" + + " Outputs under 500 bytes are skipped; default limits are 1 MiB per file\n" + + " and 20 files. Use #nosnip when you need unfiltered output.", + ) else if (teeMode === "failures") warn( "tee.mode=failures — raw output only saved when commands fail.\n" + - " For headroom-style full reversibility set in ~/.config/snip/config.toml:\n" + + " To save it for successful commands too, set in ~/.config/snip/config.toml:\n" + ' [tee]\n mode = "always"', ) else warn("could not read snip tee mode") diff --git a/commands/snip-filter.md b/commands/snip-filter.md index 805829c..6edb981 100644 --- a/commands/snip-filter.md +++ b/commands/snip-filter.md @@ -5,8 +5,9 @@ description: Author, test, and install a custom snip filter for a command Write a custom snip filter for: $ARGUMENTS snip is a CLI proxy that filters shell output via declarative YAML pipelines -(installed at `~/.config/snip/filters/`). The opencode-smartsnip plugin -auto-detects new filters there — no further wiring needed. +(installed at `~/.config/snip/filters/`). The opencode-smartsnip plugin scans that +directory at startup, so a new filter is routed to after the next opencode restart — +no further wiring needed. ## Workflow @@ -24,7 +25,9 @@ auto-detects new filters there — no further wiring needed. 5. Test: run `snip -v ` and compare against the raw output. Iterate on the pipeline until the output is minimal but sufficient. 6. Verify the plugin picks it up: `bunx opencode-smartsnip doctor` should show - the allowlist grew by one. + the allowlist grew by one. Restart opencode before the plugin routes to it. -If $ARGUMENTS is empty, first run `bunx opencode-smartsnip discover --days 30` -and propose filters for the top "NO FILTER" commands instead. +If $ARGUMENTS is empty, first run `bunx opencode-smartsnip discover --days 30` and +propose filters for the top "NO FILTER IN SNIP" commands. Those counts are stored +output volume, not measured savings — confirm the output is actually repetitive before +writing a filter for it. diff --git a/llms.txt b/llms.txt index 39d55cb..261ff29 100644 --- a/llms.txt +++ b/llms.txt @@ -14,10 +14,13 @@ ## For agents running under this plugin -- Output of filtered commands is compressed but information-preserving. -- If a command output contains `[full output: /path/to/file.log]`, the raw - uncompressed output is in that file — Read it instead of re-running the command. -- Append `#nosnip` to any bash command to get its full raw output. +- Output of filtered commands is condensed and lines are dropped. It is not + information-preserving — treat it as a summary, not the full output. +- If a command output contains `[full output: /path/to/file.log]`, Read that file + instead of re-running the command. snip only writes it when tee is enabled, skips + outputs under 500 bytes, cuts the file at 1 MiB and keeps only the 20 most recent. +- Append `#nosnip` to any bash command when you need its full raw output. That is the + only way to be certain nothing was dropped. - `git log --format=...`, `git status --porcelain` etc. are never filtered — explicit format flags are respected. @@ -26,9 +29,11 @@ - README.md — install, configuration, design - src/parser.ts — shell-aware top-level splitter - src/router.ts — wrap decision logic -- src/filters.ts + src/builtin-filters.ts — allowlist from snip's filter table -- bin/smartsnip.ts — `smartsnip discover` (missed-savings report from opencode - history) and `smartsnip doctor` (setup verification) +- src/filters.ts + src/builtin-filters.ts — allowlist generated from snip v0.25.2's + filter table; new user filters need an opencode restart to be routed to +- src/snip-cli.ts — binary lookup, pinned version, snip's own subcommands +- bin/smartsnip.ts — `smartsnip discover` (what opencode stored, by category) and + `smartsnip doctor` (setup verification) ## Config diff --git a/package.json b/package.json index d3edfc6..1f015b7 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "opencode-smartsnip", - "version": "0.1.2", + "version": "0.2.0", "description": "Cuts shell-output tokens in opencode. Wraps only what snip can filter — everything else runs byte-identical.", "type": "module", "main": "src/index.ts", diff --git a/scripts/generate-filters.ts b/scripts/generate-filters.ts index b84125e..d1196df 100644 --- a/scripts/generate-filters.ts +++ b/scripts/generate-filters.ts @@ -3,21 +3,37 @@ * * Usage: * bun scripts/generate-filters.ts /path/to/snip/checkout - * bun scripts/generate-filters.ts # fetches from GitHub (master) + * bun scripts/generate-filters.ts # fetches the pinned upstream tag + * + * Pinned, not master: the table has to describe the snip the user actually runs. + * Generating from master once shipped filter rules that no released snip had. + * Any fetch or parse failure aborts — a partial table is worse than a stale one, + * because a missing rule silently stops wrapping and a wrong one loses output. */ import { readdirSync, readFileSync, writeFileSync } from "node:fs" import { join } from "node:path" import { extractMatchRules, type FilterRule } from "../src/filter-yaml" +import { PINNED_SNIP_VERSION } from "../src/snip-cli" + +const REF = `v${PINNED_SNIP_VERSION}` + +async function fetchOrThrow(url: string): Promise { + const r = await fetch(url, { headers: { "user-agent": "opencode-smartsnip-generate-filters" } }) + if (!r.ok) throw new Error(`GET ${url} → ${r.status} ${r.statusText}`) + return r +} async function loadFromGitHub(): Promise<{ name: string; text: string }[]> { - const list = await fetch( - "https://api.github.com/repos/edouard-claude/snip/contents/filters", - ).then((r) => r.json() as Promise<{ name: string; download_url: string }[]>) + const list = (await ( + await fetchOrThrow(`https://api.github.com/repos/edouard-claude/snip/contents/filters?ref=${REF}`) + ).json()) as { name: string; download_url: string }[] + if (!Array.isArray(list) || list.length === 0) throw new Error(`no filters listed at ${REF}`) const out: { name: string; text: string }[] = [] for (const f of list) { if (!f.name.endsWith(".yaml")) continue - out.push({ name: f.name, text: await fetch(f.download_url).then((r) => r.text()) }) + out.push({ name: f.name, text: await (await fetchOrThrow(f.download_url)).text() }) } + if (out.length === 0) throw new Error(`no .yaml filters at ${REF}`) return out } @@ -31,20 +47,27 @@ const src = process.argv[2] const files = src ? loadFromDir(src) : await loadFromGitHub() const rules: FilterRule[] = [] +const skipped: string[] = [] for (const f of files) { - const rule = extractMatchRules(f.text) - if (rule) rules.push(rule) - else console.warn(`skip (no match rule): ${f.name}`) + const fileRules = extractMatchRules(f.text) + if (fileRules.length === 0) skipped.push(f.name) + rules.push(...fileRules) } -rules.sort((a, b) => a.command.localeCompare(b.command) || (a.subcommand ?? "").localeCompare(b.subcommand ?? "")) +if (skipped.length > 0) { + console.error(`no match rule extracted from: ${skipped.join(", ")}`) + process.exit(1) +} +rules.sort( + (a, b) => + a.command.localeCompare(b.command) || (a.subcommand ?? "").localeCompare(b.subcommand ?? ""), +) -const banner = `// GENERATED by scripts/generate-filters.ts from edouard-claude/snip filters/*.yaml +const body = `// GENERATED by scripts/generate-filters.ts from edouard-claude/snip@${REF} filters/*.yaml // Do not edit by hand. Regenerate with: bun run generate:filters -` -const body = `${banner} + import type { FilterRule } from "./filter-yaml" export const BUILTIN_FILTERS: FilterRule[] = ${JSON.stringify(rules, null, 2)} ` writeFileSync(new URL("../src/builtin-filters.ts", import.meta.url), body) -console.log(`wrote ${rules.length} rules to src/builtin-filters.ts`) +console.log(`wrote ${rules.length} rules from ${files.length} filters (${REF}) to src/builtin-filters.ts`) diff --git a/src/builtin-filters.ts b/src/builtin-filters.ts index 0ac6185..d17519e 100644 --- a/src/builtin-filters.ts +++ b/src/builtin-filters.ts @@ -1,4 +1,4 @@ -// GENERATED by scripts/generate-filters.ts from edouard-claude/snip filters/*.yaml +// GENERATED by scripts/generate-filters.ts from edouard-claude/snip@v0.25.2 filters/*.yaml // Do not edit by hand. Regenerate with: bun run generate:filters import type { FilterRule } from "./filter-yaml" @@ -101,7 +101,7 @@ export const BUILTIN_FILTERS: FilterRule[] = [ }, { "command": "docker", - "subcommand": null, + "subcommand": "build", "excludeFlags": [ "--version", "-v" @@ -195,7 +195,9 @@ export const BUILTIN_FILTERS: FilterRule[] = [ { "command": "gh", "subcommand": "pr", - "excludeFlags": [] + "excludeFlags": [ + "diff" + ] }, { "command": "gh", @@ -230,8 +232,14 @@ export const BUILTIN_FILTERS: FilterRule[] = [ "subcommand": "diff", "excludeFlags": [ "--stat", + "--shortstat", + "--numstat", "--name-only", - "--name-status" + "--name-status", + "-p", + "--patch", + "-U", + "--unified" ] }, { @@ -246,7 +254,13 @@ export const BUILTIN_FILTERS: FilterRule[] = [ "--format", "--pretty", "--graph", - "--oneline" + "--oneline", + "-p", + "--patch", + "--stat", + "--numstat", + "--name-only", + "--name-status" ] }, { @@ -267,7 +281,11 @@ export const BUILTIN_FILTERS: FilterRule[] = [ "--format", "--pretty", "--name-only", - "--name-status" + "--name-status", + "-p", + "--patch", + "-U", + "--unified" ] }, { @@ -425,6 +443,11 @@ export const BUILTIN_FILTERS: FilterRule[] = [ "subcommand": null, "excludeFlags": [] }, + { + "command": "markdownlint-cli2", + "subcommand": null, + "excludeFlags": [] + }, { "command": "mise", "subcommand": null, @@ -463,7 +486,215 @@ export const BUILTIN_FILTERS: FilterRule[] = [ }, { "command": "npm", - "subcommand": null, + "subcommand": "add", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "ci", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "clean-install", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "i", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "ic", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "in", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "ins", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "inst", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "insta", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "instal", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "install", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "install-clean", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "isnt", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "isnta", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "isntal", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "isntall", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "isntall-clean", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "r", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "remove", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "rm", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "udpate", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "un", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "uninstall", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "unlink", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "up", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "update", + "excludeFlags": [ + "--version", + "-v" + ] + }, + { + "command": "npm", + "subcommand": "upgrade", "excludeFlags": [ "--version", "-v" @@ -529,7 +760,17 @@ export const BUILTIN_FILTERS: FilterRule[] = [ }, { "command": "pnpm", - "subcommand": null, + "subcommand": "add", + "excludeFlags": [] + }, + { + "command": "pnpm", + "subcommand": "i", + "excludeFlags": [] + }, + { + "command": "pnpm", + "subcommand": "install", "excludeFlags": [] }, { @@ -537,6 +778,41 @@ export const BUILTIN_FILTERS: FilterRule[] = [ "subcommand": "list", "excludeFlags": [] }, + { + "command": "pnpm", + "subcommand": "remove", + "excludeFlags": [] + }, + { + "command": "pnpm", + "subcommand": "rm", + "excludeFlags": [] + }, + { + "command": "pnpm", + "subcommand": "un", + "excludeFlags": [] + }, + { + "command": "pnpm", + "subcommand": "uninstall", + "excludeFlags": [] + }, + { + "command": "pnpm", + "subcommand": "up", + "excludeFlags": [] + }, + { + "command": "pnpm", + "subcommand": "update", + "excludeFlags": [] + }, + { + "command": "pnpm", + "subcommand": "upgrade", + "excludeFlags": [] + }, { "command": "poetry", "subcommand": null, @@ -847,7 +1123,37 @@ export const BUILTIN_FILTERS: FilterRule[] = [ }, { "command": "yarn", - "subcommand": null, + "subcommand": "", + "excludeFlags": [] + }, + { + "command": "yarn", + "subcommand": "add", + "excludeFlags": [] + }, + { + "command": "yarn", + "subcommand": "install", + "excludeFlags": [] + }, + { + "command": "yarn", + "subcommand": "remove", + "excludeFlags": [] + }, + { + "command": "yarn", + "subcommand": "up", + "excludeFlags": [] + }, + { + "command": "yarn", + "subcommand": "upgrade", + "excludeFlags": [] + }, + { + "command": "yarn", + "subcommand": "upgrade-interactive", "excludeFlags": [] } ] diff --git a/src/filter-yaml.ts b/src/filter-yaml.ts index 0562ef7..8edb2b5 100644 --- a/src/filter-yaml.ts +++ b/src/filter-yaml.ts @@ -1,28 +1,81 @@ /** * Minimal extraction of `match:` rules from snip filter YAML files. * Not a YAML parser — only reads the handful of scalar/list keys we need, - * and fails soft (returns null) on anything unexpected. + * and fails soft (returns []) on anything unexpected. */ export interface FilterRule { command: string + /** + * null — `subcommand:` absent: the filter matches any first argument. + * "" — listed explicitly by snip as the bare-invocation case (`yarn`). + * other — that exact first argument. + */ subcommand: string | null excludeFlags: string[] - /** snip `match.require_flags`: only wrap when ALL of these flags are present. */ + /** snip `match.require_flags`: only wrap when ALL of these are present. */ requireFlags?: string[] } -export function extractMatchRules(yamlText: string): FilterRule | null { +function unquote(s: string): string { + return s.trim().replace(/^["']|["']$/g, "") +} + +function parseInlineList(rest: string): string[] { + return rest + .replace(/^\[|\]$/g, "") + .split(",") + .map(unquote) +} + +/** + * Join YAML flow sequences that upstream wraps over several lines: + * + * exclude_flags: + * ["--stat", "--name-only", + * "-p", "--patch"] + * + * Without this the list reads as empty and `git diff -p` looks wrap-eligible, + * which destroys the patch body the agent asked for. + */ +const FLOW_START_RE = /^(?:(?:subcommand|exclude_flags|require_flags):\s*)?\[/ + +function joinFlowSequences(yamlText: string): string[] { + const out: string[] = [] + let pending: string | null = null + for (const raw of yamlText.split("\n")) { + if (pending !== null) { + pending += ` ${raw.trim()}` + if (!pending.includes("]")) continue + out.push(pending) + pending = null + continue + } + if (FLOW_START_RE.test(raw.trim()) && !raw.includes("]")) { + pending = raw + continue + } + out.push(raw) + } + if (pending !== null) out.push(pending) + return out +} + +/** + * One YAML file yields one rule per matched subcommand: snip v0.19+ accepts + * `match.subcommand` as a list, and npm/pnpm/yarn now use it to cover only the + * dependency-changing subcommands. Collapsing that list to a single rule is what + * made `npm view` look filterable and lose its output to the install filter. + */ +export function extractMatchRules(yamlText: string): FilterRule[] { let command: string | null = null - let subcommand: string | null = null + let subcommands: string[] | null = null const excludeFlags: string[] = [] const requireFlags: string[] = [] let inMatch = false let listTarget: string[] | null = null - const unquote = (s: string) => s.trim().replace(/^["']|["']$/g, "") - - for (const raw of yamlText.split("\n")) { + for (const raw of joinFlowSequences(yamlText)) { if (!raw.trim() || raw.trim().startsWith("#")) continue const indent = raw.length - raw.trimStart().length const line = raw.trim() @@ -38,28 +91,47 @@ export function extractMatchRules(yamlText: string): FilterRule | null { command = unquote(line.slice("command:".length)) listTarget = null } else if (line.startsWith("subcommand:")) { - subcommand = unquote(line.slice("subcommand:".length)) - listTarget = null + const rest = line.slice("subcommand:".length).trim() + const values: string[] = [] + if (rest.startsWith("[")) { + values.push(...parseInlineList(rest)) + listTarget = null + } else if (rest) { + values.push(unquote(rest)) + listTarget = null + } else { + listTarget = values // block list follows + } + subcommands = values } else if (line.startsWith("exclude_flags:") || line.startsWith("require_flags:")) { const key = line.startsWith("exclude_flags:") ? "exclude_flags:" : "require_flags:" const target = key === "exclude_flags:" ? excludeFlags : requireFlags const rest = line.slice(key.length).trim() if (rest.startsWith("[")) { - for (const item of rest.replace(/^\[|\]$/g, "").split(",")) { - const v = unquote(item) - if (v) target.push(v) - } + for (const v of parseInlineList(rest)) if (v) target.push(v) listTarget = null } else { listTarget = target } - } else if (line.startsWith("- ") && listTarget) { - listTarget.push(unquote(line.slice(2))) + } else if (line.startsWith("-") && listTarget) { + listTarget.push(unquote(line.slice(1))) + } else if (line.startsWith("[") && listTarget) { + // flow sequence on the line after its key + for (const v of parseInlineList(line)) if (v || listTarget === subcommands) listTarget.push(v) + listTarget = null } else { listTarget = null } } - if (!command) return null - return { command, subcommand, excludeFlags, ...(requireFlags.length ? { requireFlags } : {}) } + if (!command) return [] + // An explicit but empty list registers under no key at all in snip's registry, + // so the filter is dead there too — emit nothing rather than "matches anything". + const keys: (string | null)[] = subcommands === null ? [null] : [...new Set(subcommands)] + return keys.map((subcommand) => ({ + command, + subcommand, + excludeFlags: [...excludeFlags], + ...(requireFlags.length ? { requireFlags: [...requireFlags] } : {}), + })) } diff --git a/src/filters.ts b/src/filters.ts index b27baaf..669759c 100644 --- a/src/filters.ts +++ b/src/filters.ts @@ -5,26 +5,30 @@ import { BUILTIN_FILTERS } from "./builtin-filters" import { extractMatchRules, type FilterRule } from "./filter-yaml" import type { SmartSnipConfig } from "./config" +/** + * Keys mirror snip's registry: `null` is the wildcard filter that omits + * `match.subcommand`, `""` is the bare-invocation entry, anything else is an + * exact first argument. + */ +export type SubKey = string | null + export interface MatchEntry { - /** null in the set means "any subcommand" */ - subcommands: Set - /** exclude flags per subcommand key ("" for null) */ - excludeFlags: Map - /** require flags per subcommand key ("" for null) — wrap only if ALL present */ - requireFlags: Map + subcommands: Set + excludeFlags: Map + requireFlags: Map } export type MatchTable = Map -function addRule(table: MatchTable, rule: FilterRule): void { +export function addRule(table: MatchTable, rule: FilterRule): void { let entry = table.get(rule.command) if (!entry) { entry = { subcommands: new Set(), excludeFlags: new Map(), requireFlags: new Map() } table.set(rule.command, entry) } entry.subcommands.add(rule.subcommand) - entry.excludeFlags.set(rule.subcommand ?? "", rule.excludeFlags) - if (rule.requireFlags?.length) entry.requireFlags.set(rule.subcommand ?? "", rule.requireFlags) + entry.excludeFlags.set(rule.subcommand, rule.excludeFlags) + if (rule.requireFlags?.length) entry.requireFlags.set(rule.subcommand, rule.requireFlags) } /** Scan user-authored snip filters so custom filters become wrap-eligible automatically. */ @@ -35,8 +39,7 @@ export function scanUserFilters(dir = join(homedir(), ".config", "snip", "filter for (const f of readdirSync(dir)) { if (!f.endsWith(".yaml") && !f.endsWith(".yml")) continue try { - const rule = extractMatchRules(readFileSync(join(dir, f), "utf8")) - if (rule) rules.push(rule) + rules.push(...extractMatchRules(readFileSync(join(dir, f), "utf8"))) } catch { // unreadable filter — snip itself will deal with it; we just don't route to it } diff --git a/src/index.ts b/src/index.ts index 076735d..26ff133 100644 --- a/src/index.ts +++ b/src/index.ts @@ -2,6 +2,7 @@ import type { Plugin, PluginModule } from "@opencode-ai/plugin" import { loadConfig } from "./config" import { buildMatchTable } from "./filters" import { rewrite } from "./router" +import { resolveSnip } from "./snip-cli" import { formatTokens, nowUtcSnipFormat, savingsSince } from "./stats" // Use opencode's V1 plugin module shape. If the default export is not a @@ -9,19 +10,18 @@ import { formatTokens, nowUtcSnipFormat, savingsSince } from "./stats" // every runtime export and treats each one as a plugin. Keep this entry to a // single default export; import library helpers from their own modules. -const SmartSnipPlugin: Plugin = async ({ $, client, directory }) => { +const SmartSnipPlugin: Plugin = async ({ client, directory }) => { // POSIX parser — PowerShell/native Windows is a non-goal for now if (process.platform === "win32") return {} const config = loadConfig(directory) if (!config.enabled) return {} - try { - await $`command -v ${config.snipPath}`.quiet() - } catch { + if (!resolveSnip(config.snipPath)) { console.warn( `[smartsnip] '${config.snipPath}' not found in PATH — plugin disabled. ` + - "Install: brew install edouard-claude/tap/snip", + "Install: brew install edouard-claude/tap/snip, " + + "or go install github.com/edouard-claude/snip@latest", ) return {} } diff --git a/src/router.ts b/src/router.ts index f9faa1d..a805675 100644 --- a/src/router.ts +++ b/src/router.ts @@ -4,8 +4,9 @@ import { splitTopLevel, type SegmentInfo, } from "./parser" -import type { MatchTable } from "./filters" +import type { MatchEntry, MatchTable, SubKey } from "./filters" import type { SmartSnipConfig } from "./config" +import { SNIP_NATIVE_SUBCOMMANDS } from "./snip-cli" /** Shell builtins and shell-internal words that must never be wrapped. */ export const BUILTINS = new Set([ @@ -17,29 +18,35 @@ export const BUILTINS = new Set([ ]) /** Agent opt-out marker: a `#nosnip` comment anywhere disables wrapping for the call. */ -const OPT_OUT_RE = /(^|\s)#\s*nosnip\b/ +export const OPT_OUT_RE = /(^|\s)#\s*nosnip\b/ -/** - * Peel stray `snip` prefixes (one or more) off a segment, preserving leading - * whitespace and any env-assignment prefix. Returns the segment unchanged when - * there is nothing to strip or it can't be analyzed. This is what lets wrapping - * be re-decided from a clean slate: `snip snip pnpm` → `pnpm`, `snip sed` → `sed`. - */ function snipHeadNames(snipPath: string): Set { const base = snipPath.includes("/") ? snipPath.split("/").pop()! : snipPath return new Set(["snip", snipPath, base]) } -function isSnipHead(head: string, snipPath: string): boolean { +export function isSnipHead(head: string, snipPath: string): boolean { return snipHeadNames(snipPath).has(head) } -function stripSnipPrefix(segment: string, snipPath: string): string { +/** + * Peel stray `snip` prefixes (one or more) off a segment, preserving leading + * whitespace and any env-assignment prefix. This is what lets wrapping be + * re-decided from a clean slate: `snip snip pnpm` → `pnpm`, `snip sed` → `sed`. + * + * A prefix is only stray when snip would treat the next token as a program to + * run. `snip config`, `snip gain --daily` and `snip --version` are snip's own + * CLI, and stripping the head there turned them into `config`, `gain --daily` + * and a bare `--version`. + */ +export function stripSnipPrefix(segment: string, snipPath: string): string { const names = snipHeadNames(snipPath) let cur = segment for (;;) { const info = analyzeSegment(cur) if (!info || !names.has(info.head) || info.tokens.length < 2) return cur + const next = info.tokens[1]! + if (next.startsWith("-") || SNIP_NATIVE_SUBCOMMANDS.has(next)) return cur // body starts at the head; drop the first token and its trailing whitespace cur = info.leading + info.envPrefix + info.body.replace(/^\S+\s+/, "") } @@ -71,43 +78,40 @@ export function shouldWrap( const entry = table.get(info.head) if (!entry) return null - - // subcommand matching: a `null` rule matches anything; otherwise the segment's - // first non-flag argument must equal a listed subcommand - let subKey: string - if (entry.subcommands.has(null)) { - subKey = "" - } else if (info.subcommand !== null && entry.subcommands.has(info.subcommand)) { - subKey = info.subcommand - } else { - return null - } - - // honor snip's own exclude_flags (prefix matching, mirroring snip's matcher) - const excludes = entry.excludeFlags.get(subKey) ?? [] - if (excludes.length > 0) { - for (const token of info.tokens.slice(1)) { - if (!token.startsWith("-")) continue - const bare = token.split("=")[0]! - if (excludes.some((ex) => bare.startsWith(ex))) return null - } - } - - // honor snip's require_flags: wrap only if ALL required flags are present. - // NOTE: snip (≤0.15.0) checks require_flags against args[1:] only — a required - // flag in the first-argument slot is not seen (fixed on master). We mirror the - // stricter reading (tokens after the first argument), which is correct on - // 0.15.0 and merely conservative (safe passthrough) on fixed versions. - const requires = entry.requireFlags.get(subKey) ?? [] - if (requires.length > 0) { - const flags = info.tokens.slice(2).filter((t) => t.startsWith("-")).map((t) => t.split("=")[0]!) - if (!requires.every((req) => flags.some((f) => f.startsWith(req)))) return null - } + if (!matchesEntry(entry, info)) return null if (isDenied(info, config)) return null return info } +/** + * Mirror of snip's `Registry.Match` + `matchesFlags` (internal/filter/registry.go, + * v0.25.2). Three details that all cost output when they were approximated: + * + * - the subcommand key is literally the first argument, flag or not, so + * `git --no-pager log` reaches no filter and must not be wrapped; + * - an absent first argument is the `""` key, which is how `yarn` alone matches; + * - `exclude_flags`/`require_flags` are prefix-matched against *every* argument, + * positionals included, so `gh pr` excludes the literal `diff`. + * + * `info.subcommand` stays the first non-flag argument: that is the useful + * reading for the user's own deny/allow entries, not for snip's table. + */ +function matchesEntry(entry: MatchEntry, info: SegmentInfo): boolean { + const args = info.tokens.slice(1) + const firstArg = args[0] ?? "" + const candidates: SubKey[] = [] + if (entry.subcommands.has(firstArg)) candidates.push(firstArg) + if (entry.subcommands.has(null)) candidates.push(null) + + return candidates.some((key) => { + const excludes = entry.excludeFlags.get(key) ?? [] + if (excludes.some((ex) => args.some((a) => a.startsWith(ex)))) return false + const requires = entry.requireFlags.get(key) ?? [] + return requires.every((req) => args.some((a) => a.startsWith(req))) + }) +} + /** * Rewrite a bash command, prefixing wrap-eligible top-level segments with `snip`. * Returns the input unchanged whenever anything is uncertain. diff --git a/src/snip-cli.ts b/src/snip-cli.ts new file mode 100644 index 0000000..86c942e --- /dev/null +++ b/src/snip-cli.ts @@ -0,0 +1,22 @@ +export const PINNED_SNIP_VERSION = "0.25.2" + +export const SNIP_NATIVE_SUBCOMMANDS = new Set([ + "run", "check", "init", "hook", "hook-audit", "gain", "cc-economics", + "discover", "learn", "verify", "config", "trust", "untrust", "proxy", + "inspect", +]) + +export function resolveSnip(snipPath: string): string | null { + // Explicit PATH reflects runtime changes; Bun.which otherwise uses its startup snapshot. + return Bun.which(snipPath, { PATH: process.env["PATH"] ?? "" }) +} + +export function snipVersion(resolved: string): string | null { + try { + const r = Bun.spawnSync([resolved, "--version"], { stderr: "ignore" }) + if (r.exitCode !== 0) return null + return r.stdout.toString().match(/\d+\.\d+\.\d+/)?.[0] ?? null + } catch { + return null + } +} diff --git a/test/corpus.test.ts b/test/corpus.test.ts index 337ebb8..a504a03 100644 --- a/test/corpus.test.ts +++ b/test/corpus.test.ts @@ -15,6 +15,7 @@ import { join } from "node:path" import { buildMatchTable } from "../src/filters" import { rewrite } from "../src/router" import { DEFAULT_DENY, type SmartSnipConfig } from "../src/config" +import { SNIP_NATIVE_SUBCOMMANDS } from "../src/snip-cli" const config: SmartSnipConfig = { enabled: true, @@ -43,11 +44,12 @@ const corpus = loadCorpus() describe(`corpus replay (${corpus.length} real commands)`, () => { test("unwrap restores original", () => { + // `snip gain`, `snip config`, … are snip's own CLI, not a wrapped command + const native = [...SNIP_NATIVE_SUBCOMMANDS].join("|") + const unwrap = new RegExp(`(^|\\s|;|&|\\|)snip (?!${native}|-)`, "g") for (const cmd of corpus) { const out = rewrite(cmd, table, config) - const restored = out.replaceAll(/(^|\s|;|&|\|)snip (?!gain|init|config|proxy|discover)/g, "$1") - const restoredOriginal = cmd.replaceAll(/(^|\s|;|&|\|)snip (?!gain|init|config|proxy|discover)/g, "$1") - expect(restored).toBe(restoredOriginal) + expect(out.replaceAll(unwrap, "$1")).toBe(cmd.replaceAll(unwrap, "$1")) } }) diff --git a/test/discover.test.ts b/test/discover.test.ts new file mode 100644 index 0000000..70ec2e9 --- /dev/null +++ b/test/discover.test.ts @@ -0,0 +1,111 @@ +/** + * `discover` reports what opencode stored, over a fixture history database. + * The numbers it prints used to read as "raw output" and "missed savings" while + * they were neither: snip-filtered entries were stored already condensed, and a + * pipeline's stored output belongs to its last stage, not its head. + */ +import { afterAll, beforeAll, describe, expect, test } from "bun:test" +import { Database } from "bun:sqlite" +import { mkdirSync, mkdtempSync, rmSync } from "node:fs" +import { tmpdir } from "node:os" +import { join } from "node:path" + +const HISTORY: { cmd: string; output: string }[] = [ + { cmd: "snip git status", output: "x".repeat(400) }, // already filtered when stored + { cmd: "git diff", output: "y".repeat(800) }, // routing would wrap this today + { cmd: "cargo fmt", output: "z".repeat(1200) }, // snip has no filter + { cmd: "git log | head -5", output: "p".repeat(2000) }, // stored output is head's + { cmd: "cargo fmt #nosnip", output: "q".repeat(1600) }, // an explicit decision + { cmd: "for f in *; do echo $f; done", output: "u".repeat(600) }, // unparseable + { cmd: "git diff --name-only", output: "w".repeat(700) }, // declined by exclude_flags + { cmd: "git --no-pager log", output: "g".repeat(500) }, + { cmd: "curl -s https://example.com", output: "c".repeat(900) }, // denied data channel +] + +let dataHome: string +let report: string + +beforeAll(async () => { + dataHome = mkdtempSync(join(tmpdir(), "smartsnip-discover-")) + const dbDir = join(dataHome, "opencode") + mkdirSync(dbDir, { recursive: true }) + const db = new Database(join(dbDir, "opencode.db")) + db.run( + "CREATE TABLE part (id text PRIMARY KEY, message_id text NOT NULL, session_id text NOT NULL, time_created integer NOT NULL, time_updated integer NOT NULL, data text NOT NULL)", + ) + const insert = db.prepare("INSERT INTO part VALUES (?, ?, ?, ?, ?, ?)") + HISTORY.forEach((row, i) => { + const data = JSON.stringify({ + type: "tool", + tool: "bash", + state: { + status: "completed", + input: { command: row.cmd }, + output: row.output, + time: { start: Date.now() }, + }, + }) + insert.run(`p${i}`, "m", "s", Date.now(), Date.now(), data) + }) + db.close() + + const proc = Bun.spawnSync(["bun", join(import.meta.dir, "..", "bin", "smartsnip.ts"), "discover"], { + env: { ...process.env, XDG_DATA_HOME: dataHome }, + }) + expect(proc.exitCode).toBe(0) + report = proc.stdout.toString() +}) + +afterAll(() => rmSync(dataHome, { recursive: true, force: true })) + +const sectionOf = (head: string): string => { + const rest = report.slice(report.indexOf(head) + head.length) + const end = rest.search(/\n[A-Z][A-Z ,'-]+[:(]/) + return end === -1 ? rest : rest.slice(0, end) +} + +describe("discover classification", () => { + test("counts every stored bash call once", () => { + expect(report).toContain(`${HISTORY.length} commands`) + }) + + test("a historical `snip git status` counts as filtered, not as a missed gap", () => { + expect(sectionOf("RAN UNDER SNIP")).toMatch(/git\s+1 calls/) + expect(sectionOf("NO FILTER IN SNIP")).not.toMatch(/^ git\s+\d+ calls/m) + }) + + test("wrap-eligible commands that ran raw are listed separately", () => { + expect(sectionOf("WRAP-ELIGIBLE, RAN RAW")).toMatch(/git\s+1 calls/) + }) + + test("a pipeline is never credited to its head", () => { + expect(report).toMatch(/PIPED[^\n]*: 1 calls/) + }) + + test("#nosnip is a decision, not a gap", () => { + expect(report).toMatch(/OPTED OUT[^\n]*: 1 calls/) + expect(sectionOf("NO FILTER IN SNIP")).not.toContain("cargo fmt #nosnip") + }) + + test("declined, denied and unparseable each land in their own bucket", () => { + expect(report).toMatch(/DECLINED[^\n]*: 1 calls/) + expect(report).toMatch(/UNPARSEABLE[^\n]*: 1 calls/) + expect(sectionOf("DENIED by config")).toContain("curl") + }) + + test("a leading flag is a literal matcher key, not an excluded format", () => { + expect(report).toMatch(/DECLINED[^\n]*: 1 calls/) + expect(sectionOf("NO FILTER IN SNIP")).toContain("git log") + }) + + test("a genuine gap is still surfaced", () => { + expect(sectionOf("NO FILTER IN SNIP")).toContain("cargo") + }) + + test("never calls stored characters raw output or savings", () => { + expect(report).toContain("est. tokens of stored output") + expect(report).not.toContain("raw output") + expect(report.toLowerCase()).not.toContain("missed savings") + expect(report).not.toContain("~5 min of YAML") + }) +}) diff --git a/test/doctor.test.ts b/test/doctor.test.ts new file mode 100644 index 0000000..97a25ba --- /dev/null +++ b/test/doctor.test.ts @@ -0,0 +1,55 @@ +import { afterEach, describe, expect, test } from "bun:test" +import { chmodSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs" +import { tmpdir } from "node:os" +import { join } from "node:path" +import { PINNED_SNIP_VERSION } from "../src/snip-cli" + +const dirs: string[] = [] + +afterEach(() => { + for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }) +}) + +function doctor(version: string): string { + const dir = mkdtempSync(join(tmpdir(), "smartsnip-doctor-")) + const home = mkdtempSync(join(tmpdir(), "smartsnip-doctor-home-")) + dirs.push(dir, home) + + const snip = join(dir, "snip") + writeFileSync( + snip, + `#!/bin/sh\nif [ "$1" = "--version" ]; then echo "snip ${version}"; else printf 'tee.mode: always\\ndisplay.quiet_no_filter: true\\n'; fi\n`, + ) + chmodSync(snip, 0o755) + mkdirSync(join(dir, ".opencode")) + writeFileSync(join(dir, ".opencode", "smartsnip.json"), JSON.stringify({ snipPath: snip })) + + const result = Bun.spawnSync( + ["bun", join(import.meta.dir, "..", "bin", "smartsnip.ts"), "doctor"], + { cwd: dir, env: { ...process.env, HOME: home, XDG_DATA_HOME: join(home, "data") } }, + ) + expect(result.exitCode).toBe(0) + return result.stdout.toString() +} + +describe("doctor version guidance", () => { + test("older Snip recommends upgrading Snip", () => { + const output = doctor("0.24.9") + expect(output).toContain(`upgrade to ${PINNED_SNIP_VERSION} or newer`) + expect(output).toContain("brew upgrade snip") + }) + + test("matching Snip reports matching routing rules", () => { + const output = doctor(PINNED_SNIP_VERSION) + expect(output).toContain("routing table is generated from this release") + expect(output).not.toContain("upgrade") + }) + + test("newer Snip recommends checking for SmartSnip rules", () => { + const output = doctor("0.26.0") + expect(output).toContain(`embedded routing rules target ${PINNED_SNIP_VERSION}`) + expect(output).toContain("Check for a SmartSnip update") + expect(output).not.toContain("upgrade") + expect(output).not.toContain("downgrade") + }) +}) diff --git a/test/router.test.ts b/test/router.test.ts index 037a7ed..3898f70 100644 --- a/test/router.test.ts +++ b/test/router.test.ts @@ -1,5 +1,6 @@ import { describe, expect, test } from "bun:test" -import { buildMatchTable } from "../src/filters" +import { addRule, buildMatchTable } from "../src/filters" +import { extractMatchRules } from "../src/filter-yaml" import { rewrite } from "../src/router" import type { SmartSnipConfig } from "../src/config" import { DEFAULT_DENY } from "../src/config" @@ -73,7 +74,7 @@ describe("chains and pipes", () => { describe("idempotency (issue #15)", () => { test("already-snipped segments are not re-wrapped", () => { expect(rw("snip git status")).toBe("snip git status") - expect(rw("cd /x && snip pnpm lint")).toBe("cd /x && snip pnpm lint") + expect(rw("cd /x && snip pnpm install")).toBe("cd /x && snip pnpm install") }) test("rewrite is idempotent end-to-end", () => { @@ -94,8 +95,10 @@ describe("de-mimicry: strip stray snip the agent learned from persisted history" }) test("collapses agent-typed snip stacking (issue #15)", () => { - expect(rw("snip snip pnpm lint")).toBe("snip pnpm lint") + expect(rw("snip snip pnpm install")).toBe("snip pnpm install") expect(rw("snip snip snip git status")).toBe("snip git status") + // a command snip cannot filter collapses all the way down + expect(rw("snip snip pnpm lint")).toBe("pnpm lint") }) test("strips a stray snip on a pipe consumer", () => { @@ -203,28 +206,25 @@ describe("config: deny / allow / opt-out", () => { }) describe("require_flags honored (user filters like node --test)", () => { - const { extractMatchRules } = require("../src/filter-yaml") - const rule = extractMatchRules( + const rules = extractMatchRules( 'name: "node-test"\nversion: 1\nmatch:\n command: "node"\n require_flags: ["--test"]\npipeline:\n - action: "head"\n n: 5\n', ) test("yaml extraction picks up require_flags", () => { - expect(rule).toEqual({ command: "node", subcommand: null, excludeFlags: [], requireFlags: ["--test"] }) + expect(rules).toEqual([ + { command: "node", subcommand: null, excludeFlags: [], requireFlags: ["--test"] }, + ]) }) test("wraps only when required flag present", () => { const t = buildMatchTable(config) - // simulate a scanned user filter - t.set("node", { - subcommands: new Set([null]), - excludeFlags: new Map([["", []]]), - requireFlags: new Map([["", ["--test"]]]), - }) + for (const rule of rules) addRule(t, rule) expect(rewrite("node --import tsx --test app.test.ts", t, config)).toBe( "snip node --import tsx --test app.test.ts", ) expect(rewrite("node server.js", t, config)).toBe("node server.js") - // snip ≤0.15.0 cannot see a required flag in the first-arg slot — must not wrap - expect(rewrite("node --test app.test.ts", t, config)).toBe("node --test app.test.ts") + // snip matches require_flags against every argument including the first + // (registry.go Match prepends the subcommand to allArgs, v0.25.2) + expect(rewrite("node --test app.test.ts", t, config)).toBe("snip node --test app.test.ts") }) }) diff --git a/test/startup.test.ts b/test/startup.test.ts new file mode 100644 index 0000000..e008a51 --- /dev/null +++ b/test/startup.test.ts @@ -0,0 +1,125 @@ +// Linux lacks the /usr/bin/command fallback that hid issue #2 on macOS. +import { afterEach, beforeEach, describe, expect, test } from "bun:test" +import { chmodSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs" +import { tmpdir } from "node:os" +import { join } from "node:path" +import plugin from "../src/index" + +const noopClient = { tui: { showToast: async () => {} } } + +function makeStub(dir: string, name: string): string { + mkdirSync(dir, { recursive: true }) + const path = join(dir, name) + writeFileSync(path, "#!/bin/sh\nexit 0\n") + chmodSync(path, 0o755) + return path +} + +async function start(projectDir: string) { + // The Plugin input carries more than the probe path needs; opencode supplies + // the rest at runtime. + return await plugin.server({ client: noopClient, directory: projectDir } as never) +} + +function project(config?: Record): string { + const dir = mkdtempSync(join(tmpdir(), "smartsnip-startup-")) + if (config) { + mkdirSync(join(dir, ".opencode"), { recursive: true }) + writeFileSync(join(dir, ".opencode", "smartsnip.json"), JSON.stringify(config)) + } + return dir +} + +describe("startup probe", () => { + let stubDir: string + let originalPath: string | undefined + let originalHome: string | undefined + const dirs: string[] = [] + + beforeEach(() => { + stubDir = mkdtempSync(join(tmpdir(), "smartsnip-bin-")) + dirs.push(stubDir) + originalPath = process.env["PATH"] + originalHome = process.env["HOME"] + const home = mkdtempSync(join(tmpdir(), "smartsnip-home-")) + dirs.push(home) + process.env["HOME"] = home + }) + + afterEach(() => { + if (originalPath === undefined) delete process.env["PATH"] + else process.env["PATH"] = originalPath + if (originalHome === undefined) delete process.env["HOME"] + else process.env["HOME"] = originalHome + for (const d of dirs.splice(0)) rmSync(d, { recursive: true, force: true }) + }) + + test("finds an executable that is only on PATH — no shell involved", async () => { + makeStub(stubDir, "snip") + process.env["PATH"] = stubDir + const dir = project() + dirs.push(dir) + + const hooks = await start(dir) + expect(Object.keys(hooks)).toContain("tool.execute.before") + }) + + test("disables itself when the binary is genuinely missing", async () => { + process.env["PATH"] = stubDir // exists, holds no snip + const dir = project() + dirs.push(dir) + + expect(await start(dir)).toEqual({}) + }) + + test("an absolute snipPath works with PATH empty", async () => { + const abs = makeStub(stubDir, "snip-custom") + process.env["PATH"] = "" + const dir = project({ snipPath: abs }) + dirs.push(dir) + + const hooks = await start(dir) + expect(Object.keys(hooks)).toContain("tool.execute.before") + }) + + test("a non-executable file at snipPath counts as missing", async () => { + const path = join(stubDir, "snip-noexec") + writeFileSync(path, "#!/bin/sh\n") + chmodSync(path, 0o644) + const dir = project({ snipPath: path }) + dirs.push(dir) + + expect(await start(dir)).toEqual({}) + }) + + test("the started plugin's bash hook actually rewrites the command", async () => { + makeStub(stubDir, "snip") + process.env["PATH"] = stubDir + const dir = project() + dirs.push(dir) + + const hooks = await start(dir) + const before = hooks["tool.execute.before"]! + + const wrapped = { args: { command: "git status" } } + await before({ tool: "bash", sessionID: "s", callID: "c" } as never, wrapped as never) + expect(wrapped.args.command).toBe("snip git status") + + const untouched = { args: { command: "npm view react version" } } + await before({ tool: "bash", sessionID: "s", callID: "c2" } as never, untouched as never) + expect(untouched.args.command).toBe("npm view react version") + + const notBash = { args: { command: "git status" } } + await before({ tool: "read", sessionID: "s", callID: "c3" } as never, notBash as never) + expect(notBash.args.command).toBe("git status") + }) + + test("a disabled plugin registers no hooks at all", async () => { + makeStub(stubDir, "snip") + process.env["PATH"] = stubDir + const dir = project({ enabled: false }) + dirs.push(dir) + + expect(await start(dir)).toEqual({}) + }) +}) diff --git a/test/upstream-parity.test.ts b/test/upstream-parity.test.ts new file mode 100644 index 0000000..f7a2422 --- /dev/null +++ b/test/upstream-parity.test.ts @@ -0,0 +1,156 @@ +/** + * The routing table has to describe snip v0.25.2 exactly. Every case below is + * one where it did not, and each one lost output the agent had asked for. + */ +import { describe, expect, test } from "bun:test" +import { extractMatchRules } from "../src/filter-yaml" +import { buildMatchTable } from "../src/filters" +import { rewrite } from "../src/router" +import { DEFAULT_DENY, type SmartSnipConfig } from "../src/config" + +const config: SmartSnipConfig = { + enabled: true, + deny: [...DEFAULT_DENY], + allow: [], + snipPath: "snip", + scanUserFilters: false, + toast: false, + stripMimicry: true, +} +const table = buildMatchTable(config) +const rw = (cmd: string) => rewrite(cmd, table, config) + +describe("subcommand lists (npm/pnpm/yarn)", () => { + test("only the dependency-changing subcommands wrap", () => { + expect(rw("npm install")).toBe("snip npm install") + expect(rw("npm ci")).toBe("snip npm ci") + expect(rw("npm uninstall left-pad")).toBe("snip npm uninstall left-pad") + expect(rw("pnpm add react")).toBe("snip pnpm add react") + }) + + test("npm view keeps its output — the install filter would reduce it to 'ok'", () => { + expect(rw("npm view react version")).toBe("npm view react version") + expect(rw("npm run build")).toBe("npm run build") + expect(rw("npm ls --depth 0")).toBe("npm ls --depth 0") + expect(rw("pnpm run dev")).toBe("pnpm run dev") + }) + + test("an empty string in the list is snip's bare-invocation key", () => { + expect(rw("yarn")).toBe("snip yarn") + expect(rw("yarn add react")).toBe("snip yarn add react") + expect(rw("yarn test")).toBe("yarn test") + }) + + test("docker only wraps the four subcommands upstream filters", () => { + expect(rw("docker ps")).toBe("snip docker ps") + expect(rw("docker build .")).toBe("snip docker build .") + expect(rw("docker exec -it web sh")).toBe("docker exec -it web sh") + }) +}) + +describe("the subcommand key is the first argument, flag or not", () => { + test("a global flag before the subcommand reaches no filter in snip", () => { + expect(rw("git --no-pager log -5")).toBe("git --no-pager log -5") + expect(rw("git --no-pager status")).toBe("git --no-pager status") + expect(rw("npm --silent install")).toBe("npm --silent install") + expect(rw("pnpm -r install")).toBe("pnpm -r install") + expect(rw("docker --debug ps")).toBe("docker --debug ps") + }) + + test("a flag-only invocation falls to the wildcard filter's exclude list", () => { + expect(rw("yarn --version")).toBe("yarn --version") + expect(rw("tsc --version")).toBe("tsc --version") + expect(rw("tsc --noEmit")).toBe("snip tsc --noEmit") + }) + + test("user deny still reads the first non-flag argument", () => { + const cfg = { ...config, deny: [...config.deny, "git status"] } + expect(rewrite("git status", table, cfg)).toBe("git status") + }) +}) + +describe("exclude_flags match positional args too", () => { + test("gh pr excludes the literal 'diff' (snip issue #87)", () => { + expect(rw("gh pr diff 12")).toBe("gh pr diff 12") + expect(rw("gh pr list")).toBe("snip gh pr list") + expect(rw("gh pr view 12")).toBe("snip gh pr view 12") + }) +}) + +describe("exclude_flags written as a multi-line flow sequence", () => { + test("git diff/show/log keep their patch when one is asked for", () => { + expect(rw("git diff -p")).toBe("git diff -p") + expect(rw("git diff --name-only")).toBe("git diff --name-only") + expect(rw("git show -p HEAD")).toBe("git show -p HEAD") + expect(rw("git log --graph")).toBe("git log --graph") + expect(rw("git diff")).toBe("snip git diff") + expect(rw("git log -5")).toBe("snip git log -5") + }) + + test("the extractor reads a list that opens on the next line", () => { + const rules = extractMatchRules( + [ + 'name: "x"', + "match:", + ' command: "git"', + ' subcommand: "diff"', + " exclude_flags:", + ' ["--stat", "--name-only",', + ' "-p", "--patch"]', + "pipeline:", + ' - action: "head"', + ].join("\n"), + ) + expect(rules).toEqual([ + { + command: "git", + subcommand: "diff", + excludeFlags: ["--stat", "--name-only", "-p", "--patch"], + }, + ]) + }) + + test("inline and block subcommand lists both yield one rule each", () => { + const head = ['name: "x"', "match:", ' command: "pkg"'] + const tail = ["pipeline:", ' - action: "head"'] + const inline = extractMatchRules( + [...head, ' subcommand: ["a", "b"]', ...tail].join("\n"), + ) + const block = extractMatchRules( + [...head, " subcommand:", ' - "a"', ' - "b"', ...tail].join("\n"), + ) + expect(inline.map((r) => r.subcommand)).toEqual(["a", "b"]) + expect(block).toEqual(inline) + }) + + test("an omitted subcommand still means 'any first argument'", () => { + const rules = extractMatchRules( + ['name: "x"', "match:", ' command: "tsc"', "pipeline:", ' - action: "head"'].join("\n"), + ) + expect(rules).toEqual([{ command: "tsc", subcommand: null, excludeFlags: [] }]) + }) +}) + +describe("snip's own CLI is not a command to unwrap", () => { + test("native invocations survive stripMimicry intact", () => { + for (const cmd of [ + "snip --version", + "snip config", + "snip gain --daily", + "snip discover", + "snip verify --require-all", + "snip init --agent cursor", + "snip run -- git log -10", + "snip check -- npm install", + "snip -u git log", + ]) { + expect(rw(cmd)).toBe(cmd) + } + }) + + test("stray prefixes on real programs still collapse", () => { + expect(rw("snip sed -n 1p f")).toBe("sed -n 1p f") + expect(rw("snip snip git status")).toBe("snip git status") + expect(rw("git log | snip python3 -c x")).toBe("snip git log | python3 -c x") + }) +})