Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,31 @@ All notable changes to this project are documented here. The format is based on
## [Unreleased]

### Added
- **Markdown flattening** (`--markdown` / `-m`; on in `--email` / `--plain`):
drops headings, `**bold**`/`*italic*`/`~~strike~~`, inline `` `code` `` and list
markers, rewrites `[text](url)` → `text (url)`, and unwraps autolinks/images.
Fenced and inline code are kept **verbatim** — shielded from inline stripping
and typography via a sentinel pass. New `flattenMarkdown` transform.
- **HTML → text** (`--html`): strip tags, drop script/style, decode entities.
- **PowerShell cleanup** (`--powershell` / `--ps`): drop `~~~~` underlines and the
`+ ` continuation gutter from pasted error records (block-anchored so diffs are
safe). New `cleanPowerShell` transform.
- **Shell-prompt stripping** (`--prompts`): `$ `, `PS C:\>`, `>>>`, `user@host:…$`.
- **Paragraph reflow** (`--reflow`): rejoin hard-wrapped prose for proportional
fonts (conservative; under-joins rather than over-joins).
- **`--agent` preset**: denoise for feeding output *into* a model — strip
ANSI/box/glyph noise but keep Markdown structure and Unicode.
- **UTF-16 / BOM input decoding**: Windows files and clipboard exports no longer
mojibake. New `decodeInput` helper.
- **MCP server** (`socb-mcp`): exposes `sanitize_text` to Claude Desktop / Claude
Code / Codex. `@modelcontextprotocol/sdk` is an optional dependency.
- **Experimental PreToolUse hook** (`hooks/socb-clean.mjs`) that pipes noisy Bash
output through `socb --agent` before it reaches the model's context.
- Markdown pipe tables with GitHub alignment delimiters (`| :--: |`) are now
recognized by the table engine.
- Strip Markdown code fences (```` ``` ````/`~~~`) and PowerShell `~~~~~` error
underlines so pasted output is fence-free (on by default; `--no-fences` to
keep them). New `stripFences` option / exported transform.
- Initial release: `sickofcodeblocks` / `socb` CLI.
- Core pipeline: OSC 8 hyperlink rewrite, carriage-return/spinner collapse,
augmented ANSI/OSC/DCS/APC escape stripping, Nerd Font/Powerline glyph
Expand Down
107 changes: 94 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,22 @@
[![license: MIT](https://img.shields.io/npm/l/sickofcodeblocks.svg)](./LICENSE)
[![node](https://img.shields.io/node/v/sickofcodeblocks.svg)](https://nodejs.org)

> Paste terminal output into email, Slack, and docs as clean text — no screenshots, no triple-backtick code blocks.
> Paste terminal output — and Markdown / AI answers — into email, Slack, and docs as clean text. No screenshots, no triple-backtick code blocks.

Raw terminal output is full of stuff that turns to garbage the moment it leaves the terminal: ANSI color codes, progress bars that redrew themselves 200 times, spinner frames, Nerd Font icons that show up as `□`, and box-drawing tables that fall apart in a proportional font. `socb` cleans all of it and gives you plain, readable text.

Stripping colors is the easy 10%. `socb` also does the parts other tools skip:

- **Collapses progress bars & spinners** to their final state (resolves carriage-return / backspace / erase-line redraws).
- **Flattens multi-line redraws** (docker / cargo / pip) with an optional headless-terminal mode (`--emulate`).
- **Removes Nerd Font / Powerline glyphs** (Private Use Area) that render as tofu.
- **Rebuilds box-drawing tables** into clean aligned columns.
- **Flattens Markdown to readable prose** — drops `#` headings, `**bold**`, inline `` `code` `` and list markers, and rewrites `[text](url)` to `text (url)`; fenced code is kept **verbatim**. (On with `--email` / `--plain`, or `-m`.)
- **Converts HTML to text** — strips tags and decodes entities (`&` → `&`).
- **Tidies PowerShell errors** (`~~~~` underlines + the `+` gutter) and **strips shell prompts** (`$`, `PS C:\>`, `>>>`).
- **Reflows hard-wrapped paragraphs** so prose flows in a proportional-font email instead of breaking mid-line.
- **Removes Nerd Font / Powerline glyphs** (Private Use Area) that render as tofu, and **rebuilds box-drawing tables** (incl. Markdown pipe tables) into clean aligned columns.
- **Strips embedded escape strings** that `strip-ansi` leaks — OSC, plus DCS/APC (sixel, Kitty graphics).
- **Rewrites hyperlinks** (`OSC 8`) to `text (url)` instead of dropping the URL.
- **Normalizes** smart quotes/dashes/ellipses to ASCII and tidies whitespace.
- **Normalizes** smart quotes/dashes/ellipses to ASCII, **decodes UTF-16 / BOM input** (no Windows mojibake), and tidies whitespace.
- **Optionally redacts** secrets/PII before you share output up the chain (`--redact`).

Zero runtime dependencies on the default path. Works on Windows, macOS, and Linux.
Expand Down Expand Up @@ -117,6 +120,13 @@ Input priority: [file] argument > --clip > piped stdin
| `--no-links` | | Don't rewrite OSC 8 hyperlinks |
| `--strip-emoji` | | Remove emoji (+ ZWJ / variation selectors / skin tones) |
| `--no-glyphs` | | Keep Nerd Font / Private-Use glyphs |
| `--no-fences` | | Keep Markdown code fences + PowerShell `~` underlines (default strips them) |
| `--markdown` | `-m` | Flatten Markdown to readable text (code kept verbatim); on in `--email` / `--plain` |
| `--no-markdown` | | Keep Markdown markup literal (the default) |
| `--html` | | Strip HTML tags and decode entities (`&` → `&`) |
| `--prompts` | | Strip leading shell prompts (`$`, `PS C:\>`, `>>>`) |
| `--reflow` | | Rejoin hard-wrapped prose into flowing paragraphs |
| `--powershell` / `--ps` | | Tidy pasted PowerShell errors (`~` underlines + `+` gutter) |
| `--no-typographic` | | Keep smart quotes / em-dashes / ellipsis (don't convert to ASCII) |
| `--arrows` | | Also convert arrows (`→` becomes `->`) |
| `--expand-tabs` / `--tab-width <n>` | | Convert tabs to spaces (default width 4) |
Expand All @@ -125,18 +135,47 @@ Input priority: [file] argument > --clip > piped stdin
| `--redact` | `-r` | Mask API keys, JWTs, emails, IPs, and home-dir paths |
| `--watch` | `-w` | Keep cleaning the clipboard in place (Ctrl+C to stop) |
| `--interval <ms>` | | `--watch` poll interval (default 800) |
| `--slack` / `--email` / `--plain` | | Presets (below) |
| `--slack` / `--email` / `--plain` / `--agent` | | Presets (below) |
| `--help` / `--version` | `-h` / `-v` | |

### Presets

Presets set a bundle of defaults; any explicit flag still overrides them.
Presets set a bundle of defaults; any explicit flag still overrides them (e.g. `--email --no-markdown`).

| Preset | Tables | Emoji | Typography | Notes |
|---|---|---|---|---|
| `--slack` | reconstruct | keep | → ASCII | Slack renders Unicode; aligned tables read well |
| `--email` | strip | strip | → ASCII | Proportional fonts can't align columns |
| `--plain` | strip | strip | → ASCII | Maximum compatibility; arrows + tabs→spaces too |
| Preset | Tables | Markdown / HTML | Notes |
|---|---|---|---|
| `--slack` | reconstruct | left as-is | Slack renders Unicode; aligned tables read well |
| `--email` | strip | **flattened** | + reflow wraps, strip prompts, tidy PowerShell. For proportional fonts (email / Docs / Word) |
| `--plain` | strip | **flattened** | The `--email` bundle + arrows + tabs→spaces. Maximum compatibility |
| `--agent` | strip | kept (structure) | Denoise for feeding output **into** a model: strips ANSI/box/glyph noise but keeps Markdown + Unicode (no flattening, no ASCII folding) |

## Markdown & rich paste

Pasting a Claude / ChatGPT answer (or a README) into email drags along `#`, `**`, backticks, and ` ``` ` fences. `--markdown` (`-m`, and on by default in `--email` / `--plain`) flattens it to plain prose while keeping fenced code **verbatim**:

Input:

````md
## Deploy
1. Run `npm ci`, then **build**.
2. See the [runbook](https://wiki/runbook).

```bash
export TOKEN=abc # keep me exactly
```
````

`socb --email`:

```
Deploy
1. Run npm ci, then build.
2. See the runbook (https://wiki/runbook).

export TOKEN=abc # keep me exactly
```

It's **opt-in**, so a bare `socb` never mangles raw terminal output (where `*`, `_`, `#`, backticks are literal). `--html` does the same for HTML copied from a browser, Slack, or Teams.

## Tables

Expand All @@ -159,15 +198,57 @@ socb --clip

(`cmd.exe` and POSIX shells pipe bytes faithfully, so `some-command | socb` is fine there.)

## Use with AI agents (MCP + hook)

`socb` ships an MCP server so Claude Desktop, Claude Code, and Codex can clean text on demand, plus an experimental hook that denoises shell output before it reaches the model.

### MCP server

`socb-mcp` is a stdio MCP server exposing one tool, `sanitize_text(text, preset?, options?)`.

Claude Code / Claude Desktop — add to `.mcp.json` (project) or your user config:

```json
{ "mcpServers": { "socb": { "command": "socb-mcp" } } }
```

Codex — add to `~/.codex/config.toml`:

```toml
[mcp_servers.socb]
command = "socb-mcp"
```

(If `socb-mcp` isn't on `PATH`, use `npx -y sickofcodeblocks socb-mcp`.) The server needs the optional `@modelcontextprotocol/sdk` dependency, which installs by default with the package — the plain CLI path stays dependency-free.

### Auto-clean hook (experimental)

`hooks/socb-clean.mjs` is a Claude Code **PreToolUse** hook that rewrites noisy Bash commands (`npm`, `pip`, `cargo`, `docker`, `pytest`, …) to pipe their output through `socb --agent`, cutting ANSI / progress-bar tokens before they hit the model's context. It preserves the original exit code and falls back to raw output if `socb` is missing. Enable it in `.claude/settings.json`:

```json
{
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [{ "type": "command", "command": "node /abs/path/to/hooks/socb-clean.mjs" }]
}
]
}
}
```

It is conservative (only known-noisy tools; never multi-line / heredoc commands) and **experimental** — the MCP `sanitize_text` tool is the dependable path. On Windows it relies on the Bash tool running through git-bash (byte-faithful pipes); native PowerShell piping is not, so don't wire it to a PowerShell runner.

## Use as a library

```ts
import { sanitize } from "sickofcodeblocks";

const clean = await sanitize(rawTerminalOutput, { tableMode: "strip", redact: true });
const clean = await sanitize(rawTerminalOutput, { tableMode: "strip", markdown: true, redact: true });
```

Individual transforms (`stripEscapes`, `resolveOverwrites`, `transformTables`, `redact`, …) are exported too.
Individual transforms (`stripEscapes`, `resolveOverwrites`, `transformTables`, `flattenMarkdown`, `htmlToText`, `cleanPowerShell`, `stripPrompts`, `reflowParagraphs`, `redact`, `decodeInput`, …) are exported too.

## License

Expand Down
99 changes: 99 additions & 0 deletions hooks/socb-clean.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
#!/usr/bin/env node
// EXPERIMENTAL, opt-in PreToolUse hook for Claude Code.
//
// Goal: denoise the output of noisy Bash commands through `socb --agent` BEFORE
// it enters the model's context, to cut tokens and strip ANSI / progress-bar /
// box-drawing / glyph noise.
//
// Why PreToolUse and not PostToolUse: a PostToolUse hook fires AFTER the tool
// output is already committed to context and cannot rewrite it. A PreToolUse hook
// CAN rewrite the command (hookSpecificOutput.updatedInput), so we wrap it to
// capture combined stdout+stderr to a temp file, print the cleaned version, and
// preserve the original exit code.
//
// Conservative by design: only wraps a known set of noisy build/test/infra tools,
// and never a command that is multi-line, already pipes through socb, or reads a
// heredoc. Everything else passes through byte-for-byte. Requires `socb` on PATH
// (`npm i -g sickofcodeblocks`); if socb is missing at run time the wrapper falls
// back to the raw output so nothing is lost.
//
// Caveats: the Claude Code Bash tool runs commands through bash (git-bash on
// Windows, where pipes are byte-faithful — note that native PowerShell piping is
// not, so do NOT wire this to a PowerShell-based runner without the temp-file
// form below). Wrapping changes how stdout/stderr interleave (both go to one
// stream). Verify behavior with your Claude Code version before relying on it.
//
// Install in .claude/settings.json:
// {
// "hooks": {
// "PreToolUse": [
// {
// "matcher": "Bash",
// "hooks": [
// { "type": "command", "command": "node /ABS/PATH/hooks/socb-clean.mjs" }
// ]
// }
// ]
// }
// }

import { readFileSync } from "node:fs";

const NOISY = new Set([
"npm", "pnpm", "yarn", "npx", "bun",
"pip", "pip3", "poetry", "uv",
"cargo", "go", "mvn", "gradle", "make",
"docker", "docker-compose", "kubectl", "helm", "terraform",
"pytest", "tox", "jest", "vitest", "mocha", "vite", "webpack", "tsc", "eslint",
"apt", "apt-get", "brew", "dnf", "yum",
]);

/** Emit a no-op result (leave the command untouched) and exit. */
function passthrough() {
process.stdout.write("{}");
process.exit(0);
}

let event;
try {
event = JSON.parse(readFileSync(0, "utf8"));
} catch {
passthrough();
}

if (!event || event.tool_name !== "Bash") passthrough();

const command = event.tool_input && event.tool_input.command;
if (typeof command !== "string" || command.trim() === "") passthrough();

// Skip anything we shouldn't touch.
if (command.includes("\n") || /\bsocb\b/.test(command) || command.includes("<<")) {
passthrough();
}

// First meaningful token (skip leading VAR=val assignments and "sudo").
const tokens = command.trim().split(/\s+/);
let i = 0;
while (i < tokens.length && /^[A-Za-z_][A-Za-z0-9_]*=/.test(tokens[i])) i++;
if (tokens[i] === "sudo") i++;
const tool = (tokens[i] || "").split("/").pop();
if (!NOISY.has(tool)) passthrough();

// Run the command, capture combined output to a temp file, clean it, and keep
// the original exit code. (bash syntax — the Bash tool runs through bash.)
const inner = command.trim().replace(/;+$/, "");
const wrapped =
'__socb_t="$(mktemp)"; { ' +
inner +
'; } >"$__socb_t" 2>&1; __socb_rc=$?; ' +
'socb --agent <"$__socb_t" 2>/dev/null || cat "$__socb_t"; ' +
'rm -f "$__socb_t"; exit $__socb_rc';

process.stdout.write(
JSON.stringify({
hookSpecificOutput: {
hookEventName: "PreToolUse",
updatedInput: { command: wrapped },
},
}),
);
Loading
Loading