Skip to content

Latest commit

 

History

History
107 lines (82 loc) · 15.2 KB

File metadata and controls

107 lines (82 loc) · 15.2 KB

CLI Architecture

Command Pattern

The CLI uses argh for declarative arg parsing:

  • Each command is a FromArgs struct in its own module under src/cli/commands/
  • cli::TopLevel holds the Subcommand enum; main.rs calls argh::from_env() and dispatches
  • argh has no struct-flattening attribute, so the shared input fields (--content, --stdin, --parser, file path) are declared per command and assembled into cli::input::InputArgs for resolution

Adding Commands: Create src/cli/commands/newcmd.rs with a FromArgs struct and a run() method, add a variant to Subcommand in cli/mod.rs.

Shared Infrastructure

tsv_cli exports CLI infrastructure as a library, reused by tsv_debug for consistent UX:

  • Input handling (file, --content, --stdin) — cli/input.rs
  • File/directory discovery with extension filter, gitignore-aware ignore evaluation (hierarchical .gitignore/.formatignore/.prettierignore), and the non-git heuristic fallback — cli/discover.rs
  • JSON utilities (tab-indented serialization) — json_utils.rs

Binary Structure

  • tsv (production): Pure Rust, no external tool dependencies
    • Crates: tsv_cli
    • Commands: parse, format
  • tsv npm bin (WASM): crates/tsv_wasm/npm/cli.js, shipped in @fuzdev/tsv_wasm — a hand-written Node mirror of this CLI's contract (subcommands, flags, exit codes, output streams, traversal rules; single-threaded — --jobs is accepted for drop-in parity and ignored). Behavioral changes to format/parse here must be mirrored there and in scripts/test_npm.ts's CLI tests.
  • tsv_debug (development): Uses embedded Deno sidecar for external tools
    • Reuses tsv_cli infrastructure
    • Commands: check, compare, ast_diff, line_width, canonical_parse, format_prettier, fixture_init, fixtures_validate, fixtures_update, fixtures_update_parsed, fixtures_update_formatted, fixtures_audit, ts_fixture_audit, conformance_audit, swallow_audit (requires --features swallow_check, so default builds profile production-shaped render code), scan_audit, authoring_audit, build_fanout_audit, metrics, profile, json_profile, arena_stats, buffer_sizes, lex_diff, test262, tsc_conformance (nested: query, roundtrip, index, run, check-test — the tsgo typechecker-conformance harness for the experimental tsv_check crate, which may never ship; command surface in typechecker.md)

External Tools (via Embedded Deno Sidecar)

tsv_debug calls these external tools via an embedded Deno sidecar (spawned lazily on first use; bulk commands spawn a small pool of sidecar processes — see crates/tsv_debug/CLAUDE.md):

  1. prettier + prettier-plugin-svelte

    • Used by: compare, format_prettier, fixture management
    • Purpose: Format code, compare outputs, validate formatter behavior
  2. svelte

    • Used by: canonical_parse, ast_diff, fixture management
    • Purpose: Parse Svelte code with official compiler
  3. acorn + @sveltejs/acorn-typescript

    • Used by: canonical_parse, ast_diff, fixture management
    • Purpose: Parse TypeScript code (matches Svelte's TS parser)

Versions are pinned (exact) in crates/tsv_debug/src/deno/sidecar.ts — the source of truth; they are not repeated here. benches/js/package.json pins the same versions independently for the bench harness; keep the two in sync.

Input Handling

All content-processing commands support three input methods:

  • File path: command <file> - Auto-detects parser/type from extension
  • Content: command --content <string> --parser <type> - Requires explicit --parser svelte|typescript|css
  • Stdin: command --stdin --parser <type> - Requires explicit --parser svelte|typescript|css

parse and format also take --goal script|module (TypeScript only; default module). It selects the parse goal: at script, await is an ordinary identifier and import/export/import.meta are errors. It applies to --content/--stdin only — file paths are always formatted as modules (Svelte and CSS have no goal), and passing --goal with a path argument is a usage error (exit 2 for format). Both goals are strict; see conformance_test262.md §Strict Mode Only, Explicit Goal Axis.

parse also takes --no-locations: it emits the span-only wire — start/end offsets but no per-node loc (line/column) object, and for Svelte no name_loc either. loc is derivable from the offsets plus source, so nothing is lost for a consumer that has the source; it mirrors acorn's locations: false. No-op for CSS (parseCss emits no loc). Orthogonal to --goal (goal drives the parser, --no-locations the writer), so the two compose.

Implemented in tsv_cli/src/cli/input.rs

Multi-File Formatting

tsv format accepts any mix of files and directories:

  • Discovery: directories recurse over the JS/TS family (.ts/.mts/.cts/.js/.mjs/.cjs, all parsed as TypeScript — .jsx/.tsx are out of scope), .svelte, and .css (compound forms like .svelte.ts included). The safety nets .git, node_modules, .sl, .hg, .svn, .jj are always pruned. Explicit args are trusted to the extent the caller named the target: a file arg is included regardless of extension and regardless of the ignore files, and a hidden dir passed explicitly recurses (the heuristic doesn't prune the root it was pointed at). A directory arg is still subject to the ignore files, including via an ignored ancestor — naming pkg/dist doesn't override a dist/ rule, so an ignore rule is a scope boundary a directory arg can't step past. Symlinks inside directories are not followed; pass them explicitly.

  • Ignore files (two regimes, keyed on .git): for each directory root, the format root — the scope boundary, derived from the argument, never the cwd — is the repo root inside a git tree (a hard stop where the upward walk ends, so nothing above the repo is read and --check is reproducible) or the filesystem root outside one. The regime is decided once at the target root, and any ignored directory is pruned (its whole subtree is skipped).

    • Inside a repo, discovery honors, relative to the repo root:
      • .gitignore — hierarchical and repo-rooted exactly like git (gitignore syntax, matched against git check-ignore on case-sensitive filesystems). This goes beyond Prettier, which reads only one .gitignore and one .prettierignore, both relative to its own directory (the cwd by default), and ignores nested ones entirely.
      • .formatignore — hierarchical (one per directory from the repo root down, deeper wins), applied after .gitignore so its ! can re-include a gitignore'd path (subject to git's parent-directory rule).
      • .prettierignore — drop-in compat, honored hierarchically as well (one per directory from the repo root down, deeper wins), read as the tsv-layer fallback in any directory with no .formatignore of its own; a sibling .formatignore shadows it per-directory (used alone when present, even if that .formatignore is present-but-unreadable — a read error can't silently demote tsv's native file to prettier's). Like the hierarchical .gitignore above, this goes beyond Prettier's single cwd-relative .prettierignore — so a monorepo that runs prettier per-package (each package with its own .prettierignore) is honored from one repo-root tsv invocation. Because the shadow silently drops the sibling .prettierignore's rules for that directory (Prettier applies both files), tsv emits a non-fatal stderr warning wherever a .formatignore shadows a .prettierignore, pointing at merging the patterns into .formatignore. Compat caveat: as a tsv layer a .prettierignore ! can re-include a path .gitignore excluded (subject to git's parent-directory rule), whereas Prettier treats .gitignore and .prettierignore as independent sources OR'd together, where a .prettierignore ! can't rescue a gitignore'd file — tsv's model is the more powerful superset, and the divergence only surfaces for a .prettierignore ! targeting a gitignore'd path (rare).
    • Outside a repo, .gitignore and .prettierignore are not read (as git itself does); only .formatignore governs, hierarchically from the filesystem root down — so a ~/.formatignore is global config for loose files. A .prettierignore in the target root (the directory tsv was pointed at, where prettier would have read it) raises a non-fatal stderr warning — rename it to .formatignore, or git init — without changing what gets formatted. The warning is bounded to the target root: outside a repo tsv's regime is .formatignore-only at every depth, so this is one courtesy heads-up at the entry point (not a per-directory scan), and an ancestor of a subdirectory target has no repo boundary to anchor on.
    • Heuristic fallback: a .gitignore in scope is authoritative and turns the heuristic off; with no .gitignore, the heuristic — hidden directories plus dist/build/target — is the fallback "not source" guess, except that an explicit tsv-layer ! re-include overrides it.
    • Re-include idiom: to selectively re-include under a pruned (or otherwise ignored) directory, re-include the directory itself first — !dist/ admits the whole directory, then dist/* + !dist/keep.ts narrows it back to just the files you want. A bare !dist/keep.ts (without !dist/) is a no-op — the heuristic prunes dist before descending, mirroring git's parent-directory rule (a gitignored dist/ likewise blocks a later !dist/keep.ts). tsv emits a stderr warning for this case (non-fatal — no effect on the exit code, stdout, or --list/--check output), pointing at the !dir/ escape.
    • Subdirectory invocation: because the boundary is found by walking up, the repo-root rules apply even from a subdirectory, and formatting a subdirectory directly gives the same result as formatting it via an ancestor. But a tree that contains repos (a non-repo directory with .git subdirectories below it) does not honor the inner repos' .gitignores — run tsv per repo.
    • Unreadable ignore files: a .gitignore/.formatignore/.prettierignore that is present but can't be read (invalid UTF-8 — reading is strict UTF-8 on both the native and WASM CLIs — or a permission error) is not silently treated as absent: tsv emits a non-fatal stderr warning and drops that file's rules (so an unreadable .gitignore also leaves the build-output heuristic on for its subtree). A file that genuinely isn't there, or is deleted between the directory listing and the read, stays silent. This is also a --check reproducibility hazard — surfacing it is the point.
    • --check reproducibility assumes the ignore files are committed: a local/uncommitted .formatignore or .prettierignore (or git's unread .git/info/exclude / core.excludesFile) makes a clean CI checkout disagree.
    • Shared by construction: the matcher is the tsv_ignore crate's IgnoreStack; the per-directory prune/descend policy (heuristic, safety nets, the shadow warning) is the tsv_discover crate's verdict. The WASM CLI and editors call into the same two crates, so all three surfaces agree rather than hand-mirroring the logic. See cli/discover.rs.
  • Fail-fast args, isolated traversal: path args that don't resolve to a file or directory fail the whole run before anything is written (every bad arg reported); traversal errors below a valid root (e.g. an unreadable subdirectory) report to stderr and discovery continues.

  • No per-file options: formatting style is fixed (see CLAUDE.md §Configuration). In particular <svelte:options preserveWhitespace /> is not detected — whitespace handling is uniform, with only <pre>/<textarea> content whitespace-sensitive; see conformance_svelte.md §Template Whitespace.

  • Deduplication: with multiple path args, overlapping spellings of the same file (src vs ./src, absolute vs relative, symlink aliases) dedupe by canonical path, keeping the first spelling in sorted order. A single root can't produce duplicates, so the canonicalization cost is skipped.

  • In-place writes: files are rewritten only when output differs (no mtime churn). --content/--stdin keep printing to stdout.

  • --check: lists files that would change without writing; exits 1 if any would. For CI. Also works with --content/--stdin (nothing printed to stdout; the exit code is the API) for editor integrations.

  • --list: prints the discovered in-scope files (one per line) without formatting — a read-only view of the set format would touch, after the ignore files are applied. Path mode only (errors with --content/--stdin) and mutually exclusive with --check. Unlike the format action, an empty scope is a valid answer (exit 0, no output) rather than the "no supported files" error; traversal errors still exit 2. Useful for debugging ignore-file scoping and for scripting over the set.

  • Parallelism: files format concurrently on std::thread::scope workers claiming one file at a time from a shared queue — dynamic load balancing with no thread-pool dependency. --jobs N overrides the worker count; path mode only, an error with --content/--stdin.

    The default is min(logical CPUs, ceil(1.5 × physical cores)), not one worker per logical CPU. This workload does not scale onto SMT siblings — the per-file work is memory-bound, and on a large tree the discovery walk is the bottleneck, so extra workers compete with it for cores. One worker per logical CPU costs up to 28% on walk-bound trees while buying nothing on flat repos. The SMT width is read once from /sys/devices/system/cpu/cpu0/topology/thread_siblings_list; where that is unavailable (no SMT, or a non-Linux platform) the cap is inert and the default is the logical count, so it can only ever lower the worker count.

  • Streaming discovery: a single directory root — the common invocation — feeds the workers as the walk finds files, so the directory walk runs beside the first files' parse+format rather than in front of an idle pool. It is worth having: the walk is 5–10% of the wall on an application repo, and 40–67% on a repo with a large tree, where it can outrun what the pool consumes. Other argument shapes (explicit files, multiple roots) discover the whole set first, because the canonical-path dedup above is set-wide. The set of files formatted is identical either way, as is the reporting order below — only the order work is handed out differs.

  • Error isolation: a per-file read/parse/write error (or panic, caught via catch_unwind — effective only in builds with panic = "unwind"; release uses panic = "abort") reports to stderr and processing continues.

  • Deterministic reporting: changed paths print to stdout in sorted-path order regardless of completion order; errors (traversal and per-file) and the summary line go to stderr.

  • Exit codes: 0 clean, 1 would-change (--check only), 2 errors.