Skip to content

chore(cli): stop installing libraries the bundle already contains - #457

Merged
theCodeDrift merged 1 commit into
mainfrom
chore/bundled-deps-to-dev
Oct 6, 2026
Merged

theCodeDrift merged 1 commit into
mainfrom
chore/bundled-deps-to-dev

Conversation

@theCodeDrift

@theCodeDrift theCodeDrift commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Every install of @taskless/cli downloads 12 JavaScript libraries that never run. vite inlines every npm package into dist/: rollupOptions.external in vite.config.ts lists only Node builtins. So zod, yaml, citty, @clack/prompts, posthog-node and the rest are already compiled into the bundle, and the copies in node_modules are never loaded.

This moves those 12 to devDependencies. Two things stay as real install-time dependencies, because the CLI looks them up at runtime:

  • tsx: src/rules/runtime/invoke.ts resolves tsx/package.json and spawns it to run runtime rules.
  • The ast-grep and Vale platform binaries: src/rules/platform-binary.ts resolves them by package name. They stay in optionalDependencies, unchanged.

No versions change; the lockfile diff only reclassifies dependency types.

Verification

  • pnpm typecheck, pnpm lint (which builds the CLI and runs its own check), and pnpm test (2007/2007) all pass.
  • Packed tarball in an empty project: npm install of the pnpm pack output installs 6 packages: the CLI, tsx (plus esbuild), and one ast-grep and one Vale binary for the platform. None of the 12 libraries are installed. From that install:
    • --version, init --no-interactive and agent create-sg-rule work.
    • check reports an ast-grep finding (a planted eval()) and Vale findings (no-hedging on a README).
  • Not exercised: a runtime rule, which is the tsx path. That needs a signed rule. tsx is unchanged by this PR either way.

The trade-off: the bundled libraries no longer show up in a consumer's npm ls or dependency scanners, which matches what actually runs.

Rebased onto main after #457 conflicted with 057683c, which dropped picocolors from the CLI entirely, so 12 libraries move rather than 13. Checks re-run after the rebase: pnpm typecheck, pnpm lint, pnpm test (2098/2098).

@theCodeDrift

Copy link
Copy Markdown
Member Author

@claude /review please verify that inlining these dependencies are unlikely to cause issues across installed architectures or in different deployments... specifically in situations where vite's node externals would be a better solution

@claude

claude Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Claude finished @theCodeDrift's task in 3m 24s —— View job


Review: architecture/deployment risk of moving 13 libs to devDependencies

Review mode: incremental — read 0 prior review thread(s) before reviewing. .prior-review.json contained only the trigger comment itself (no prior review summaries or threads), so this is the first review pass on this PR — nothing to mark as already-addressed.

Answer: no, this is not expected to cause architecture- or deployment-specific issues. Here's the verification behind that:

1. The bundling itself is unchanged by this diff. packages/cli/vite.config.ts externalizes only /^node:/ and builtinModules (rollupOptions.external: [/^node:/, ...builtinModules]) — this PR touches package.json and pnpm-lock.yaml only, not vite.config.ts. So whatever dist/ already bundled today, it bundles identically after this PR; the change only affects what a consumer's npm install downloads (devDependencies of an installed package are never installed by the installer, which is the entire mechanism this PR relies on).

2. None of the 13 relocated packages (@clack/prompts, @jedmao/ini-parser, chalk, citty, jose, openapi-fetch, picocolors, posthog-node, smol-toml, sprintf-js, uqr, yaml, zod) ship native addons or per-platform os/cpu-restricted builds. They're all pure JS/TS with no compiled binaries, so there's no architecture (x64/arm64/musl) axis for pnpm to resolve differently per host — unlike the two packages that do have that axis.

3. The two genuinely platform-sensitive pieces are untouched and already handled correctly, not through vite externals but through runtime resolution:

  • src/rules/platform-binary.ts resolves @ast-grep/cli-<platform>-<arch>[-gnu|-msvc] / @taskless/vale-<platform> via createRequire().resolve(...)/package.json, then execs the binary by path. These stay optionalDependencies, so npm/pnpm's os/cpu filtering installs only the matching variant. Correct, and unchanged.
  • src/rules/runtime/invoke.ts's resolveTsxCli() does the same createRequire().resolve("tsx/package.json") dance to find and spawn tsx's CLI as an isolated subprocess for running a signed runtime rule's check.ts. This is exactly the case where vite's bundling can't substitute — the code needs a real installed package on disk to spawn as a separate process, not inlined JS — and the diff correctly keeps tsx in dependencies rather than moving it. This is the one place in the codebase where "vite externals vs. inlining" is actually load-bearing, and the PR's split lines up with it exactly.

Both of these are resolved via createRequire(...).resolve(...), which is a dynamic Node CJS interop call vite/rollup doesn't try to statically bundle away — so neither needed (nor has) a rollupOptions.external entry; they were never candidates for inlining in the first place.

4. Nothing in the CLI's own code branches on whether a package is listed in dependencies vs. devDependencies of packages/cli/package.json. I checked the other places the codebase reads dependency fields (src/detect/scan.ts:316-317, src/install/pinned-cli.ts:55) — those inspect a target project's package.json during detection/stale-pin scanning, not the CLI's own manifest, so this reclassification can't change that behavior.

One thing I couldn't fully verify (not a confirmed issue, flagging for awareness): src/telemetry.ts's getTelemetry() wraps all of posthog-node initialization in a try/catch that falls back to a silent no-op client on any error (line ~278-288). If posthog-node (or any of the other 12) had some runtime self-reference that assumes it's unpacked at its own node_modules path rather than inlined — e.g., reading a sibling file via a relative path rather than a static import — a resulting breakage would fail silently and wouldn't necessarily surface in the PR's own tarball verification (which checked --version, init, agent create-sg-rule, and check, but didn't assert that a telemetry event was actually received). This sandbox has no installed node_modules to inspect posthog-node's internals directly, and I want to be explicit that this is speculative rather than a known defect — plus it would be a pre-existing bundling behavior, not something this PR introduces, since the bundling itself doesn't change here. Worth a quick manual check (e.g. hit the real PostHog endpoint once from the packed tarball with telemetry enabled) if you want to close that gap, but I wouldn't block on it.

Mechanical check of the diff itself: the package.json/pnpm-lock.yaml split matches the PR description exactly — all 13 named libraries moved to devDependencies, tsx remains the sole dependencies entry, optionalDependencies (ast-grep/Vale platform binaries) untouched, and the previously-existing dev tooling (@ast-grep/cli, typescript, vite, vitest, etc.) is unchanged apart from alphabetical reordering. No versions changed, consistent with the stated "lockfile diff only reclassifies dependency types."

vite inlines every npm package into dist/ (rollupOptions.external is node builtins only), so the 13 libraries listed under dependencies were downloaded on every install and never loaded. They move to devDependencies. tsx stays: invoke.ts resolves and spawns it at runtime. The ast-grep and Vale binaries stay in optionalDependencies, unchanged.
@theCodeDrift
theCodeDrift force-pushed the chore/bundled-deps-to-dev branch from 07bbcc7 to 7c7fa23 Compare October 6, 2026 16:20
@theCodeDrift
theCodeDrift merged commit 92b3715 into main Oct 6, 2026
8 checks passed
@theCodeDrift
theCodeDrift deleted the chore/bundled-deps-to-dev branch October 6, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant