From e41c2f5f7248bfdf29e598fba2837fc0efb4af04 Mon Sep 17 00:00:00 2001 From: Anthony Ettinger Date: Tue, 8 Sep 2026 13:38:11 +0000 Subject: [PATCH] Mobile companions: adb, expo and eas, installed on request `adb` forces a fourth kind of companion install. It is not on npm -- what is published under those names is either a Node reimplementation of the wire protocol or somebody's mirror of Google's zip, and a debugging bridge that gets a root shell on every attached device is the last thing to take from a mirror. Google publishes no installer either, only platform-tools-latest-.zip. So `archive` joins npm, script and go: scripts/install-archive.ts fetches it, unpacks it under vendor/platform-tools and links `adb` and `fastboot`, which is the arrangement install.sh already has with the Stripe CLI moved somewhere `cli-tools update` can reach. An adb already on PATH is left alone rather than shadowed, a real file in the prefix is never overwritten, and the unpacked tree is swapped in at the end so a failed download leaves a working install alone. There is no checksum to check -- Google publishes none beside that alias -- so the sha256 of what landed is printed instead of the silence. `expo` and `eas` are why companions now have groups. Between them they are about half a gigabyte, and a web server has no use for either, so installing them because somebody ran the installer is exactly the surprise `diskpush --cli-only` exists to avoid. Grouped companions are installed when asked for -- `cli-tools companions --install mobile`, or a name, or `all` -- and `update` follows the same rule: it updates the mobile commands a box has and never adopts the ones it does not. `link` and install.sh name the group once rather than nagging. Verified end to end: the archive install lands adb 1.0.41 and fastboot from platform-tools 37.0.1, is idempotent on a second run, and steps aside for a foreign adb on PATH; expo 57.0.22 and eas-cli 23.2.0 both give working global binaries. The npm-companion test insisting on a scoped package is relaxed -- that held only while every one was ours or Bitwarden's. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01RsvBXsJ6X3KU6bfC5H4m9J --- README.md | 61 ++++++++- bin/cli-tools.ts | 84 +++++++++++-- install.sh | 11 +- scripts/install-archive.ts | 246 +++++++++++++++++++++++++++++++++++++ src/companions.ts | 218 +++++++++++++++++++++++++++++++- test/companions.test.ts | 143 ++++++++++++++++++++- 6 files changed, 736 insertions(+), 27 deletions(-) create mode 100644 scripts/install-archive.ts diff --git a/README.md b/README.md index dea4972..0d41ecd 100644 --- a/README.md +++ b/README.md @@ -57,6 +57,8 @@ One thing here is not a `PATH` command and does not need Node: [torlnk](https://www.npmjs.com/package/torlnk) running - **ImageMagick** (`magick`) — `img` only, and only for what sharp cannot do (PDF, PSD, animated GIF); sharp ships with this repo as an optional dependency +- **`unzip` or `bsdtar`** — the `adb` companion only, and only while installing + it: the archive Google publishes is a zip, and Node cannot read one - **Network on first use** — `favicon` only: the generation is [`@profullstack/favicon-generator`](https://github.com/profullstack/favicon-generator), fetched by `npx` rather than installed here (about seven seconds the first @@ -99,16 +101,27 @@ implement, because they are distributed in their own right: | `devdb` | [terrablue/devdb](https://github.com/terrablue/devdb) — spin up a throwaway local database for development or testing | | `kali` | [`@profullstack/kali`](https://github.com/profullstack/kali) — install a Kali-style web pentesting toolbelt on Debian/Ubuntu | +And one group that is installed only when you ask for it, with +`cli-tools companions --install mobile`: + +| | | +| --- | --- | +| `adb` | [platform-tools](https://developer.android.com/tools/adb) — talk to Android devices and emulators: install, log, shell, port-forward. Brings `fastboot` | +| `expo` | [Expo CLI](https://docs.expo.dev/more/expo-cli/) — create and run Expo apps, with no project to hand | +| `eas` | [EAS CLI](https://docs.expo.dev/eas/) — build, sign and submit iOS and Android apps in the cloud, and ship OTA updates | + They are not `bin/*.ts` like everything else here for a reason: they run on Windows, which this install cannot (it is symlinks into a git checkout executed -through an `npx tsx` shebang), and they are useful with no checkout at all — +through an `npx tsx` shebang — `adb` is the one exception, and says so below), +and they are useful with no checkout at all — under any agentic CLI, from a Dockerfile, on a box that has never heard of this repository. Vendoring them to make one list tidier would cost them all of that. So `cli-tools` is their front door, not their implementation. -Four come from npm. `bw` is the only companion that is nobody's but its -vendor's; it earns the place on the same terms as the rest, and covers the -secrets `cli-tools` deliberately does not — the `vault` helpers read a logicsrc +Six come from npm. `bw` was the first companion that is nobody's but its +vendor's — `adb`, `expo` and `eas` are the others — and it earns the place on +the same terms as the rest: published, self-installing, useful on a box with no +checkout. It covers the secrets `cli-tools` deliberately does not — the `vault` helpers read a logicsrc team vault of shared API keys, which is a different thing from one person's passwords. Note that its binary is `bw` while its package is `@bitwarden/cli`: the two need not match, which is why the binary name is stated rather than @@ -144,7 +157,45 @@ tool, so there is no need to run Kali itself. The tools are dual-use, which is the point of keeping it a front door: it equips a box you are authorized to test, and nothing about a target lives in this repository. -All three kinds are idempotent, which is what lets install, re-install and update be +`adb` is the odd one, and the reason there is a fourth kind of install. It is +not on npm — what is published under those names is either a Node +reimplementation of the wire protocol or somebody's mirror of Google's zip, and +a debugging bridge that gets a root shell on every attached device is the last +thing to take from a mirror. Google publishes no installer either, only +`platform-tools-latest-.zip`. So `scripts/install-archive.ts` fetches it, +unpacks it under `vendor/platform-tools` and links `adb` and `fastboot` — the +same arrangement `install.sh` already has with the Stripe CLI, moved somewhere +`cli-tools update` can reach. An `adb` already on PATH, apt's or an Android +Studio SDK's, is left alone rather than shadowed; `--force` is how you say +otherwise, and it names what it is now competing with. There is no checksum to +check, which is worth saying rather than quietly skipping: Google publishes none +beside that alias, so the guarantee is TLS to dl.google.com and the sha256 of +what landed is printed. It needs `unzip` (or `bsdtar`) on the box, and there is +no Windows entry on purpose — this install is symlinks into a vendor directory, +which is not how a command reaches PATH there. Use Android Studio's SDK Manager. + +`expo` and `eas` are why groups exist at all. The Expo CLI has no package of its +own: it ships inside `expo`, and Expo's own advice is `npx expo` from inside a +project so the CLI matches that project's SDK — this global copy is for the +other half, creating an app before a project exists. `eas` is the half that is +meant to be global, since builds, signing, store submission and OTA updates all +happen on their infrastructure; like `myna` it is a front door and `eas login` +does the credential handling. Between them they are about half a gigabyte, which +a web server has no use for, so they are installed on request rather than by +`link` — the same judgement `diskpush --cli-only` gets. `cli-tools update` +follows the same rule: it updates the mobile commands you have and never adopts +the ones you do not. + +Deliberately not here, for native mobile work: `scrcpy` (mirror and control an +Android screen) is a distro package — `apt install scrcpy` or `snap install +scrcpy`; the Android `cmdline-tools` that carry `sdkmanager`, `avdmanager` and +the emulator want a JDK and a licence-acceptance flow, which is Android +Studio's job; and [Maestro](https://maestro.mobile.dev)'s installer is a bash +script that appends to your shell rc files and needs Java, which is not a thing +this should do to a box on your behalf — `curl -Ls https://get.maestro.mobile.dev +| bash` if you want it. + +All four kinds are idempotent, which is what lets install, re-install and update be the same command. `CLI_TOOLS_NO_COMPANIONS=1` skips them, and a failure warns rather than failing the install. diff --git a/bin/cli-tools.ts b/bin/cli-tools.ts index 9f5789d..b107edc 100755 --- a/bin/cli-tools.ts +++ b/bin/cli-tools.ts @@ -54,10 +54,15 @@ import { } from '../src/registry.ts'; import { COMPANIONS, + core as coreCompanions, ensure as ensureCompanions, + forUpdate as companionsToUpdate, + groups as companionGroups, installCommand, + select as selectCompanions, source as companionSource, statuses as companionStatuses, + type Companion, } from '../src/companions.ts'; export const USAGE = `Usage: @@ -66,7 +71,7 @@ export const USAGE = `Usage: cli-tools autoupdate [--install [--hours N] | --remove] cli-tools link [--force] cli-tools unlink - cli-tools companions [--install [--force]] + cli-tools companions […] [--install [--force]] cli-tools aliases [--install] cli-tools config [pull | set [value] | unset ] cli-tools [args…] @@ -79,8 +84,11 @@ Commands: autoupdate A systemd user timer that runs "update --auto" for you link Symlink the commands into ~/.local/bin, and install the companions unlink Remove the symlinks we own (companions are left installed) - companions The commands that come from npm rather than this checkout + companions The commands that come from elsewhere rather than this checkout "--install" installs the missing ones, "--force" updates them all + Name a companion or a group to reach past the default set: + "companions --install mobile" adds adb, expo and eas; "all" adds + every one of them aliases Print the moshcode pit aliases, or write them with --install config API keys: what is set, where it came from, and how to change it "config pull" imports them from the logicsrc team vault @@ -137,7 +145,13 @@ function runLinks(root: string, args: readonly string[]): number { * for `cli-tools link` to report that the linking did not happen. The line is * printed to stderr so it stays out of anything reading stdout. */ -function installCompanions({ latest = false, quiet = false } = {}): ReturnType { +function installCompanions( + { latest = false, quiet = false, list = coreCompanions() }: { + latest?: boolean; + quiet?: boolean; + list?: readonly Companion[]; + } = {}, +): ReturnType { const results = ensureCompanions({ onPath: (name) => whichOnPath(name), run: ({ command, args }) => { @@ -146,6 +160,7 @@ function installCompanions({ latest = false, quiet = false } = {}): ReturnType entry.group === group); + if (members.every((entry) => whichOnPath(entry.name))) continue; + lines.push( + `${group}: ${members.map((entry) => entry.name).join(', ')} — ` + + `\`cli-tools companions --install ${group}\``, + ); + } + return lines.join('\n'); +} + /** Pull and relink. Dependencies come first so a new one is present before use. */ function update(root: string): number { const git = spawnSync('git', ['pull', '--ff-only'], { cwd: root, stdio: 'inherit' }); @@ -190,7 +226,11 @@ function update(root: string): number { // After the links, so a failed npm never hides a failed relink. `--latest` // here is what makes `update` mean update for the companions too: a bare // `npm install -g ` leaves an already-satisfied version in place. - installCompanions({ latest: true }); + // + // Only what this box actually has, though: update means update, not adopt. + // Someone who never asked for the mobile set should not find `eas` on their + // machine because they ran `cli-tools update`. + installCompanions({ latest: true, list: companionsToUpdate((name) => whichOnPath(name)) }); return linked; } @@ -625,15 +665,20 @@ export async function run(argv: readonly string[]): Promise { // A separate block, because they are a different kind of thing: these // come from npm and run with no checkout, so the *-ours / !-shadowed // marks above would be answering a question that does not apply. - process.stdout.write('\nFrom npm:\n'); + process.stdout.write('\nCompanions:\n'); for (const entry of companions) { const mark = entry.state === 'installed' ? '*' : ' '; - process.stdout.write(`${mark} ${entry.name.padEnd(16)} ${entry.summary}\n`); + const tag = entry.group ? ` [${entry.group}]` : ''; + process.stdout.write(`${mark} ${entry.name.padEnd(16)} ${entry.summary}${tag}\n`); } const other = all.filter((entry) => entry.status === 'other'); const missing = all.filter((entry) => entry.status === 'missing'); - const absent = companions.filter((entry) => entry.state === 'missing'); + // Only the default set counts as missing. A grouped companion nobody + // asked for is not a gap in the install, and `--install` would not + // install it anyway -- saying "3 not installed" and then not installing + // them is the kind of lie a status line only gets to tell once. + const absent = companions.filter((entry) => entry.state === 'missing' && !entry.group); process.stdout.write('\n'); if (absent.length > 0) { @@ -681,6 +726,8 @@ export async function run(argv: readonly string[]): Promise { case 'link': { const linked = runLinks(root, options.flags.has('--force') ? ['--force'] : []); installCompanions(); + const hint = groupHint(); + if (hint) process.stdout.write(`\nInstalled only when asked for:\n ${hint}\n`); return linked; } @@ -692,24 +739,39 @@ export async function run(argv: readonly string[]): Promise { return runLinks(root, ['--remove']); case 'companions': { + // Positional, so `cli-tools companions --install mobile` reads as the + // sentence it is. Nothing named is the default set, which is what the + // installer and `link` ask for. + const { list: selected, unknown } = selectCompanions(options.positional); + if (unknown.length > 0) { + process.stderr.write( + `companions: no companion or group called ${unknown.join(', ')}.\n` + + ` groups: ${companionGroups().join(', ')}, or "all"\n`, + ); + return 1; + } + if (options.flags.has('--json')) { const rows = options.flags.has('--install') - ? installCompanions({ latest: options.flags.has('--force'), quiet: true }) + ? installCompanions({ latest: options.flags.has('--force'), quiet: true, list: selected }) : companionStatuses((name) => whichOnPath(name)); process.stdout.write(`${JSON.stringify({ companions: rows }, null, 2)}\n`); return 0; } if (options.flags.has('--install')) { - installCompanions({ latest: options.flags.has('--force') }); + installCompanions({ latest: options.flags.has('--force'), list: selected }); return 0; } - process.stdout.write('Published separately, installed from npm:\n\n'); + process.stdout.write('Published separately, installed alongside the commands here:\n\n'); for (const entry of companionStatuses((name) => whichOnPath(name))) { const mark = entry.state === 'installed' ? '*' : ' '; - process.stdout.write(`${mark} ${entry.name.padEnd(16)} ${entry.summary}\n`); + const tag = entry.group ? ` [${entry.group}]` : ''; + process.stdout.write(`${mark} ${entry.name.padEnd(16)} ${entry.summary}${tag}\n`); process.stdout.write(`${' '.repeat(19)}${companionSource(entry)}\n`); } process.stdout.write('\nInstall or update them with `cli-tools companions --install`.\n'); + const hint = groupHint(); + if (hint) process.stdout.write(`Installed only when asked for:\n ${hint}\n`); return 0; } diff --git a/install.sh b/install.sh index aece8ab..7afbf29 100755 --- a/install.sh +++ b/install.sh @@ -105,9 +105,9 @@ CLI_TOOLS_PREFIX="$PREFIX" node "$HOME_DIR/scripts/install-links.mjs" $LINK_ARGS # ── Companions ─────────────────────────────────────────────────────────────── # -# Commands this set ships but does not implement: published npm packages that -# bring their own binary. The list lives in src/companions.ts and is read from -# there rather than repeated here, so adding one is a single-file change. +# Commands this set ships but does not implement: published packages that bring +# their own binary. The list lives in src/companions.ts and is read from there +# rather than repeated here, so adding one is a single-file change. # # Run through the checkout's own dispatcher rather than $PREFIX/cli-tools: the # link above is refused when another checkout already owns that name, and this @@ -118,7 +118,10 @@ CLI_TOOLS_PREFIX="$PREFIX" node "$HOME_DIR/scripts/install-links.mjs" $LINK_ARGS # succeeded — and CLI_TOOLS_NO_COMPANIONS=1 skips it entirely for anyone who # would rather manage those packages themselves. if [ "${CLI_TOOLS_NO_COMPANIONS:-0}" != "1" ]; then - say "Installing npm companions (timer, billing)" + say "Installing companions (timer, billing, bw, …)" + # The default set only. The grouped ones -- `mobile`, which is adb plus half + # a gigabyte of Expo -- are installed when somebody asks for them and not + # because they ran an installer. `cli-tools companions` names them. "$HOME_DIR/bin/cli-tools.ts" companions --install || printf 'cli-tools: companions skipped. Install them later with: cli-tools companions --install\n' >&2 fi diff --git a/scripts/install-archive.ts b/scripts/install-archive.ts new file mode 100644 index 0000000..b15a1a5 --- /dev/null +++ b/scripts/install-archive.ts @@ -0,0 +1,246 @@ +#!/usr/bin/env node +/** + * Install an `archive` companion: download the vendor's release, unpack it + * under `vendor/`, link the binaries it contains into the prefix. + * + * node scripts/install-archive.ts adb # install if it is missing + * node scripts/install-archive.ts adb --force # reinstall, and take the name + * + * Why this exists at all. The other companion kinds hand the work to somebody + * else's installer -- npm, `go install`, a project's own `curl | sh`. An + * `archive` companion has no such thing: Google publishes platform-tools as a + * zip on dl.google.com and nothing more. So the fetching, unpacking and + * linking is ours to do, and it is one script rather than a shell line built + * up inside `companions.ts` so that it can be read, and so the careful parts + * below are somewhere a person will find them. + * + * The careful parts, all of them about not being destructive on a machine that + * was working before this ran: + * + * An `adb` already on PATH from somewhere else -- apt's, or the one inside an + * Android Studio SDK -- is left alone. Ours would shadow it or not depending + * on the order of PATH, which is the worst of both. `--force` is the way to + * say you meant it, and it says out loud what it is now shadowing. + * + * A real file in the prefix is never overwritten, `--force` or not. That is + * the same trade `install-links.mjs` refuses to make: clobbering someone's + * actual binary to install a convenience. + * + * The unpacked tree lands beside the old one and is swapped in at the end, + * so a failed download leaves the working install where it was. + * + * There is no checksum to verify, and that is worth stating plainly rather than + * quietly skipping: Google publishes `platform-tools-latest-.zip` with no + * checksum file beside it, and the sums in the SDK repository manifest are for + * the versioned artefacts, not this alias. So the guarantee here is TLS to + * dl.google.com, and the sha256 of what actually landed is printed -- which is + * what lets two boxes be compared, or one be recorded, after the fact. + */ + +import { spawnSync } from 'node:child_process'; +import { createHash } from 'node:crypto'; +import { + chmodSync, + existsSync, + lstatSync, + mkdirSync, + mkdtempSync, + readlinkSync, + renameSync, + rmSync, + symlinkSync, + unlinkSync, + writeFileSync, +} from 'node:fs'; +import { homedir } from 'node:os'; +import { dirname, join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +import { archiveUrl, findCompanion, type Companion } from '../src/companions.ts'; + +const here = dirname(fileURLToPath(import.meta.url)); +const repoRoot = resolve(here, '..'); +const prefix = process.env.CLI_TOOLS_PREFIX ?? join(homedir(), '.local', 'bin'); + +function die(message: string): never { + process.stderr.write(`install-archive: ${message}\n`); + process.exit(1); +} + +/** Where the command of this name lives, if anywhere. */ +function onPath(name: string): string | null { + // `command -v` is a shell builtin, so this needs a shell -- but as an + // argument to `sh -c`, never concatenated into one. Node 22 deprecates + // `shell: true` with args for exactly that reason. + const found = spawnSync('sh', ['-c', 'command -v "$1"', 'sh', name], { encoding: 'utf8' }); + const line = (found.stdout ?? '').trim().split('\n')[0]?.trim(); + return found.status === 0 && line ? line : null; +} + +function isSymlink(path: string): boolean { + try { + return lstatSync(path).isSymbolicLink(); + } catch { + return false; + } +} + +/** A link of ours is one already pointing into this checkout's `vendor/`. */ +function isOurLink(path: string): boolean { + try { + if (!lstatSync(path).isSymbolicLink()) return false; + return resolve(dirname(path), readlinkSync(path)).startsWith(join(repoRoot, 'vendor')); + } catch { + return false; + } +} + +async function download(url: string, to: string): Promise { + const response = await fetch(url, { redirect: 'follow' }); + if (!response.ok) die(`${url} answered ${response.status} ${response.statusText}`); + const body = Buffer.from(await response.arrayBuffer()); + writeFileSync(to, body); + return createHash('sha256').update(body).digest('hex'); +} + +/** + * Unpack, with whatever the box has. + * + * Node cannot read a zip, so this shells out. `unzip` is the one Debian and + * macOS both have; `bsdtar` is the fallback because a minimal container often + * has it and nothing else, and it reads zips as happily as tarballs. + */ +function extract(archive: string, into: string): void { + mkdirSync(into, { recursive: true }); + const zipped = /\.zip$/i.test(archive); + const attempts: Array<{ command: string; args: string[] }> = zipped + ? [ + { command: 'unzip', args: ['-q', archive, '-d', into] }, + { command: 'bsdtar', args: ['-xf', archive, '-C', into] }, + ] + : [{ command: 'tar', args: ['-xzf', archive, '-C', into] }]; + + const tried: string[] = []; + for (const { command, args } of attempts) { + const result = spawnSync(command, args, { encoding: 'utf8' }); + if (!result.error && result.status === 0) return; + tried.push( + result.error + ? `${command} is not installed` + : `${command} exited ${result.status}: ${(result.stderr ?? '').trim().split('\n').at(-1) ?? ''}`, + ); + } + die(`could not unpack ${archive} — ${tried.join('; ')}`); +} + +/** The version string a freshly linked binary reports, for the closing line. */ +function version(binary: string): string { + const result = spawnSync(binary, ['--version'], { encoding: 'utf8' }); + if (result.error || result.status !== 0) return ''; + return ((result.stdout ?? '') + (result.stderr ?? '')).trim().split('\n')[0]?.trim() ?? ''; +} + +function link(from: string, to: string): void { + if (existsSync(to) || isSymlink(to)) { + if (!isSymlink(to)) { + // Someone's real binary. Never taken over, and not a reason to fail the + // rest: the download worked and the file is in `vendor/` either way. + process.stderr.write(` SKIP ${to} — is a real file, left alone\n`); + return; + } + unlinkSync(to); + } + symlinkSync(from, to); + process.stdout.write(` ${to} -> ${from}\n`); +} + +async function main(): Promise { + const args = process.argv.slice(2); + const force = args.includes('--force'); + const name = args.find((arg) => !arg.startsWith('-')); + if (!name) die('usage: install-archive.ts [--force]'); + + const companion: Companion | null = findCompanion(name); + if (!companion) die(`no companion called ${name}`); + if (companion.install.kind !== 'archive') { + die(`${companion.name} is ${companion.install.kind === 'npm' ? 'an npm' : `a ${companion.install.kind}`} companion, not an archive one`); + } + + const url = archiveUrl(companion); + if (!url) { + // Not an error in the abstract, but an error for this box, and saying so + // beats a zero exit that installs nothing. On Windows the answer is + // Android Studio's SDK Manager, which the summary points at. + die(`no ${companion.name} build for ${process.platform} — see ${companion.home}`); + } + + const { dir, bins } = companion.install; + const vendor = join(repoRoot, 'vendor'); + const destination = join(vendor, dir); + + const existing = onPath(companion.name); + if (existing && !force && !isOurLink(existing)) { + process.stdout.write(`${companion.name} already on PATH at ${existing} — left alone.\n`); + return; + } + if (existing && !force && isOurLink(existing) && existsSync(destination)) { + process.stdout.write(`${companion.name} already installed at ${destination}.\n`); + return; + } + if (existing && force && !isOurLink(existing)) { + // --force said to install anyway. Whether ours then wins is a question + // about the order of PATH, which this cannot answer and should not + // pretend to, so it names the other one rather than claiming a takeover. + process.stdout.write( + `${companion.name} is also at ${existing}; ours is used only where ${prefix} comes first on PATH.\n`, + ); + } + + // Staged inside `vendor/` rather than in $TMPDIR, because the last step is a + // rename and a rename does not cross filesystems: /tmp is a tmpfs on most of + // these boxes while the checkout is on disk, which would make the swap fail + // exactly where it matters. `vendor/` is gitignored, so the staging + // directory is invisible to git even if a crash leaves one behind. + mkdirSync(vendor, { recursive: true }); + const staging = mkdtempSync(join(vendor, '.staging-')); + try { + const archive = join(staging, url.split('/').at(-1) || 'archive.zip'); + process.stdout.write(`Downloading ${url}\n`); + const sha256 = await download(url, archive); + extract(archive, join(staging, 'unpacked')); + + const unpacked = join(staging, 'unpacked', dir); + if (!existsSync(unpacked)) die(`${archive} did not contain a ${dir}/ directory`); + + // Beside the old one first, so a half-written tree never replaces a + // working install: the swap below is the only moment either is missing. + const incoming = `${destination}.incoming`; + rmSync(incoming, { recursive: true, force: true }); + renameSync(unpacked, incoming); + rmSync(destination, { recursive: true, force: true }); + renameSync(incoming, destination); + + mkdirSync(prefix, { recursive: true }); + for (const bin of bins) { + const binary = join(destination, bin); + if (!existsSync(binary)) { + process.stderr.write(` SKIP ${bin} — not in the archive\n`); + continue; + } + chmodSync(binary, 0o755); + link(binary, join(prefix, bin)); + } + + const reported = version(join(destination, bins[0] ?? companion.name)); + process.stdout.write(`${companion.name}: ${reported || 'installed'}\n`); + process.stdout.write(` sha256 ${sha256}\n`); + + if (!process.env.PATH?.split(':').includes(prefix)) { + process.stderr.write(` WARN: ${prefix} is not on PATH. Add it:\n export PATH="${prefix}:$PATH"\n`); + } + } finally { + rmSync(staging, { recursive: true, force: true }); + } +} + +await main(); diff --git a/src/companions.ts b/src/companions.ts index c4241c1..ae01525 100644 --- a/src/companions.ts +++ b/src/companions.ts @@ -18,6 +18,8 @@ * what lets install, re-install and update all be the same command. */ +import { fileURLToPath } from 'node:url'; + /** * How a companion gets onto the machine. * @@ -29,13 +31,32 @@ * registry: `go install` fetches, builds and places it in one step, which is * how that ecosystem distributes a command. * - * All three are idempotent, which is what lets install, re-install and update + * `archive` is the last resort, for a vendor who publishes a binary and no + * installer at all: download the release, unpack it under `vendor/`, link what + * it contains. Google's platform-tools is the case that forced it -- `adb` is + * not on npm, has no install script, and the packages that claim to be it are + * either a Node reimplementation of the protocol or somebody's mirror of the + * zip. Vendoring it here is the same arrangement `install.sh` already has with + * the Stripe CLI, moved somewhere `cli-tools update` can reach. + * + * All four are idempotent, which is what lets install, re-install and update * stay the same command. */ export type InstallMethod = | { kind: 'npm'; package: string } | { kind: 'script'; url: string; args?: readonly string[] } - | { kind: 'go'; module: string }; + | { kind: 'go'; module: string } + | { + kind: 'archive'; + /** What to print as the source: a url with the platform left as ``. */ + source: string; + /** The download, per `process.platform`. Absent means no build for that box. */ + urls: Readonly>>; + /** The directory the archive unpacks into, and the name it keeps under `vendor/`. */ + dir: string; + /** The binaries inside it to link onto PATH. The first one is `name`. */ + bins: readonly string[]; + }; export interface Companion { /** The binary the package puts on PATH. */ @@ -44,6 +65,18 @@ export interface Companion { summary: string; /** Where to read about it, for the message printed when installing fails. */ home: string; + /** + * A set this companion belongs to, and is installed only when asked for. + * + * No group means the default set: small, useful on any box, installed by + * `link` and by the installer. A group is for the ones that are neither -- + * `mobile` puts half a gigabyte of Expo on a machine, which a web server has + * no use for. It is the same judgement `diskpush` gets with `--cli-only`: + * a command-line toolbelt does not quietly place things this size. + * + * `cli-tools companions --install mobile` is how you say yes to one. + */ + group?: string; } export const COMPANIONS: readonly Companion[] = [ @@ -123,13 +156,176 @@ export const COMPANIONS: readonly Companion[] = [ summary: 'Install a Kali-style web pentesting toolbelt on Debian/Ubuntu', home: 'https://github.com/profullstack/kali', }, + { + name: 'adb', + // The Android Debug Bridge, and `fastboot` out of the same archive: one + // download, two commands. Neither is on npm. What is published under those + // names is either a Node reimplementation of the wire protocol (adbkit) or + // somebody's mirror of this very zip, and a debugging bridge with root on + // every attached device is the last thing to take from a mirror. Google + // publishes no installer either, only the archive -- which is what the + // `archive` kind exists for. + // + // Vendored under `vendor/platform-tools` for the reason the Stripe CLI is: + // the name should exist once. An `adb` already on PATH -- apt's, or the one + // inside an Android Studio SDK -- is left alone rather than shadowed, and + // `--force` is what says otherwise. + // + // platform-tools deliberately, not the whole SDK. `adb` and `fastboot` are + // what a command line needs against a device; the emulator, `sdkmanager` + // and the build tools want a JDK and a licence-acceptance flow, which is + // Android Studio's job rather than a toolbelt's. + install: { + kind: 'archive', + source: 'https://dl.google.com/android/repository/platform-tools-latest-.zip', + // Windows is missing on purpose rather than for want of a build: Google + // publishes that zip too, but this install is symlinks into a vendor + // directory, which is not how a command gets onto PATH there. On Windows + // it is Android Studio's SDK Manager. + urls: { + linux: 'https://dl.google.com/android/repository/platform-tools-latest-linux.zip', + darwin: 'https://dl.google.com/android/repository/platform-tools-latest-darwin.zip', + }, + dir: 'platform-tools', + bins: ['adb', 'fastboot'], + }, + summary: 'Talk to Android devices and emulators — install, log, shell, port-forward (brings fastboot)', + home: 'https://developer.android.com/tools/adb', + group: 'mobile', + }, + { + name: 'expo', + // The Expo CLI has no package of its own: it ships inside `expo`, and + // Expo's own advice is `npx expo` from inside a project so the CLI always + // matches that project's SDK. This global copy is for the other half — + // creating an app before a project exists, and `expo` meaning something on + // a box you have just sat down at. The two do not fight: `npx expo` still + // prefers the project's own copy when there is one. + // + // Unscoped, unlike everything above it. That was an accident of every + // companion having been ours or Bitwarden's, never a rule. + // + // It puts `fingerprint` and `expo-modules-autolinking` on PATH beside + // `expo`. Only `expo` is checked here, because the package is one thing. + install: { kind: 'npm', package: 'expo' }, + summary: 'Create and run Expo apps — the CLI that npx would fetch, with no project to hand', + home: 'https://docs.expo.dev/more/expo-cli/', + group: 'mobile', + }, + { + name: 'eas', + // The half of Expo that is meant to be global: builds, signing, store + // submission and OTA updates all happen on their infrastructure rather + // than in a project, and Expo documents `npm install -g eas-cli` for + // exactly that reason. It holds credentials for the App Store and Play + // Console once authenticated, so like `myna` this is a front door and + // nothing about the account lives here: `eas login` does that. + install: { kind: 'npm', package: 'eas-cli' }, + summary: 'Build, sign and submit iOS and Android apps in the cloud, and ship OTA updates', + home: 'https://docs.expo.dev/eas/', + group: 'mobile', + }, ]; +/** + * The script that installs an `archive` companion. + * + * Resolved from this file rather than passed in by the caller. Every caller + * would have to know the checkout root otherwise, and they would all compute + * it from somewhere near here anyway -- while this module already knows + * exactly which checkout it is part of, which is the one whose `vendor/` the + * download belongs in. + */ +export const ARCHIVE_INSTALLER = fileURLToPath( + new URL('../scripts/install-archive.ts', import.meta.url), +); + +/** + * The download for this box, or null when the vendor publishes nothing for it. + * + * Null is an answer, not a failure to find one: it is what makes an install on + * an unsupported platform say so instead of downloading a Linux binary onto a + * Mac. + */ +export function archiveUrl( + companion: Companion, + platform: NodeJS.Platform = process.platform, +): string | null { + if (companion.install.kind !== 'archive') return null; + return companion.install.urls[platform] ?? null; +} + export function findCompanion(name: string): Companion | null { const key = String(name ?? '').trim().toLowerCase(); return COMPANIONS.find((entry) => entry.name === key) ?? null; } +/** Every group named by the list, in the order they first appear. */ +export function groups(list: readonly Companion[] = COMPANIONS): string[] { + const seen: string[] = []; + for (const companion of list) { + if (companion.group && !seen.includes(companion.group)) seen.push(companion.group); + } + return seen; +} + +/** The companions installed by default: the ones in no group. */ +export function core(list: readonly Companion[] = COMPANIONS): Companion[] { + return list.filter((companion) => !companion.group); +} + +/** + * Which companions a request names. + * + * Nothing named is the default set, because that is what `link` and the + * installer ask for and neither should adopt half a gigabyte of Expo on the + * strength of running an installer. A group name takes that group, a companion + * name takes that one, and `all` takes everything -- and anything else comes + * back in `unknown` rather than being quietly ignored, which is how a typo + * ends up looking like a group that installed nothing. + */ +export function select( + selectors: readonly string[], + list: readonly Companion[] = COMPANIONS, +): { list: Companion[]; unknown: string[] } { + if (selectors.length === 0) return { list: core(list), unknown: [] }; + + const chosen: Companion[] = []; + const unknown: string[] = []; + for (const raw of selectors) { + const key = String(raw ?? '').trim().toLowerCase(); + if (key === 'all') { + for (const companion of list) if (!chosen.includes(companion)) chosen.push(companion); + continue; + } + const matches = list.filter( + (companion) => companion.name === key || companion.group === key, + ); + if (matches.length === 0) { + unknown.push(raw); + continue; + } + for (const companion of matches) if (!chosen.includes(companion)) chosen.push(companion); + } + // Back into list order, so the output does not depend on how it was asked for. + return { list: list.filter((companion) => chosen.includes(companion)), unknown }; +} + +/** + * What an update should touch: the default set, plus whatever grouped + * companions this box actually has. + * + * Update means update, not adopt. Someone who never asked for the mobile set + * should not find `eas` installed because they ran `cli-tools update`, and + * someone who did ask should not have to ask again every time. + */ +export function forUpdate( + onPath: (name: string) => string | null, + list: readonly Companion[] = COMPANIONS, +): Companion[] { + return list.filter((companion) => !companion.group || onPath(companion.name)); +} + export interface InstallCommand { command: string; args: string[]; @@ -160,6 +356,17 @@ export function installCommand(companion: Companion, { latest = false } = {}): I return { command: 'go', args: ['install', spec], display: `go install ${spec}` }; } + if (companion.install.kind === 'archive') { + // The only companion kind whose installer is ours. There is no upstream + // command to run: something has to fetch the archive, unpack it under + // `vendor/` and link what it contains, and that something is + // `scripts/install-archive.ts`. `--force` is this kind's `@latest` -- + // without it the script leaves an existing install, or somebody else's + // binary of that name, exactly where it is. + const args = latest ? [ARCHIVE_INSTALLER, companion.name, '--force'] : [ARCHIVE_INSTALLER, companion.name]; + return { command: 'node', args, display: `node ${args.join(' ')}` }; + } + const { url, args = [] } = companion.install; // Piped into sh the same way the project documents it, so this and a manual // install take the same path and cannot drift apart. @@ -174,6 +381,8 @@ export function source(companion: Companion): string { return companion.install.package; case 'go': return companion.install.module; + case 'archive': + return companion.install.source; default: return companion.install.url; } @@ -228,7 +437,10 @@ export function ensure( onPath, run, latest = false, - list = COMPANIONS, + // The default set, not every companion. A caller that forgets to say which + // ones it wants should get the small, universally useful ones -- never half + // a gigabyte of Expo on a box that asked for a link. + list = core(), }: { onPath: (name: string) => string | null; run: (command: InstallCommand) => { status: number | null; stderr?: string }; diff --git a/test/companions.test.ts b/test/companions.test.ts index 428d669..cdc269f 100644 --- a/test/companions.test.ts +++ b/test/companions.test.ts @@ -1,10 +1,18 @@ +import { existsSync } from 'node:fs'; + import { describe, expect, it } from 'vitest'; import { + ARCHIVE_INSTALLER, + archiveUrl, COMPANIONS, + core, ensure, findCompanion, + forUpdate, + groups, installCommand, + select, source, statuses, type Companion, @@ -24,6 +32,9 @@ describe('the companion list', () => { 'myna', 'devdb', 'kali', + 'adb', + 'expo', + 'eas', ]); }); @@ -34,11 +45,18 @@ describe('the companion list', () => { } }); - it('names an npm companion by a scoped package', () => { + it('names an npm companion by a package that exists, scoped or not', () => { + // This used to insist on a scope, which held only while every npm + // companion was ours or Bitwarden's. `expo` and `eas-cli` are neither, and + // an unscoped name is not a malformed one -- so what is asserted now is + // the thing that was actually meant: a package name, not a url or a path. for (const companion of COMPANIONS) { if (companion.install.kind !== 'npm') continue; - expect(companion.install.package.startsWith('@'), companion.name).toBe(true); - expect(companion.install.package.includes('/'), companion.name).toBe(true); + expect(companion.install.package.length, companion.name).toBeGreaterThan(0); + expect(companion.install.package.startsWith('http'), companion.name).toBe(false); + if (companion.install.package.startsWith('@')) { + expect(companion.install.package.includes('/'), companion.name).toBe(true); + } } }); @@ -149,6 +167,9 @@ describe('statuses', () => { ['myna', 'missing'], ['devdb', 'missing'], ['kali', 'missing'], + ['adb', 'missing'], + ['expo', 'missing'], + ['eas', 'missing'], ]); expect(rows[0]?.path).toBe('/usr/local/bin/timer'); expect(rows[1]?.path).toBeNull(); @@ -219,7 +240,9 @@ describe('ensure', () => { return { status: 1, stderr: 'npm ERR! code EACCES\nnpm ERR! permission denied' }; }, }); - expect(attempted).toHaveLength(COMPANIONS.length); + // The default set, not every companion: a caller that names no list gets + // the ones that install on any box. + expect(attempted).toHaveLength(core().length); expect(results.every((r) => r.action === 'failed')).toBe(true); expect(results[0]?.message).toBe('npm ERR! permission denied'); expect(results[0]?.state).toBe('missing'); @@ -246,3 +269,115 @@ describe('ensure', () => { expect(results[0]?.message).toMatch(/npm is not available/); }); }); + +describe('the archive kind', () => { + const adb = findCompanion('adb') as Companion; + + it('links the name it is checked by, first', () => { + // `statuses` asks whether `adb` is on PATH; the archive is what puts it + // there. If the first binary linked were not the companion's own name, an + // install would report success for a command still missing. + for (const companion of COMPANIONS) { + if (companion.install.kind !== 'archive') continue; + expect(companion.install.bins[0], companion.name).toBe(companion.name); + } + }); + + it('has a build for the platforms this install can actually work on', () => { + expect(archiveUrl(adb, 'linux')).toContain('platform-tools-latest-linux.zip'); + expect(archiveUrl(adb, 'darwin')).toContain('platform-tools-latest-darwin.zip'); + // Google publishes a Windows zip. This is symlinks into a vendor + // directory, which is not how a command reaches PATH there, so the entry + // is absent on purpose and the installer says so rather than half-working. + expect(archiveUrl(adb, 'win32')).toBeNull(); + }); + + it('answers null for a companion that is not an archive', () => { + expect(archiveUrl(findCompanion('timer') as Companion, 'linux')).toBeNull(); + }); + + it('runs our own installer, because there is no upstream one to run', () => { + const command = installCommand(adb); + expect(command.command).toBe('node'); + expect(command.args).toEqual([ARCHIVE_INSTALLER, 'adb']); + }); + + it('passes --force on an update, which is this kind of @latest', () => { + expect(installCommand(adb, { latest: true }).args).toEqual([ARCHIVE_INSTALLER, 'adb', '--force']); + }); + + it('points at a script that is actually there', () => { + // The path is resolved from companions.ts, so a rename of the script would + // otherwise fail at install time on somebody's machine rather than here. + expect(existsSync(ARCHIVE_INSTALLER)).toBe(true); + }); + + it('shows the download, with the platform left as ', () => { + expect(source(adb)).toBe( + 'https://dl.google.com/android/repository/platform-tools-latest-.zip', + ); + }); +}); + +describe('groups', () => { + it('keeps the heavy set out of the default one', () => { + // Expo and eas-cli are about half a gigabyte between them, and a web + // server has no use for either. Installing them because someone ran the + // installer is the surprise `diskpush --cli-only` exists to avoid. + expect(groups()).toEqual(['mobile']); + expect(core().map((c) => c.name)).not.toContain('expo'); + expect(COMPANIONS.filter((c) => c.group === 'mobile').map((c) => c.name)).toEqual([ + 'adb', + 'expo', + 'eas', + ]); + }); + + it('selects the default set when nothing is named', () => { + expect(select([]).list).toEqual(core()); + }); + + it('takes a group, a name, or everything', () => { + expect(select(['mobile']).list.map((c) => c.name)).toEqual(['adb', 'expo', 'eas']); + expect(select(['eas']).list.map((c) => c.name)).toEqual(['eas']); + expect(select(['all']).list).toEqual([...COMPANIONS]); + }); + + it('is case-insensitive, and answers in list order however it was asked', () => { + expect(select(['EAS', 'mobile']).list.map((c) => c.name)).toEqual(['adb', 'expo', 'eas']); + }); + + it('names what it did not recognise rather than installing nothing quietly', () => { + // A typo would otherwise look exactly like a group whose members were all + // already installed. + const { list, unknown } = select(['mobil']); + expect(list).toEqual([]); + expect(unknown).toEqual(['mobil']); + }); + + it('updates what the box has, and never adopts what it does not', () => { + // `cli-tools update` must not be how `eas` arrives on a machine. + expect(forUpdate(present('adb')).map((c) => c.name)).toEqual([ + ...core().map((c) => c.name), + 'adb', + ]); + expect(forUpdate(nothing).map((c) => c.name)).toEqual(core().map((c) => c.name)); + }); + + it('installs a whole group when asked for one', () => { + const calls: string[] = []; + ensure({ + onPath: nothing, + run: ({ display }) => { + calls.push(display); + return { status: 0 }; + }, + list: select(['mobile']).list, + }); + expect(calls).toEqual([ + `node ${ARCHIVE_INSTALLER} adb`, + 'npm install -g expo', + 'npm install -g eas-cli', + ]); + }); +});