feat(safe-bash): add llm command pack with injected providers

[kamilio]

Summary

Rebuild the llm CLI as a safe-bash command pack. safe-bash ships only the primitives: argument parsing, stdin and attachment handling, text or binary output, the provider contract and model lookup. Providers and the actual querying are injected by the consumer, following the networkCommands({ transport }) pattern in packages/safe-bash/src/commands/network. The pack accepts many providers at once and routes each request by model id. Ship OpenAI and ElevenLabs providers as reference plugins.

Scope

Command pack packages/safe-bash/src/commands/llm

• createLlmCommands(options) and llmCommands(options) plugin, same shape as createNetworkCommands / networkCommands.
• LlmCommandsOptions: providers: readonly LlmProvider[], defaultModel?: string, replace?: boolean.
• Registers one command: llm.
• Model lookup: every provider contributes its models. -m resolves an id or alias across all providers; duplicate ids across providers are a registration error.

CLI surface (mirrors llm, cli only)

• llm [prompt]. Prompt from args, stdin, or both. When both are present stdin is the content and the arg is the instruction, as in cat file | llm 'summarize'.
• -m, --model <id>: pick a model by id or alias. Falls back to defaultModel. Unknown model: exit 1 with Unknown model: <id> on stderr.
• -s, --system <text>: system prompt.
• -o, --option <key> <value>: repeatable, passed through to the provider untouched.
• -a, --attachment <path>: repeatable. Path is read from the sandbox filesystem, mime type sniffed from magic bytes with extension fallback. Images, audio and videos are the primary use case; any type the model declares is accepted.
• --at <path> <mimetype>: repeatable, same as -a with an explicit mime type.
• Attachments the selected model does not accept fail before the request with exit 1 and Model <id> does not accept <mimetype> on stderr. Attachment bytes count against existing input budgets.
• llm models: list provider/model ids, aliases, accepted attachment types and output type from the injected providers.
• Output: text models write UTF-8 chunks as they arrive followed by a newline. Binary models (image/png, video/mp4, audio/mpeg, ...) write raw bytes with no trailing newline, so > file.png and | base64 work like any other command. Mixing text and binary in one response is a provider error.
• Honors the shell abort signal and existing output budgets.

Provider contract exported from the pack

interface LlmProvider {
  readonly name: string;
  readonly models: readonly LlmModel[];
  complete(request: LlmRequest): AsyncIterable<string | Uint8Array>;
}
interface LlmModel {
  readonly id: string;
  readonly aliases?: readonly string[];
  readonly attachmentTypes?: readonly string[]; // mime types or "image/*", "video/*", "audio/*"
  readonly outputType?: string;                 // default "text/plain"; e.g. "image/png", "audio/mpeg"
}
interface LlmRequest {
  model: string;
  prompt: string;
  system?: string;
  attachments: readonly { mimeType: string; bytes: Uint8Array }[];
  options: Readonly<Record<string, string>>;
  signal: AbortSignal;
}

Examples

Consumer wiring, one shell with two providers:

const transport = createFetchTransport({ authorize: createOriginAuthorizer({ allow: ["api.openai.com", "api.elevenlabs.io"] }) });

const openai = createOpenAiProvider({
  transport,
  apiKey: env.OPENAI_API_KEY,
  models: [
    { id: "gpt-4.1", aliases: ["4.1"], endpoint: "chat", attachmentTypes: ["image/*"] },
    { id: "gpt-image-1", endpoint: "images", attachmentTypes: ["image/*"], outputType: "image/png" },
    { id: "sora-2", endpoint: "videos", attachmentTypes: ["image/*"], outputType: "video/mp4" },
  ],
});

const elevenlabs = createElevenLabsProvider({
  transport,
  apiKey: env.ELEVENLABS_API_KEY,
  models: [
    { id: "eleven_multilingual_v2", aliases: ["tts"], endpoint: "tts", outputType: "audio/mpeg" },
    { id: "music_v1", aliases: ["music"], endpoint: "music", outputType: "audio/mpeg" },
  ],
});

const shell = new Shell({ fs, cwd: "/work" })
  .use(agentCommands())
  .use(llmCommands({ providers: [openai, elevenlabs], defaultModel: "gpt-4.1" }));

Scripts running inside that shell:

# text
cat notes.md | llm 'turn these into a changelog'

# describe an image
llm -a photo.jpg 'what is in this picture'

# create an image
llm -m gpt-image-1 'a watercolor fox reading a newspaper' -o size 1024x1024 > fox.png

# edit an image (attachments present -> images/edits endpoint)
llm -m gpt-image-1 -a fox.png 'give the fox a red scarf' > fox-scarf.png

# create a video
llm -m sora-2 'the fox folds the newspaper and walks away' -o seconds 8 > fox.mp4

# animate a still image into a video
llm -m sora-2 -a fox-scarf.png 'slow pan, gentle wind' > fox-pan.mp4

# text to speech via elevenlabs
llm -m tts -o voice_id JBFqnCBsd6RMkjVDRZzb 'The fox has left the building.' > fox.mp3
cat script.txt | llm -m tts > narration.mp3

# create music via elevenlabs
llm -m music 'lo-fi jazz loop, brushed drums, warm upright bass' -o music_length_ms 30000 > loop.mp3

# instrumental only, longer track
llm -m music 'cinematic strings building to a brass climax' -o music_length_ms 120000 -o force_instrumental true > score.mp3

# chain: describe -> narrate
llm -a fox.png 'one sentence caption' | llm -m tts > caption.mp3

# chain: describe the mood of a video -> soundtrack for it
llm -a fox.mp4 'describe the mood in one line as a music prompt' | llm -m music -o music_length_ms 8000 > fox-soundtrack.mp3

Reference plugins

createOpenAiProvider({ transport, apiKey, baseUrl?, models })

• Uses the same injected NetworkTransport as curl, so no direct network access from the pack.
• Each configured model carries endpoint: "chat" | "images" | "videos". The provider has no hardcoded model knowledge.
• chat: streams POST /chat/completions, yields delta.content. Image attachments become image_url parts with base64 data URIs.
• images: POST /images/generations, or POST /images/edits when attachments are present. Yields the decoded b64_json bytes. -o keys map straight to request fields (size, quality, background).
• videos: POST /videos, polls the job until completed honoring the abort signal, then downloads /videos/{id}/content and yields the bytes. An image attachment is sent as input_reference. -o seconds, -o size pass through.
• Works against any OpenAI-compatible endpoint, including Poe, by setting baseUrl.

createElevenLabsProvider({ transport, apiKey, baseUrl?, models })

• Each configured model carries endpoint: "tts" | "music". The provider has no hardcoded model knowledge.
• tts: POST /v1/text-to-speech/{voice_id} with model_id set to the selected model, output_format derived from the model's outputType. Yields the streamed audio bytes. voice_id comes from -o voice_id <id> or a per-model defaultVoiceId in the consumer config; missing voice exits 1 with a clear message. Remaining -o keys pass through as voice_settings fields (stability, similarity_boost, speed).
• music: POST /v1/music with the prompt as prompt, model_id set to the selected model, output_format derived from outputType. -o music_length_ms and -o force_instrumental pass through as request fields. Yields the audio bytes. No attachments accepted.
• Proves the multi-provider and binary-output paths with a provider that is not OpenAI-compatible.

Out of scope

• No key storage or llm keys. Auth belongs to the injected provider.
• No llm chat or any interactive session. Single request, single response.
• No SQLite logging, llm logs, templates, embeddings, llm install, or conversation continuation (-c).
• No URL attachments in v1. Files come from the sandbox filesystem only.
• Not a poe-code CLI command. This is a safe-bash tool only.

Requirements

• TDD. Tests use fake in-memory providers, never the network. OpenAI and ElevenLabs provider tests use a fake transport and fixed fixture bytes.
• README for the pack listing every option, the provider contract, attachment and binary-output behavior, linked from the safe-bash README command list.
• No provider-specific branching in the command. Everything is derived from the injected LlmProvider list and each model's declared attachmentTypes and outputType.

[kamilio]

Implemented and verified on remote main in a46e034059bbe80f84070becaa6f7ff406ee6e05, with qualification recorded in a92c830abd5b7d2dba76742b1f456d9a493117db.

The optional LLM pack supports injected model/provider routing, sandbox attachments and input budgets, text/binary streaming, cancellation, and OpenAI/ElevenLabs reference providers through the existing HttpTransport contract. Root/subpath Node, Bun, browser, and consumer types are covered.

Validation passed: full maintained npm test including posttest, normal build, type gates, workflow lint, all 10,775 configured ESLint files, 35 packed-consumer stages, and inspected terminal screenshots. Provider checks use fake transports; no live API acceptance is claimed. The normal build was run after unit testing to restore browser bundles before the successful packed checks.

Release verification completed: poe-code 15.0.19 is published as npm latest; SafeFS, SafeJS, and SafeBash are published as latest 0.1.545. Toolcraft, toolcraft-schema, and toolcraft-openapi are published as latest 0.0.230. Root release, scoped safe release, toolcraft release, and Pages workflows all succeeded. Registry versions and dist-tags were checked separately from workflow success.