Skip to content
1 change: 1 addition & 0 deletions docs/TRAPS.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ Every number here comes from our vocabulary measurements: a synthetic TTS set an
- **Token file readable by others** [T4.45] (M1). Given a fresh install, the token file must be mode 0600 (or a user-only ACL on Windows) and created atomically. Test: stat after `akou doctor`.
- **A 413 the client never reads** [CI flake] (M1). Given a body over 64 KB, akou must answer 413 with its JSON error, and the client must read it whole. Answering while the body is still arriving and then closing the socket leaves unread bytes on it; the kernel resets the connection instead of closing it, and on macOS and Windows a reset drops the answer already in flight, so the client sees `ECONNRESET`. Bun 1.4 closes at once when `maxRequestBodySize` is exceeded (Bun 1.3 did not), which made 5 % of 70 KB requests and two thirds of 512 KB requests fail. The 64 KB limit is the guard's and `readBody`'s; the server reads the rest of the body out (up to 1 MB, and for at most 2 s, since Bun's idle timeout is reset by every chunk and a trickled body could otherwise hold a refusal open) before any answer that did not consume it, whether a 413, a 401 or a 404. Test: a 512 KB body ten times over, a chunked body with no `Content-Length`, a bad token with a large body, each with the answer's JSON read; the limit exactly at 64 KB with the byte under it accepted as the positive control; the drain's two bounds on a fake reader, with a body that ends inside them read to its end.
- **A stalled upload holds its socket** [SV-D3] (M1). Given an upload that stops sending mid-body, or a peer that drops off the network without closing, the server must cut it and delete the spool file. The upload routes turn Bun's idle cut off before reading the body, so each such request held a socket, a file descriptor and an open `<uuid>.upload` until the process ended. The idle cut cannot simply stay on: in Bun 1.4 it counts from the start of the request, not from the last body chunk (measured: a body sending a byte a second is cut at the idle limit), so it would cut every upload longer than 60 s. The body has its own clock instead: no chunk for 60 s and the read fails with a 400 `bad_multipart` that closes the connection (`Connection: close`; without it the kept-alive socket stays open, since the idle cut is off for that request). Test: [upload-idle.e2e.test.ts](../tests/upload-idle.e2e.test.ts), a silent upload is answered 400, its socket closed and its file deleted. Positive control: under a 2 s idle timeout, an upload that sends a chunk a second for 5 s completes.
- **An upload held in memory whole** [UPL-1] (M1). Given `akou transcribe`, `akou dictate FILE` or `akou_transcribe` with a long recording, the client must read the file from disk as it sends it. `fetch` with a `FormData` body builds the whole multipart body in memory before it sends a byte, even when the file part is a `Bun.file` (measured on Bun 1.4.2: a 256 MiB upload grew the CLI by about 535 MB with either a read file or a `Bun.file`). The client sends every form as its own streamed body instead (`uploadBody`), with the Content-Length it computes, so a server still refuses an upload over `server.max_upload_mb` before reading it; a file that changes size while it is sent fails the request. Test: [upload-stream.e2e.test.ts](../tests/upload-stream.e2e.test.ts), the CLI as a child sends a 256 MiB file to a fake server and its peak memory stays within 96 MiB of a 1 KiB upload; the streamed body goes through the server's multipart reader intact with and without a Content-Length. Positive control: a child that sends the same `FormData` with plain `fetch` grows past the limit.
- **An upload fills the server's disk** [DSK-F1] (M1). Given a `jobs` key that keeps sending uploads (in parallel, or with `keep_audio`, which no retention sweep removes), the data volume filled: nothing checked free space before or while an upload was written, and the queue's room check cannot see uploads still arriving. Every multipart upload must keep the 1 GB a model download keeps (SV-M2): under it, 507 `insufficient_storage` before the first byte, and a cut, with its spool file deleted, when the volume falls under it while the body arrives (a look every 16 MB). A per-key cap on kept audio is a separate decision. Test: [multipart.test.ts](../tests/multipart.test.ts) and [upload-floor.e2e.test.ts](../tests/upload-floor.e2e.test.ts), with the same upload accepted when there is room as the positive control.
- **A 507 read as a server that is down** [DSK-F2] (M1). Given a server that refuses an upload with 507 because its disk is under the floor, its clients must read a refusal with its reason. The forwarding primary read any 5xx as a remote that is down: it marked it down, probed it and sent the job again in a loop. The dictation remote counted it toward the down-probe and told the user the remote was down. A 507 is now a rejection like a 422 on the primary: the remote stays up but gets no job for one probe period, so the job goes to another remote that offers it, else runs here or fails. It never counts toward the dictation probe, says `remote disk full`, and `akou transcribe` and `akou_transcribe` print the server's reason, never exit 69. Test: [remote-dispatch.e2e.test.ts](../tests/remote-dispatch.e2e.test.ts), [dictation-remote.e2e.test.ts](../tests/dictation-remote.e2e.test.ts) and [upload-floor.e2e.test.ts](../tests/upload-floor.e2e.test.ts), each with a 503 or an upload with room as the positive control; on the primary, two remotes where the first answers 507 and the job lands on the second, with the first alone as the control, where the job fails with `insufficient_storage`.
- **A WAV header's channel count sizes the read buffer** [SV-P6] (M1). Given a 16 kHz PCM WAV whose header claims up to 65535 channels, the no-ffmpeg reader must read it with a buffer of a few MB. It sized the buffer as 65536 frames, so one crafted header asked for 8.6 GB whatever the file held. The buffer is about 4 MB of whole frames, at least one. Test: [server-audio.test.ts](../tests/server-audio.test.ts), a 65535-channel WAV read with no allocation over 8 MB. Positive control: a stereo WAV longer than one buffer averages every frame.
Expand Down
16 changes: 13 additions & 3 deletions src/main/cli/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ import {
stopList,
stopStrayLaunchers,
} from "./heal.ts";
import { uploadBody } from "./upload-body.ts";

/** Exit codes (DESIGN 6.1). */
export const EXIT = {
Expand Down Expand Up @@ -242,7 +243,10 @@ export interface ClientOptions {

export interface RequestOptions {
body?: unknown;
/** A multipart body instead of JSON: an upload route (`akou transcribe`). */
/**
* A multipart body instead of JSON: an upload route (`akou transcribe`). It is sent as a stream
* (`uploadBody`), so a file part is read from disk as it goes, never held in memory whole.
*/
form?: FormData;
query?: Record<string, string | number | boolean | undefined>;
/** Launch the app if it is not running. Default true. */
Expand Down Expand Up @@ -418,14 +422,20 @@ export class ApiClient {
if (v !== undefined) url.searchParams.set(k, String(v));
}
const hasBody = method !== "GET" && method !== "HEAD";
// A form is streamed, built again for each send: a retry after a launch reads the file anew.
const upload = o.form ? uploadBody(o.form) : null;
return fetch(url, {
method,
headers: {
authorization: `Bearer ${remote ? to.key : this.token()}`,
"x-akou-client": o.client ?? this.o.client,
...(hasBody && !o.form ? { "content-type": "application/json" } : {}),
...(upload
? { "content-type": upload.contentType, "content-length": String(upload.length) }
: hasBody
? { "content-type": "application/json" }
: {}),
},
body: o.form ?? (hasBody ? JSON.stringify(o.body ?? {}) : undefined),
body: upload ? upload.body : hasBody ? JSON.stringify(o.body ?? {}) : undefined,
signal: o.signal
? AbortSignal.any([o.signal, AbortSignal.timeout(o.timeoutMs ?? 60_000)])
: AbortSignal.timeout(o.timeoutMs ?? 60_000),
Expand Down
14 changes: 5 additions & 9 deletions src/main/cli/commands/dictate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@
* prints the new text, leaving the dictation as it was.
*/

import { readFileSync, statSync } from "node:fs";
import { basename } from "node:path";
import { bool, duration, int, str } from "../args.ts";
import { EXIT, Hung, Unreachable } from "../client.ts";
import { api, type Body, type Command, type Ctx, finish, wall } from "../context.ts";
import { uploadFile } from "../upload-body.ts";
import { usage } from "./calls.ts";

const ENGINES = ["auto", "fast", "best", "live", "remote"];
Expand Down Expand Up @@ -110,15 +110,11 @@ export const dictateCommand: Command = {
if (engine !== undefined && !ENGINES.includes(engine)) {
return usage(ctx, `--engine is one of ${ENGINES.join(", ")}`);
}
let bytes: Uint8Array;
try {
if (!statSync(file).isFile()) return usage(ctx, `${file} is not a file`);
bytes = readFileSync(file);
} catch (err) {
return usage(ctx, `cannot read ${file}: ${(err as Error).message}`);
}
// Read only as it is sent (`uploadBody`), never held in memory whole.
const media = uploadFile(file);
if (typeof media === "string") return usage(ctx, media);
const form = new FormData();
form.append("file", new Blob([bytes]), basename(file));
form.append("file", media, basename(file));
if (engine) form.append("engine", engine);
const language = str(p, "language");
if (language) form.append("language", language);
Expand Down
15 changes: 6 additions & 9 deletions src/main/cli/commands/transcribe.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,12 @@
* here: the server's own refusal is printed, so its limits live in one place (docs/ux/CLI.md rule 1).
*/

import { readFileSync, statSync } from "node:fs";
import { readFileSync } from "node:fs";
import { basename } from "node:path";
import { bool, repeated, str } from "../args.ts";
import { EXIT } from "../client.ts";
import { api, type Body, type Command, finish } from "../context.ts";
import { uploadFile } from "../upload-body.ts";
import { usage } from "./calls.ts";

const PRESETS = ["lite", "fast", "best", "fusion", "auto"];
Expand Down Expand Up @@ -74,15 +75,11 @@ export const transcribeCommand: Command = {
return usage(ctx, `cannot read ${keywordsFile}: ${(err as Error).message}`);
}
}
let bytes: Uint8Array;
try {
if (!statSync(file).isFile()) return usage(ctx, `${file} is not a file`);
bytes = readFileSync(file);
} catch (err) {
return usage(ctx, `cannot read ${file}: ${(err as Error).message}`);
}
// Read only as it is sent (`uploadBody`): a long recording is never held in memory whole.
const media = uploadFile(file);
if (typeof media === "string") return usage(ctx, media);
const form = new FormData();
form.append("file", new Blob([bytes]), basename(file));
form.append("file", media, basename(file));
if (preset) form.append("preset", preset);
const language = str(p, "language");
if (language) form.append("language", language);
Expand Down
124 changes: 124 additions & 0 deletions src/main/cli/upload-body.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
/**
* An upload's multipart body, read from disk as it is sent (docs/TRAPS.md, "An upload held in
* memory whole"). `fetch` with a `FormData` body builds the whole body in memory before it sends a
* byte, even when the file part is a `Bun.file`: a 2 GB recording made the CLI hold 2 GB or more.
* Here every file part is its own `Blob`'s stream, so the client holds one chunk at a time.
*
* The body declares its length, as `fetch` does for a `FormData`, so a server refuses an upload
* over its `server.max_upload_mb` before a byte of the file is read. A file that changes size
* while it is sent fails the request instead of sending a body of another length.
*/

import { closeSync, openSync, statSync } from "node:fs";
import { basename } from "node:path";

/** A multipart body to send, with its headers. */
export interface UploadBody {
body: ReadableStream<Uint8Array>;
contentType: string;
length: number;
}

/** A name or filename inside a quoted header parameter, escaped as browsers do (WHATWG). */
function quoted(s: string): string {
return s.replace(/\r/g, "%0D").replace(/\n/g, "%0A").replace(/"/g, "%22");
}

/**
* `form` as a streamed `multipart/form-data` body. A fresh call reads every file again. A file
* part's filename is its `File`'s own name: Bun's `FormData.entries()` drops the filename given to
* `append`, so name the `File` itself (`mediaFile`).
*/
export function uploadBody(form: FormData): UploadBody {
const enc = new TextEncoder();
const boundary = `akou-${crypto.randomUUID()}`;
// Each file with its size now: the length declared is what the body must hold.
const parts: (Uint8Array | { blob: Blob; size: number })[] = [];
for (const [name, value] of form.entries()) {
if (typeof value === "string") {
parts.push(
enc.encode(
`--${boundary}\r\nContent-Disposition: form-data; name="${quoted(name)}"\r\n\r\n${value}\r\n`,
),
);
continue;
}
const file = value as File;
// A `Bun.file` appended as it is keeps its whole path as its name, whatever filename
// `append` was given: only the last segment ever goes out.
parts.push(
enc.encode(
`--${boundary}\r\nContent-Disposition: form-data; name="${quoted(name)}"; filename="${quoted(basename(file.name || "blob"))}"\r\n` +
`Content-Type: ${file.type || "application/octet-stream"}\r\n\r\n`,
),
{ blob: file, size: file.size },
enc.encode("\r\n"),
);
}
parts.push(enc.encode(`--${boundary}--\r\n`));
const length = parts.reduce((n, p) => n + (p instanceof Uint8Array ? p.byteLength : p.size), 0);

let next = 0;
/** The file part being sent: its reader, and the bytes it still owes. */
let reading: {
reader: {
read(): Promise<{ done: boolean; value?: Uint8Array }>;
cancel(r?: unknown): Promise<void>;
};
left: number;
} | null = null;
const body = new ReadableStream<Uint8Array>({
async pull(ctl) {
for (;;) {
if (reading) {
const r = await reading.reader.read();
if (r.value) {
reading.left -= r.value.byteLength;
if (reading.left < 0) throw new Error("a file grew while it was being uploaded");
ctl.enqueue(r.value);
return;
}
if (reading.left !== 0) throw new Error("a file shrank while it was being uploaded");
reading = null;
}
const part = parts[next++];
if (part === undefined) {
ctl.close();
return;
}
if (part instanceof Uint8Array) {
ctl.enqueue(part);
return;
}
reading = { reader: part.blob.stream().getReader(), left: part.size };
}
},
async cancel(reason) {
await reading?.reader.cancel(reason).catch(() => {});
},
});
return { body, contentType: `multipart/form-data; boundary=${boundary}`, length };
}

/**
* The file at `path` to upload, read only as it is sent, or why it cannot be: it is opened once
* here, so a missing or unreadable file is a usage error before anything is sent.
*/
export function uploadFile(path: string): File | string {
try {
if (!statSync(path).isFile()) return `${path} is not a file`;
closeSync(openSync(path, "r"));
} catch (err) {
return `cannot read ${path}: ${(err as Error).message}`;
}
return mediaFile(path);
}

/**
* `path` as a `File` named by its last segment, still read from disk only as it is sent. A
* `Bun.file` in a `FormData` keeps its whole path as its name.
*/
export function mediaFile(path: string): File {
const disk = Bun.file(path);
return new File([disk], basename(path), { type: disk.type });
}
6 changes: 4 additions & 2 deletions src/main/mcp/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ import { APP_VERSION } from "../app-info.ts";
import { callModeWords } from "../capture/call-mode.ts";
import type { ApiClient, ApiResponse, RequestOptions } from "../cli/client.ts";
import { type Body, describeError, wall } from "../cli/context.ts";
import { mediaFile } from "../cli/upload-body.ts";
import { listPresets, usesSpeaker } from "../notes/presets.ts";
import { estimateTokens, type PackState, packState, quoteCallText } from "../query/render.ts";
import { capAnswer, type ToolResult } from "./bound.ts";
Expand Down Expand Up @@ -665,7 +666,8 @@ export const MAX_ANSWER_BYTES = 16 * 1024 * 1024;

/**
* A file to upload, or why not: missing, not a file, over `max` bytes, or not audio or video. The
* file goes out as a `Bun.file`, read as it is sent, never held in memory whole.
* file goes out in the client's streamed body (`uploadBody`), read from disk as it is sent: a
* `FormData` handed to `fetch` as it is would be built whole in memory first, `Bun.file` or not.
*/
export function readMedia(
path: string,
Expand Down Expand Up @@ -699,7 +701,7 @@ export function readMedia(
if (!isMediaHead(head)) {
return { error: `${path} is not an audio or video file; nothing was sent` };
}
return { file: Bun.file(path) };
return { file: mediaFile(path) };
}

/** A loopback host: a plain http request to it never leaves this machine. */
Expand Down
26 changes: 26 additions & 0 deletions tests/fixtures/peak-rss.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
/**
* Preloaded into a spawned bun (`--preload`): when the process exits, it writes its own peak
* memory, in bytes, to the file `AKOU_TEST_PEAK_FILE` names (tests/upload-stream.e2e.test.ts).
*
* On Linux `maxRSS` is not the child's own: `execve` keeps the larger of the spawning process's
* resident size and the child's, so a child spawned by a 1 GB test runner reports 1 GB whatever it
* does. `VmHWM` in `/proc/self/status` is the peak of the child's own memory map. Elsewhere
* `maxRSS` is the child's, in bytes on macOS and KiB on others.
*/

import { readFileSync, writeFileSync } from "node:fs";

const MiB = 1024 * 1024;

function peakBytes(): number {
if (process.platform === "linux") {
const hwm = /^VmHWM:\s+(\d+) kB$/m.exec(readFileSync("/proc/self/status", "utf8"));
if (!hwm?.[1]) throw new Error("no VmHWM in /proc/self/status");
return Number(hwm[1]) * 1024;
}
const maxRSS = process.resourceUsage().maxRSS;
return maxRSS > 4 * MiB ? maxRSS : maxRSS * 1024;
}

const out = process.env.AKOU_TEST_PEAK_FILE;
if (out) process.on("exit", () => writeFileSync(out, String(peakBytes())));
Loading
Loading