diff --git a/AGENTS.md b/AGENTS.md index f49628c..cc51b07 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -45,6 +45,30 @@ The non-negotiable details: `input_image` parts in messages and in tool outputs directly to DeepSeek. The entry keeps `input_modalities = ["text", "image"]` so the desktop app may issue `view_image` calls. GPT image descriptions and `DSCODEX_VISION_MODEL` are not used by the router. + + One carve-out to "forward natively", in `src/image-compaction.mjs`: Codex resends the whole + transcript every turn, images travel inside it as base64 that never shrinks, and the gateway + refuses bodies at ~47 MB, so a session that viewed enough pictures stops being sendable at all. + The answer is **shrink, don't delete**. Every image older than the + newest `keep_recent_images` is re-encoded to **lossy WebP** at `image_max_side` (default 1024) + before anything is sacrificed, so the model still sees every picture. Measured on a real + 44-image session: 47.92 MB of image payload became 10.1 MB, the request went 50.67 → 12.88 MB, + nothing was deleted, and DeepSeek still read the dice faces and their numbers out of + 2182 KB → 12 KB images. Deleting is only the last resort, and a deleted picture must leave an + `input_text` record naming its position, media type, and size — never a silent hole. The WebP + encoder is **found, never required**: this package ships exactly one runtime dependency (`ws`) + and that stays true. The router resolves `sharp` the same way it resolves `ws` — lazily, via + `createRequire` — from `webp_encoder_dir` / `DSCODEX_WEBP_ENCODER_DIR`, or from the documented + drop-in directory `~/.codex/dscodex/encoders`, and reports which one answered in the startup + banner. When nothing resolves it logs "WebP encoder unavailable" and falls back to the record + path rather than failing, so a fresh clone is slower, never broken. Never add the encoder to + `dependencies` or `optionalDependencies`: the promise is one runtime dependency, and + `npm install` would then fetch it. Transcoded bytes are cached by content hash under + `~/.codex/dscodex/image-cache/` because the same picture is re-sent every turn (cold 2.3 s vs + warm 0.24 s on that session). All five knobs — `max_upstream_bytes`, `keep_recent_images`, + `image_max_side`, `webp_quality`, `webp_encoder_dir` — resolve from the process environment and + then from `~/.codex/dscodex/config.json`, because the autostarted router never inherits a shell + variable; `max_upstream_bytes: 0` disables the compaction entirely. 8. The hosted DeepSeek Responses API accepts only the string levels `none|minimal|low|medium|high|xhigh|max`; integer Juice values and `ultra` are rejected with HTTP 400. The catalog exposes exactly two stops, High (`high`) and Max (`max`, the default), and diff --git a/README.en.md b/README.en.md index 75b6678..6146529 100644 --- a/README.en.md +++ b/README.en.md @@ -165,6 +165,7 @@ Yes. Tool calls and web search go through DeepSeek's Responses API. Flash handle - **Usage stats.** The Codex Profile page is read-only, so DeepSeek usage cannot be added to it. - **Reasoning folds mid-task.** DeepSeek emits `response.completed` after every tool round; Codex folds the reasoning block, runs the tool, and opens the next round. This is API behavior, not a bug. Tool-free turns fold once at the end. - **Native vision.** Images and tool-returned images go straight to `deepseek-flash`; GPT image descriptions and `DSCODEX_VISION_MODEL` are no longer used. +- **Request size.** Codex resends the whole transcript every turn and images sit in it as base64 that never shrinks, so a long session eventually reaches DeepSeek's ~47 MB request ceiling and every turn answers with an HTML `413 Payload Too Large` that no retry clears. When an outbound body passes 44 MB the router **shrinks the old pictures instead of deleting them**: everything older than the newest 4 is re-encoded to **lossy WebP** (long side 1024, quality 75). Measured on a real 44-image session: image payload 47.92 MB → 10.1 MB, whole request 50.67 → 12.88 MB, nothing deleted, and the model still read the dice faces and their numbers. Deleting only happens when shrinking cannot fit the body, and it always leaves a record naming the picture's position, media type, and size. Transcoded bytes are cached by content hash under `~/.codex/dscodex/image-cache/`, so each picture is encoded once. The encoder is **found, not required**: this package still declares exactly one runtime dependency (`ws`) and `npm install` fetches nothing extra. To get the shrink path, drop an encoder into `~/.codex/dscodex/encoders` (`npm install --prefix ~/.codex/dscodex/encoders sharp`) or point `webp_encoder_dir` somewhere else; without one the router falls back to records, so a fresh clone is slower rather than broken. Five knobs — `max_upstream_bytes`, `keep_recent_images`, `image_max_side`, `webp_quality`, `webp_encoder_dir` — live in `~/.codex/dscodex/config.json` and are applied on router restart; `max_upstream_bytes: 0` disables the compaction and lets oversized requests fail with the gateway's 413. An autostarted router never sees a variable set in some shell, so the config file is the setting that works. - **DeepSeek → GPT thread history.** Before forwarding to GPT the router strips foreign reasoning (any `reasoning_text` content, or an `encrypted_content` that is not ChatGPT ciphertext; DeepSeek now fills it with a UUID placeholder, see #23) and restores its own encrypted compaction summary as assistant context. Native GPT reasoning and ordinary requests keep their original bytes; rollout files are never rewritten. The same rewrite runs on HTTP SSE and on every Responses WebSocket `response.create`. - **Sub-agents.** DeepSeek-bound `agent_message` items are replayed as `user` messages and content blocks DeepSeek cannot deserialize (`encrypted_content`, …) are rewritten to `input_text`, so spawning a sub-agent no longer fails with 422 / 400 (#24). - **Official GPT WebSocket.** Desktop 26.908+ dials the loopback WS first. The router has to be running for that upgrade to reach chatgpt.com; if it is down, official models Reconnecting 5/5. A DeepSeek-hinted handshake (always present with ChatGPT auth) is rejected with HTTP 426 so the client switches to HTTP Responses with no reconnect retries; without the hint a DeepSeek model on the socket is closed with 1008 to force the same fallback. diff --git a/README.md b/README.md index 6ade929..b805752 100644 --- a/README.md +++ b/README.md @@ -166,6 +166,7 @@ ChatGPT 桌面端 26.908+ 会先连 `ws://127.0.0.1:10110//v1/responses` - **用量统计。** Codex 的 Profile 页面只读,DeepSeek 用量无法计入。 - **思考块反复折叠。** DeepSeek 每轮工具调用结束都发 `response.completed`,Codex 随之折叠思考、执行工具、再展开下一轮。这是 API 行为,不是 bug;无工具的单轮只折叠一次。 - **原生识图。** 图片和工具返回的图片直接交给 `deepseek-flash`,不再借 GPT 代读;`DSCODEX_VISION_MODEL` 不再生效。 +- **单次请求体积。** Codex 每轮重发整段历史,图片以 base64 常驻且不会变小,长会话迟早撞上 DeepSeek 网关约 47 MB 的请求上限,表现是那段会话每轮都回 HTML `413 Payload Too Large`,重试无用。路由器在出站请求超过 44 MB 时**把老图改小而不是删掉**:除最新 4 张外,全部转成 **有损 WebP**(长边 1024、质量 75)。实测一段 44 张图的真实会话:图片从 47.92 MB 降到 10.1 MB,整个请求 50.67 → 12.88 MB,**一张都没丢**,模型照样读得出骰子面数和上面的数字。只有连缩图都压不下去时才会真的丢图,且必留一条写明「第几张、什么类型、多大」的记录。转码结果按内容哈希缓存在 `~/.codex/dscodex/image-cache/`,所以同一张图只在第一次编码。**编码器是「找到就用」,不是依赖**:DSCodex 对外仍然只承诺 `ws` 一个运行依赖,`npm install` 不会因此多装东西。想要缩图能力,把编码器放到 `~/.codex/dscodex/encoders` 即可(`npm install --prefix ~/.codex/dscodex/encoders sharp`,或用 `webp_encoder_dir` 指到别处);没放就自动退回记录方式——全新克隆只会慢一点,不会坏。五个旋钮 `max_upstream_bytes` / `keep_recent_images` / `image_max_side` / `webp_quality` / `webp_encoder_dir` 写进 `~/.codex/dscodex/config.json` 即生效,重启路由器后读取;`max_upstream_bytes: 0` 完全关闭压缩。注意自启动的路由器**读不到**你在某个 shell 里临时设的变量,改写配置文件才行。 - **DeepSeek → GPT 任务历史。** 转发 GPT 前剥掉外来 reasoning(带 `reasoning_text` 内容,或 `encrypted_content` 非 ChatGPT 密文;DeepSeek 现在会填一个 UUID 占位串,见 #23),把 DSCodex 自己的加密压缩摘要恢复为助手上下文;GPT 原生 reasoning 与普通请求保持原始字节,rollout 文件不改写。HTTP SSE 与每条 Responses WebSocket `response.create` 都做这件事。 - **子 agent。** 发给 DeepSeek 的 `agent_message` 以 `user` 角色重放,`encrypted_content` 等 DeepSeek 不认识的内容块改写成 `input_text`,spawn 子 agent 不再 422 / 400(#24)。 - **官方 GPT WebSocket。** 桌面端 26.908+ 先连 loopback WS。路由器必须在跑,upgrade 才会透传到 chatgpt.com;停掉就 Reconnecting 5/5。DeepSeek 的 WS 握手带模型提示(ChatGPT 登录始终会带)时会被直接拒绝(HTTP 426),客户端零重试切到 HTTP Responses;提示缺失时仍在首帧按 close 1008 回退。 diff --git a/llms-full.txt b/llms-full.txt index 7f7ad89..01f60f4 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -44,6 +44,7 @@ Full constraints: https://github.com/fish2lab/DSCodex/blob/main/AGENTS.md - Native DeepSeek tool loops: shell, apply_patch, function calls, web search - Native vision: `input_image` parts are forwarded to `deepseek-flash`; no GPT image descriptions +- Oversized-transcript guard: DeepSeek's gateway refuses request bodies at ~47 MB and Codex resends the whole transcript (images included, as base64) every turn, so a picture-heavy session would otherwise become permanently unsendable. Above a 44 MB budget the router shrinks rather than deletes: every image older than the newest 4 is re-encoded to lossy WebP (long side 1024, quality 75), which took a real 44-image session from a 50.67 MB request to 12.88 MB with no picture lost and no loss of legibility. Deleting is the last resort and always leaves a self-describing record. Transcodes are cached by content hash. The encoder is found rather than required - the package still declares one runtime dependency (`ws`) and installs nothing extra; drop one into `~/.codex/dscodex/encoders`, and without it the guard falls back to records. Tunable through `~/.codex/dscodex/config.json` (`max_upstream_bytes`, `keep_recent_images`, `image_max_side`, `webp_quality`, `webp_encoder_dir`; `0` disables), which is what an autostarted router can actually read. - Remote compaction v2 for DeepSeek: the summary is encrypted into a DSCodex compaction item, never stored as plaintext in the rollout - GPT passthrough stays byte-for-byte unless foreign DeepSeek `reasoning_text` or an undecryptable DSCodex compaction item must be stripped (HTTP SSE and every Responses WebSocket `response.create`, not only the first frame) - Optional macOS app-server bridge for per-provider effort memory in the picker; off by default because it moves the app onto stdio and breaks Computer Use diff --git a/src/cli.mjs b/src/cli.mjs index f3ec0b2..4f1db41 100755 --- a/src/cli.mjs +++ b/src/cli.mjs @@ -32,6 +32,8 @@ import { writeProxyUrl, writeStoredKey, } from "./keys.mjs"; +import { compactionBudgetSource, compactionTunables } from "./compaction-limits.mjs"; +import { ImageTranscodeCache, loadWebpEncoder } from "./image-compaction.mjs"; import { envProxySupported, proxyEnvFor, @@ -489,6 +491,13 @@ async function serve(port) { }; const shutdownToken = randomBytes(32).toString("base64url"); const instanceId = `${process.pid}-${Date.now()}-${randomBytes(8).toString("hex")}`; + const { webpEncoderDir, ...compaction } = compactionTunables(paths); + // WebP transcodes are cached by content hash: the same picture is re-sent + // every turn, and encoding it again each time would be the whole cost of this + // feature. A missing encoder is normal, not fatal: DSCodex ships one runtime + // dependency (`ws`), and the encoder is something the environment provides. + const imageCache = new ImageTranscodeCache(join(paths.stateDir, "image-cache")); + const { encode: imageEncoder, source: encoderSource } = await loadWebpEncoder({ directory: webpEncoderDir }); let server; let shuttingDown = false; const shutdown = () => { @@ -511,6 +520,9 @@ async function serve(port) { shutdownToken, instanceId, onShutdown: shutdown, + ...compaction, + imageCache, + imageEncoder, }); // The serve process owns the pid file so `stop` works no matter who launched // it — `start`, launchd, systemd, or the Windows Task Scheduler. @@ -525,6 +537,18 @@ async function serve(port) { writePidState(paths, { pid: process.pid, port, routerToken, shutdownToken, instanceId }); console.log(`${ts()} DSCodex ${VERSION} listening at http://${HOST}:${port}/v1`); console.log(`${ts()} DeepSeek key: ${deepSeekKey ? "configured" : "missing (GPT OAuth passthrough still works)"}`); + const budget = compaction.maxUpstreamBytes; + console.log(`${ts()} Image-history compaction: ${Number.isFinite(budget) + ? `bodies over ${(budget / 1048576).toFixed(0)}MB shrink the oldest images to WebP q${compaction.webpQuality}` + + ` at ${compaction.imageMaxSide}px, keeping the ${compaction.keepRecentImages} newest untouched` + : "disabled (max_upstream_bytes=0); oversized bodies will fail with 413"}` + + ` [budget from ${compactionBudgetSource(paths)}]`); + console.log(`${ts()} WebP encoder: ${encoderSource}${imageEncoder ? "" : " — oversized bodies fall back to text records"}`); + console.log(`${ts()} Compaction knobs: DSCODEX_MAX_UPSTREAM_BYTES / max_upstream_bytes (0 disables),` + + " DSCODEX_KEEP_RECENT_IMAGES / keep_recent_images, DSCODEX_IMAGE_MAX_SIDE / image_max_side," + + " DSCODEX_WEBP_QUALITY / webp_quality, DSCODEX_WEBP_ENCODER_DIR / webp_encoder_dir — the stored ones live in" + + " ~/.codex/dscodex/config.json and are re-read on every router start, so restart-dscodex.ps1 applies them;" + + " a shell variable only reaches a manually started router."); }); } diff --git a/src/compaction-limits.mjs b/src/compaction-limits.mjs new file mode 100644 index 0000000..1ab6d55 --- /dev/null +++ b/src/compaction-limits.mjs @@ -0,0 +1,119 @@ +// Tunables for the image-history compaction in the proxy. +// +// Resolution order matches the DeepSeek key: a one-off process environment +// variable wins, then the durable value in ~/.codex/dscodex/config.json, then +// the built-in default. +// +// The stored file exists because an environment variable on its own is not +// reachable here: the router is started by the Windows Task Scheduler through a +// wscript shim, so a variable typed into a shell never reaches it, and the +// supervisor that owns the router outlives a router-only hot reload. The config +// file is re-read by the router process itself, which `restart-dscodex.ps1` +// replaces — so the setting lands without touching the scheduler. + +import { readRouterConfig } from "./keys.mjs"; +import { + DEFAULT_IMAGE_MAX_SIDE, + DEFAULT_KEEP_RECENT_IMAGES, + DEFAULT_WEBP_EFFORT, + DEFAULT_WEBP_QUALITY, +} from "./image-compaction.mjs"; +import { join } from "node:path"; + +// The defaults live with the code that uses them, but this module is where the +// effective values are resolved, so it re-exports them for callers and tests. +export { + DEFAULT_IMAGE_MAX_SIDE, + DEFAULT_KEEP_RECENT_IMAGES, + DEFAULT_WEBP_EFFORT, + DEFAULT_WEBP_QUALITY, +} from "./image-compaction.mjs"; + +// DeepSeek's gateway rejects bodies at ~47 MB (measured 2026-10-01: 46 MB is +// served, 47 MB answers 413). 44 MB leaves headroom under that ceiling. +export const DEFAULT_UPSTREAM_BYTE_BUDGET = 44 * 1024 * 1024; + +export const MAX_UPSTREAM_FIELD = "max_upstream_bytes"; +export const KEEP_RECENT_FIELD = "keep_recent_images"; +export const IMAGE_MAX_SIDE_FIELD = "image_max_side"; +export const WEBP_QUALITY_FIELD = "webp_quality"; +export const WEBP_ENCODER_DIR_FIELD = "webp_encoder_dir"; + +// Convention, not configuration: an encoder dropped here is picked up with no +// settings to edit. It sits in the user's DSCodex state directory, so the +// checkout still ships nothing but `ws`. +export const DEFAULT_ENCODER_DIRNAME = "encoders"; + +function integerFrom(value) { + if (value === undefined || value === null) return null; + if (typeof value === "string" && value.trim() === "") return null; + const parsed = Number(value); + return Number.isInteger(parsed) ? parsed : null; +} + +function tuned({ env, envName, config, field, fallback, min, max }) { + const fromEnv = integerFrom(env?.[envName]); + if (fromEnv !== null) return clamp(fromEnv, min, max); + const stored = integerFrom(config?.[field]); + if (stored !== null) return clamp(stored, min, max); + return fallback; +} + +function clamp(value, min, max) { + if (min !== undefined && value < min) return min; + if (max !== undefined && value > max) return max; + return value; +} + +function stringFrom(value) { + return typeof value === "string" && value.trim() ? value.trim() : null; +} + +// Not a limit, but resolved the same way so the banner can report every knob +// from one place. Points at a Node package directory holding the encoder. +function stringTuned({ env, envName, config, field, fallback = "" }) { + return stringFrom(env?.[envName]) ?? stringFrom(config?.[field]) ?? fallback; +} + +// A byte budget of `0` is the documented off switch: no budget is applied, so +// an oversized body is forwarded as-is and fails with the gateway's own 413 +// instead of losing older images. Anything below 1 KB would only ever fire on +// requests that cannot succeed, so it is clamped up to 1 KB. +export function compactionTunables(paths, env = process.env) { + const config = paths?.keyFile ? readRouterConfig(paths.keyFile) : {}; + const budget = tuned({ + env, envName: "DSCODEX_MAX_UPSTREAM_BYTES", config, field: MAX_UPSTREAM_FIELD, + fallback: DEFAULT_UPSTREAM_BYTE_BUDGET, + }); + const keepRecent = tuned({ + env, envName: "DSCODEX_KEEP_RECENT_IMAGES", config, field: KEEP_RECENT_FIELD, + fallback: DEFAULT_KEEP_RECENT_IMAGES, min: 1, + }); + const maxSide = tuned({ + env, envName: "DSCODEX_IMAGE_MAX_SIDE", config, field: IMAGE_MAX_SIDE_FIELD, + fallback: DEFAULT_IMAGE_MAX_SIDE, min: 64, + }); + const quality = tuned({ + env, envName: "DSCODEX_WEBP_QUALITY", config, field: WEBP_QUALITY_FIELD, + fallback: DEFAULT_WEBP_QUALITY, min: 1, max: 100, + }); + return { + maxUpstreamBytes: budget === 0 ? Number.POSITIVE_INFINITY : Math.max(1024, budget), + keepRecentImages: keepRecent, + imageMaxSide: maxSide, + webpQuality: quality, + webpEffort: DEFAULT_WEBP_EFFORT, + webpEncoderDir: stringTuned({ + env, envName: "DSCODEX_WEBP_ENCODER_DIR", config, field: WEBP_ENCODER_DIR_FIELD, + }) || (paths?.stateDir ? join(paths.stateDir, DEFAULT_ENCODER_DIRNAME) : ""), + }; +} + +// Where the effective budget came from, for the startup banner. A value the +// operator cannot see is a value they cannot correct. +export function compactionBudgetSource(paths, env = process.env) { + if (integerFrom(env?.DSCODEX_MAX_UPSTREAM_BYTES) !== null) return "env"; + const config = paths?.keyFile ? readRouterConfig(paths.keyFile) : {}; + if (integerFrom(config?.[MAX_UPSTREAM_FIELD]) !== null) return `config.json:${MAX_UPSTREAM_FIELD}`; + return "default"; +} diff --git a/src/image-compaction.mjs b/src/image-compaction.mjs new file mode 100644 index 0000000..fc118d3 --- /dev/null +++ b/src/image-compaction.mjs @@ -0,0 +1,336 @@ +// Outbound image compaction: keep every picture, make the old ones small. +// +// Codex resends the whole transcript on every turn, and images travel inside it +// as base64 that never shrinks, so a picture-heavy session eventually passes the +// gateway's request-size ceiling and stops being sendable at all. The first +// answer is to make old pictures smaller rather than delete them: on a real +// 44-image session, lossy WebP at long side 1024 took the image payload from +// 47.92 MB to 1.18 MB, and the model still read the dice faces and their +// numbers. Deleting is kept only as the last resort, and a deleted picture +// always leaves a record naming its position, media type, and size. +// +// Two properties make this affordable: +// * the newest `keepRecent` images are never touched, so the turn being +// answered keeps its pictures at full fidelity; +// * transcodes are cached by content hash, because the same picture is +// re-sent every turn and WebP encoding is far too slow to repeat. + +import { createHash } from "node:crypto"; +import { mkdirSync, readFileSync, readdirSync, statSync, unlinkSync, writeFileSync } from "node:fs"; +import { createRequire } from "node:module"; +import { join } from "node:path"; + +export const DEFAULT_KEEP_RECENT_IMAGES = 4; +export const DEFAULT_IMAGE_MAX_SIDE = 1024; +export const DEFAULT_WEBP_QUALITY = 75; +export const DEFAULT_WEBP_EFFORT = 4; +export const DEFAULT_CACHE_BYTES = 256 * 1024 * 1024; + +const IMAGE_PART_KEYS = ["content", "output"]; + +// Re-serializing the whole body on every step is O(n^2) over a body that can be +// tens of megabytes, so the running byte size is maintained by measuring only +// the block being swapped. +function sizeTracker(body) { + let bytes = Buffer.byteLength(JSON.stringify(body), "utf8"); + return { + get: () => bytes, + replace(blocks, slot, block) { + bytes += Buffer.byteLength(JSON.stringify(block), "utf8") + - Buffer.byteLength(JSON.stringify(blocks[slot]), "utf8"); + blocks[slot] = block; + }, + }; +} + +function imageUrlOf(part) { + const value = part?.image_url; + if (typeof value === "string") return value; + const nested = value?.url ?? value?.image_url; + return typeof nested === "string" ? nested : ""; +} + +// Returns each `input_image` block with the exact array slot it lives in, oldest +// first, so a caller can replace the block in place and keep the surrounding +// item (a `function_call_output` pair, a message) structurally untouched. +export function collectImageParts(input) { + const parts = []; + if (!Array.isArray(input)) return parts; + for (const item of input) { + if (!item || typeof item !== "object") continue; + for (const key of IMAGE_PART_KEYS) { + const blocks = item[key]; + if (!Array.isArray(blocks)) continue; + blocks.forEach((block, slot) => { + if (block?.type !== "input_image") return; + const url = imageUrlOf(block); + if (url.length > 512) parts.push({ blocks, slot, url }); + }); + } + } + return parts; +} + +// The record keeps the picture's provenance, so a later turn can tell what was +// lost and ask for a re-render instead of finding an anonymous hole. Replacing +// the whole block is what keeps the marker free of the image-only fields +// (`detail`, `image_url`) that DeepSeek's closed content enum does not expect on +// an `input_text`. +export function imageRecordFor(url, ordinal, total) { + const match = /^data:([^;,]+)(?:;base64)?,/.exec(url); + const mime = match ? match[1] : "image"; + const size = (Buffer.byteLength(url, "utf8") / 1048576).toFixed(2); + return `[image omitted by DSCodex: transcript image ${ordinal} of ${total} (${mime}, ${size} MB as base64)` + + " was replaced by this record because the request body passed the API size limit. The picture is no" + + " longer available here; re-render or re-attach it, or start a new chat, if it matters again.]"; +} + +// Last resort: trade the oldest pictures for text records until the body fits. +// Oldest first, so what survives is closest to the turn being answered. +export function compactImagesToBudget(body, maxBytes, { keepRecent = DEFAULT_KEEP_RECENT_IMAGES } = {}) { + const size = sizeTracker(body); + const before = size.get(); + if (before <= maxBytes) return { before, after: before, dropped: 0, images: 0 }; + const parts = collectImageParts(body.input); + const droppable = parts.slice(0, Math.max(0, parts.length - keepRecent)); + let dropped = 0; + // `droppable` is a prefix of `parts`, so the loop counter is also the image's + // ordinal in the transcript. + for (let ordinal = 0; ordinal < droppable.length; ordinal += 1) { + if (size.get() <= maxBytes) break; + const { blocks, slot, url } = droppable[ordinal]; + size.replace(blocks, slot, { type: "input_text", text: imageRecordFor(url, ordinal + 1, parts.length) }); + dropped += 1; + } + return { before, after: size.get(), dropped, images: parts.length }; +} + +// --- WebP encoder (found, never required) ------------------------------------ +// +// DSCodex advertises exactly one runtime dependency, `ws`, and that stays true: +// the encoder is an accelerator that the environment may provide, not something +// the package installs. The router therefore *looks for* an encoder the way it +// already looks for `ws` — createRequire, lazily, and degrade clearly when it is +// missing — in this order: +// +// 1. `webp_encoder_dir` / DSCODEX_WEBP_ENCODER_DIR: a Node package directory, +// so the encoder can live in the user's DSCodex state directory and never +// appear in this checkout at all; +// 2. this checkout's own node_modules, for anyone who installed one by hand. +// +// Without either, oversized bodies fall back to the record path, which is the +// pre-encoder behaviour and remains correct. + +const encoderCache = new Map(); + +function wrap(sharp) { + return typeof sharp === "function" + ? (dataUrl, options) => encodeWithSharp(sharp, dataUrl, options) + : null; +} + +async function resolveEncoder(directory) { + if (directory) { + try { + const require = createRequire(join(directory, "dscodex-encoder.cjs")); + const encode = wrap(require("sharp")); + return encode + ? { encode, source: `sharp from ${directory}` } + : { encode: null, source: `unavailable (${directory} does not export an encoder)` }; + } catch { + return { encode: null, source: `unavailable (nothing resolvable from ${directory})` }; + } + } + try { + const module = await import("sharp"); + const encode = wrap(module.default); + return encode + ? { encode, source: "sharp from the checkout" } + : { encode: null, source: "unavailable" }; + } catch { + return { encode: null, source: "unavailable" }; + } +} + +/** + * @returns {Promise<{encode: Function|null, source: string}>} `encode` is null + * when the environment provides no encoder, which is a normal outcome. + */ +export function loadWebpEncoder({ directory = "" } = {}) { + const key = directory || "\u0000checkout"; + if (!encoderCache.has(key)) encoderCache.set(key, resolveEncoder(directory)); + return encoderCache.get(key); +} + +export async function encodeWithSharp(sharp, dataUrl, { + maxSide = DEFAULT_IMAGE_MAX_SIDE, + quality = DEFAULT_WEBP_QUALITY, + effort = DEFAULT_WEBP_EFFORT, +} = {}) { + const match = /^data:([^;,]+);base64,(.*)$/s.exec(dataUrl); + if (!match) return null; + let source; + try { + source = Buffer.from(match[2], "base64"); + } catch { + return null; + } + try { + const image = sharp(source, { animated: false, autoOrient: false }); + const meta = await image.metadata(); + const longest = Math.max(meta.width ?? 0, meta.height ?? 0); + const pipeline = longest > maxSide + ? image.resize({ width: maxSide, height: maxSide, fit: "inside", withoutEnlargement: true }) + : image; + const encoded = await pipeline.webp({ quality, effort }).toBuffer(); + // Never make a picture bigger than it already was; some small images and + // already-WebP ones lose nothing by being left alone. + return encoded.length < source.length ? encoded : null; + } catch { + // A corrupt or unsupported picture must not take the whole request down. + return null; + } +} + +// --- transcode cache --------------------------------------------------------- + +export class ImageTranscodeCache { + constructor(directory, { maxBytes = DEFAULT_CACHE_BYTES, pruneEvery = 100 } = {}) { + this.directory = directory; + this.maxBytes = maxBytes; + this.pruneEvery = pruneEvery; + this.writes = 0; + this.hits = 0; + this.misses = 0; + } + + // The policy string is part of the key, so changing quality or size produces + // new entries and the old ones simply age out: no migration needed. + static keyFor(source, policy) { + return createHash("sha256").update(policy).update("\u0000").update(source).digest("hex"); + } + + pathFor(key) { + return join(this.directory, `${key}.webp`); + } + + get(key) { + try { + const data = readFileSync(this.pathFor(key)); + this.hits += 1; + return data; + } catch { + this.misses += 1; + return null; + } + } + + set(key, buffer) { + if (!this.directory) return; + try { + mkdirSync(this.directory, { recursive: true, mode: 0o700 }); + writeFileSync(this.pathFor(key), buffer, { mode: 0o600 }); + this.writes += 1; + if (this.writes % this.pruneEvery === 0) this.prune(); + } catch { + // A cache that cannot be written is a slow router, never a broken one. + } + } + + prune() { + try { + const entries = readdirSync(this.directory) + .filter((name) => name.endsWith(".webp")) + .map((name) => { + const path = join(this.directory, name); + const info = statSync(path); + return { path, size: info.size, atime: info.mtimeMs }; + }) + .sort((a, b) => a.atime - b.atime); + let total = entries.reduce((sum, entry) => sum + entry.size, 0); + for (const entry of entries) { + if (total <= this.maxBytes) break; + unlinkSync(entry.path); + total -= entry.size; + } + } catch { + // Pruning is housekeeping; failing it changes nothing for this request. + } + } +} + +// --- the ladder -------------------------------------------------------------- + +async function transcode(part, { policy, maxSide, quality, effort, cache, encode }) { + const key = cache ? ImageTranscodeCache.keyFor(part.url, policy) : ""; + if (key) { + const cached = cache.get(key); + if (cached) { + return { url: `data:image/webp;base64,${cached.toString("base64")}`, cached: true }; + } + } + const encoded = await encode(part.url, { maxSide, quality, effort }); + if (!encoded) return null; + if (key) cache.set(key, encoded); + return { url: `data:image/webp;base64,${encoded.toString("base64")}`, cached: false }; +} + +/** + * Bring `body` under `maxBytes` by shrinking pictures, and only then by trading + * the oldest ones for text records. + * + * @returns {{before:number, after:number, images:number, transcoded:number, + * fromCache:number, dropped:number, encoder:string}} + */ +export async function shrinkImagesToBudget(body, maxBytes, { + keepRecent = DEFAULT_KEEP_RECENT_IMAGES, + maxSide = DEFAULT_IMAGE_MAX_SIDE, + quality = DEFAULT_WEBP_QUALITY, + effort = DEFAULT_WEBP_EFFORT, + cache = null, + encode = null, +} = {}) { + const size = sizeTracker(body); + const before = size.get(); + const parts = collectImageParts(body.input); + const stats = { + before, + after: before, + images: parts.length, + transcoded: 0, + fromCache: 0, + dropped: 0, + encoder: encode ? "webp" : "unavailable", + }; + if (before <= maxBytes || !encode) { + if (!encode && before > maxBytes) { + // The record pass does its own accounting; this tracker is now stale. + const records = compactImagesToBudget(body, maxBytes, { keepRecent }); + stats.dropped = records.dropped; + stats.after = records.after; + } + return stats; + } + + const policy = `webp|q${quality}|s${maxSide}|e${effort}`; + const shrinkable = parts.slice(0, Math.max(0, parts.length - Math.max(0, keepRecent))); + // Every picture older than the protected window is shrunk, not just enough of + // them to scrape under the limit. Stopping at the limit would leave the + // session pressed against the ceiling, where the next screenshot trips the + // compaction again; shrinking all of them buys real headroom and then stops + // firing at all. Oldest first, so the untouched pictures are the newest. + for (const part of shrinkable) { + const smaller = await transcode(part, { policy, maxSide, quality, effort, cache, encode }); + if (!smaller) continue; + size.replace(part.blocks, part.slot, { type: "input_image", image_url: smaller.url, detail: "high" }); + stats.transcoded += 1; + if (smaller.cached) stats.fromCache += 1; + } + stats.after = size.get(); + if (stats.after > maxBytes) { + const records = compactImagesToBudget(body, maxBytes, { keepRecent }); + stats.dropped = records.dropped; + stats.after = records.after; + } + return stats; +} diff --git a/src/proxy.mjs b/src/proxy.mjs index a9fd939..33a835f 100644 --- a/src/proxy.mjs +++ b/src/proxy.mjs @@ -20,6 +20,19 @@ import { handleResponsesUpgrade, rejectUpgrade, } from "./websocket-proxy.mjs"; +import { + collectImageParts, + compactImagesToBudget, + DEFAULT_IMAGE_MAX_SIDE, + DEFAULT_KEEP_RECENT_IMAGES, + DEFAULT_WEBP_EFFORT, + DEFAULT_WEBP_QUALITY, + shrinkImagesToBudget, +} from "./image-compaction.mjs"; + +// Kept exported from here because this module is the router's public surface and +// the image tests are written against it. +export { collectImageParts, compactImagesToBudget, imageRecordFor } from "./image-compaction.mjs"; const CHATGPT_FORWARDED_REQUEST_HEADERS = new Set([ "authorization", @@ -80,6 +93,12 @@ const HOP_BY_HOP_HEADERS = new Set([ const DEFAULT_MAX_REQUEST_BYTES = 64 * 1024 * 1024; const DEFAULT_MAX_DECODED_BYTES = 128 * 1024 * 1024; +// Measured against api.deepseek.com on 2026-10-01: a 46 MB body is served, 47 MB +// and above get an HTML "413 Request Entity Too Large" from the gateway. Codex +// resends the whole transcript every turn and images travel as base64 that never +// leaves it, so a session that accumulated screenshots stops working entirely. +// 44 MB leaves headroom under the measured ceiling. +const DEFAULT_MAX_UPSTREAM_BYTES = 44 * 1024 * 1024; const SHUTDOWN_HEADER = "x-dscodex-shutdown-token"; const SHUTDOWN_PATH = "/_dscodex/shutdown"; const COMPACTION_PREFIX = "dscodex-compaction-v1:"; @@ -615,6 +634,13 @@ export function createProxyServer({ onShutdown, maxRequestBytes = DEFAULT_MAX_REQUEST_BYTES, maxDecodedBytes = DEFAULT_MAX_DECODED_BYTES, + maxUpstreamBytes = DEFAULT_MAX_UPSTREAM_BYTES, + keepRecentImages = DEFAULT_KEEP_RECENT_IMAGES, + imageMaxSide = DEFAULT_IMAGE_MAX_SIDE, + webpQuality = DEFAULT_WEBP_QUALITY, + webpEffort = DEFAULT_WEBP_EFFORT, + imageCache = null, + imageEncoder = null, openWebSocket, } = {}) { if (!validRouterToken(routerToken)) throw new Error("DSCodex routerToken is required"); @@ -679,7 +705,13 @@ export function createProxyServer({ let direction = "unknown"; try { - const raw = await readRequestBody(request, maxRequestBytes); + const raw = await readRequestBody(request, maxRequestBytes).catch((error) => { + if (error?.statusCode === 413) { + error.message = `Request body exceeds DSCodex's ${(maxRequestBytes / 1048576).toFixed(0)}MB buffering limit;` + + " an upstream 413 means the same thing. Start a new chat: a session this large cannot be sent again."; + } + throw error; + }); let decoded; try { decoded = decodeBody(raw, request.headers["content-encoding"], maxDecodedBytes); @@ -716,6 +748,43 @@ export function createProxyServer({ rewrittenBody = true; } } + // The gateway refuses bodies over its ceiling with an HTML 413 that says + // nothing useful, so shrink oversized image history here, while the reason + // is still known and the client can be told what actually happened. + if (outgoingBody.length > maxUpstreamBytes) { + const budgetBody = deepSeek ? JSON.parse(outgoingBody.toString("utf8")) : null; + if (budgetBody) { + const shrunk = await shrinkImagesToBudget(budgetBody, maxUpstreamBytes, { + keepRecent: keepRecentImages, + maxSide: imageMaxSide, + quality: webpQuality, + effort: webpEffort, + cache: imageCache, + encode: imageEncoder, + }); + outgoingBody = Buffer.from(JSON.stringify(budgetBody)); + const kept = Math.min(keepRecentImages, shrunk.images); + const detail = shrunk.encoder === "unavailable" + ? `WebP encoder unavailable, replaced the oldest ${shrunk.dropped} of ${shrunk.images} images with text records` + : `shrank ${shrunk.transcoded} of ${shrunk.images} images to WebP q${webpQuality}/${imageMaxSide}px` + + (shrunk.fromCache ? ` (${shrunk.fromCache} from cache)` : "") + + (shrunk.dropped ? `, then replaced the oldest ${shrunk.dropped} with text records` : ""); + logger.info?.( + `image compaction: ${(shrunk.before / 1048576).toFixed(1)}MB -> ${(shrunk.after / 1048576).toFixed(1)}MB` + + ` (${detail}; kept the ${kept} newest untouched)`, + ); + } + if (outgoingBody.length > maxUpstreamBytes) { + const error = new Error( + `Request body is ${(outgoingBody.length / 1048576).toFixed(1)}MB, above the ${(maxUpstreamBytes / 1048576).toFixed(0)}MB upstream limit` + + " even after image compaction. Start a new chat, or raise the limit (DSCODEX_MAX_UPSTREAM_BYTES in the router's" + + " environment, or max_upstream_bytes in ~/.codex/dscodex/config.json) and restart the router.", + ); + error.statusCode = 413; + error.upstreamBytes = outgoingBody.length; + throw error; + } + } const baseUrl = deepSeek ? deepSeekBaseUrl : chatGptBaseUrl; const target = new URL(`${baseUrl.replace(/\/$/, "")}${upstreamPath(pathname)}${url.search}`); const headers = copyRequestHeaders(request, deepSeek ? deepSeekKey : undefined); @@ -770,7 +839,12 @@ export function createProxyServer({ logger.error?.(`proxy error (${direction} ${pathname}): ${error instanceof Error ? error.message : String(error)}`); if (!response.headersSent && !response.destroyed) { const status = Number.isInteger(error?.statusCode) ? error.statusCode : 502; - json(response, status, { error: { message: status === 413 ? "Request body too large" : "DSCodex upstream request failed" } }); + const message = status !== 413 + ? "DSCodex upstream request failed" + : error?.upstreamBytes + ? error.message + : "Request body too large"; + json(response, status, { error: { message } }); } else { response.destroy(error instanceof Error ? error : undefined); } diff --git a/test/compaction-limits.test.mjs b/test/compaction-limits.test.mjs new file mode 100644 index 0000000..0a306b8 --- /dev/null +++ b/test/compaction-limits.test.mjs @@ -0,0 +1,122 @@ +import assert from "node:assert/strict"; +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import test from "node:test"; + +import { + compactionBudgetSource, + compactionTunables, + DEFAULT_IMAGE_MAX_SIDE, + DEFAULT_KEEP_RECENT_IMAGES, + DEFAULT_UPSTREAM_BYTE_BUDGET, + DEFAULT_WEBP_EFFORT, + DEFAULT_WEBP_QUALITY, +} from "../src/compaction-limits.mjs"; + +const DEFAULTS = { + maxUpstreamBytes: DEFAULT_UPSTREAM_BYTE_BUDGET, + keepRecentImages: DEFAULT_KEEP_RECENT_IMAGES, + imageMaxSide: DEFAULT_IMAGE_MAX_SIDE, + webpQuality: DEFAULT_WEBP_QUALITY, + webpEffort: DEFAULT_WEBP_EFFORT, + // No paths.stateDir in these fixtures, so no default drop-in directory either. + webpEncoderDir: "", +}; + +function withKeyFile(contents) { + const dir = mkdtempSync(join(tmpdir(), "dscodex-limits-")); + const keyFile = join(dir, "config.json"); + if (contents !== null) writeFileSync(keyFile, contents); + return { keyFile, cleanup: () => rmSync(dir, { recursive: true, force: true }) }; +} + +test("compaction falls back to the built-in budget when nothing is configured", () => { + const { keyFile, cleanup } = withKeyFile(null); + try { + assert.deepEqual(compactionTunables({ keyFile }, {}), DEFAULTS); + assert.equal(compactionBudgetSource({ keyFile }, {}), "default"); + } finally { + cleanup(); + } +}); + +// The autostarted router never sees a shell variable, so the stored config file +// is the setting that has to work — this is the deviation the recheck found. +test("compaction reads the stored config file the router re-reads on start", () => { + const { keyFile, cleanup } = withKeyFile(JSON.stringify({ + deepseek_api_key: "unused-in-this-test", + max_upstream_bytes: 12 * 1024 * 1024, + keep_recent_images: 2, + })); + try { + assert.deepEqual(compactionTunables({ keyFile }, {}), { + ...DEFAULTS, + maxUpstreamBytes: 12 * 1024 * 1024, + keepRecentImages: 2, + }); + assert.equal(compactionBudgetSource({ keyFile }, {}), "config.json:max_upstream_bytes"); + } finally { + cleanup(); + } +}); + +test("a one-off environment variable wins over the stored value", () => { + const { keyFile, cleanup } = withKeyFile(JSON.stringify({ max_upstream_bytes: 12 * 1024 * 1024 })); + try { + const tuned = compactionTunables({ keyFile }, { + DSCODEX_MAX_UPSTREAM_BYTES: "2048", + DSCODEX_KEEP_RECENT_IMAGES: "1", + DSCODEX_IMAGE_MAX_SIDE: "1568", + DSCODEX_WEBP_QUALITY: "82", + }); + assert.equal(tuned.maxUpstreamBytes, 2048); + assert.equal(tuned.keepRecentImages, 1); + assert.equal(tuned.imageMaxSide, 1568); + assert.equal(tuned.webpQuality, 82); + assert.equal(compactionBudgetSource({ keyFile }, { DSCODEX_MAX_UPSTREAM_BYTES: "2048" }), "env"); + } finally { + cleanup(); + } +}); + +// A budget below one kilobyte would only ever fire on bodies that cannot be +// sent anyway, so it is clamped rather than accepted as a "disable" in disguise. +test("a sub-kilobyte budget clamps to 1KB instead of disabling the compaction", () => { + const { keyFile, cleanup } = withKeyFile(JSON.stringify({ max_upstream_bytes: 8 })); + try { + assert.equal(compactionTunables({ keyFile }, {}).maxUpstreamBytes, 1024); + } finally { + cleanup(); + } +}); + +test("a budget of zero disables the compaction from either source", () => { + const fromEnv = withKeyFile(null); + const fromFile = withKeyFile(JSON.stringify({ max_upstream_bytes: 0 })); + const cleanups = [fromEnv.cleanup, fromFile.cleanup]; + try { + for (const paths of [fromEnv, fromFile]) { + const tuned = paths === fromEnv + ? compactionTunables({ keyFile: paths.keyFile }, { DSCODEX_MAX_UPSTREAM_BYTES: "0" }) + : compactionTunables({ keyFile: paths.keyFile }, {}); + assert.equal(tuned.maxUpstreamBytes, Number.POSITIVE_INFINITY); + } + } finally { + for (const cleanup of cleanups) cleanup(); + } +}); + +test("nonsense values fall back to the default, and a negative window clamps to 1", () => { + const { keyFile, cleanup } = withKeyFile(JSON.stringify({ + max_upstream_bytes: "not-a-number", + keep_recent_images: -5, + })); + try { + const tuned = compactionTunables({ keyFile }, { DSCODEX_MAX_UPSTREAM_BYTES: "", DSCODEX_KEEP_RECENT_IMAGES: "banana" }); + assert.equal(tuned.maxUpstreamBytes, DEFAULT_UPSTREAM_BYTE_BUDGET); + assert.equal(tuned.keepRecentImages, 1); + } finally { + cleanup(); + } +}); diff --git a/test/image-compaction.test.mjs b/test/image-compaction.test.mjs new file mode 100644 index 0000000..843b15e --- /dev/null +++ b/test/image-compaction.test.mjs @@ -0,0 +1,197 @@ +import assert from "node:assert/strict"; +import { mkdtempSync, rmSync } from "node:fs"; +import { createRequire } from "node:module"; +import { homedir, tmpdir } from "node:os"; +import { join } from "node:path"; +import test from "node:test"; + +import { + ImageTranscodeCache, + loadWebpEncoder, + shrinkImagesToBudget, +} from "../src/image-compaction.mjs"; +import { DEFAULT_ENCODER_DIRNAME } from "../src/compaction-limits.mjs"; + +const sizeOf = (body) => Buffer.byteLength(JSON.stringify(body), "utf8"); + +function imageBody(imageCount, bytesPerImage) { + const filler = "A".repeat(bytesPerImage); + return { + model: "deepseek/deepseek-flash", + input: Array.from({ length: imageCount }, (_, index) => ({ + type: "function_call_output", + call_id: `call-${index}`, + output: [{ + type: "input_image", + image_url: `data:image/png;base64,${index}${filler}`, + detail: "high", + }], + })), + }; +} + +// A stand-in for the real encoder: whatever it returns is what the source would +// become, so a test can decide the outcome in bytes instead of in pixels. +function fixedEncoder(bytes, { skip = () => false, spy } = {}) { + return async (url, options) => { + if (spy) spy(url, options); + return skip(url) ? null : Buffer.alloc(bytes, 7); + }; +} + +const blocksOf = (body) => body.input.flatMap((item) => item.output); + +test("shrinking covers every picture older than the protected window", async () => { + const imageCount = 12; + const keepRecent = 4; + const body = imageBody(imageCount, 512 * 1024); + const perImage = sizeOf(body) / imageCount; + const budget = Math.floor(perImage * 8.5); + const protectedFrom = imageCount - keepRecent; + const untouched = blocksOf(body).slice(protectedFrom).map((block) => block.image_url); + + const stats = await shrinkImagesToBudget(body, budget, { + keepRecent, + encode: fixedEncoder(1024), + }); + + assert.ok(stats.after <= budget, `expected the body to fit, got ${stats.after} vs ${budget}`); + assert.equal(stats.images, imageCount); + assert.equal(stats.transcoded, imageCount - keepRecent, + "every shrinkable picture is shrunk, so the session keeps real headroom"); + assert.equal(stats.dropped, 0, "no picture should be deleted while shrinking can still fit the body"); + assert.equal(stats.encoder, "webp"); + + const blocks = blocksOf(body); + for (const block of blocks.slice(0, stats.transcoded)) { + assert.equal(block.type, "input_image", "a shrunk picture is still a picture"); + assert.match(block.image_url, /^data:image\/webp;base64,/); + } + // The protected window is byte-identical. + assert.deepEqual(blocks.slice(protectedFrom).map((block) => block.image_url), untouched); + // The item that carried a picture keeps its identity and its output array. + assert.equal(body.input[0].type, "function_call_output"); + assert.equal(body.input[0].call_id, "call-0"); +}); + +test("a body over budget with no encoder falls back to text records", async () => { + const body = imageBody(12, 512 * 1024); + const budget = 2 * 1024 * 1024; + // keepRecent is a hard floor: the newest pictures are never sacrificed, so the + // budget has to be reachable by dropping the ones in front of them. + const stats = await shrinkImagesToBudget(body, budget, { keepRecent: 3, encode: null }); + + assert.equal(stats.encoder, "unavailable"); + assert.equal(stats.transcoded, 0); + assert.ok(stats.dropped > 0, "the record path is the last resort and must still run"); + assert.ok(stats.after <= budget); + assert.equal(blocksOf(body)[0].type, "input_text"); + assert.match(blocksOf(body)[0].text, /image omitted by DSCodex/); +}); + +test("an image the encoder cannot handle is left alone, the rest still shrink", async () => { + const body = imageBody(12, 512 * 1024); + const perImage = sizeOf(body) / 12; + const stats = await shrinkImagesToBudget(body, Math.floor(perImage * 8.5), { + keepRecent: 4, + // The second-oldest picture refuses to encode, like a corrupt file would. + encode: fixedEncoder(1024, { skip: (url) => url.includes("1AAAA") }), + }); + + const blocks = blocksOf(body); + assert.equal(blocks[0].type, "input_image"); + assert.match(blocks[0].image_url, /^data:image\/webp;base64,/); + assert.match(blocks[1].image_url, /^data:image\/png;base64,/, "the rejected picture keeps its bytes"); + assert.ok(stats.transcoded >= 1); +}); + +test("a second turn reuses the cache instead of encoding again", async () => { + const directory = mkdtempSync(join(tmpdir(), "dscodex-image-cache-")); + try { + const source = () => imageBody(12, 512 * 1024); + const budget = Math.floor((sizeOf(source()) / 12) * 8.5); + let calls = 0; + const encode = fixedEncoder(1024, { spy: () => { calls += 1; } }); + + const first = await shrinkImagesToBudget(source(), budget, { + keepRecent: 4, encode, cache: new ImageTranscodeCache(directory), + }); + const afterFirst = calls; + assert.equal(first.fromCache, 0); + assert.equal(afterFirst, first.transcoded, "every transcoded picture was a fresh encode"); + + const second = await shrinkImagesToBudget(source(), budget, { + keepRecent: 4, encode, cache: new ImageTranscodeCache(directory), + }); + assert.equal(second.transcoded, first.transcoded); + assert.equal(second.fromCache, second.transcoded, "the second turn must be all cache hits"); + assert.equal(calls, afterFirst, "the encoder must not run again"); + } finally { + rmSync(directory, { recursive: true, force: true }); + } +}); + +test("a body that already fits is never touched", async () => { + const body = imageBody(3, 1024); + const before = JSON.stringify(body); + let calls = 0; + const stats = await shrinkImagesToBudget(body, 8 * 1024 * 1024, { + encode: fixedEncoder(512, { spy: () => { calls += 1; } }), + }); + assert.equal(stats.transcoded, 0); + assert.equal(stats.dropped, 0); + assert.equal(calls, 0); + assert.equal(JSON.stringify(body), before); +}); + +test("the real encoder produces a WebP DeepSeek can read", async (t) => { + // The checkout ships no encoder, so this looks where the router looks: the + // configured directory, then the documented drop-in location under the user's + // DSCodex state directory. + const directory = process.env.DSCODEX_WEBP_ENCODER_DIR + || join(homedir(), ".codex", "dscodex", DEFAULT_ENCODER_DIRNAME); + const { encode, source } = await loadWebpEncoder({ directory }); + if (!encode) { + t.skip(`no encoder available (${source})`); + return; + } + // A real 800x600 RGBA PNG, built with the encoder's own library so the fixture + // cannot be the thing that is wrong. The pixels are noisy on purpose: a flat + // image compresses so well as PNG that WebP could not beat it, and the code is + // explicitly forbidden from making a picture bigger. + const sharp = createRequire(join(directory, "dscodex-encoder.cjs"))("sharp"); + const width = 800; + const height = 600; + const pixels = Buffer.alloc(width * height * 4); + let seed = 12345; + for (let i = 0; i < pixels.length; i += 4) { + seed = (seed * 1103515245 + 12345) & 0x7fffffff; + pixels[i] = seed & 0xff; + pixels[i + 1] = (seed >> 8) & 0xff; + pixels[i + 2] = (seed >> 16) & 0xff; + pixels[i + 3] = (i / 4) % width < width / 2 ? 255 : 0; + } + const png = await sharp(pixels, { raw: { width, height, channels: 4 } }).png({ compressionLevel: 0 }).toBuffer(); + const body = { + model: "deepseek/deepseek-flash", + input: [ + { type: "function_call_output", call_id: "call-0", output: [{ type: "input_image", image_url: `data:image/png;base64,${png.toString("base64")}`, detail: "high" }] }, + { type: "function_call_output", call_id: "call-1", output: [{ type: "input_image", image_url: `data:image/png;base64,${png.toString("base64")}`, detail: "high" }] }, + ], + }; + // A budget the original cannot meet but a WebP comfortably can. + const budget = Math.floor(sizeOf(body) * 0.6); + const stats = await shrinkImagesToBudget(body, budget, { keepRecent: 1, encode }); + + assert.ok(stats.after <= budget, `${stats.after} should fit ${budget}`); + assert.equal(stats.transcoded, 1, "only the oldest of two pictures needs to shrink"); + const shrunk = blocksOf(body)[0]; + assert.match(shrunk.image_url, /^data:image\/webp;base64,/); + const decoded = await sharp(Buffer.from(shrunk.image_url.split(",")[1], "base64")).metadata(); + assert.equal(decoded.format, "webp"); + assert.equal(decoded.width, Math.min(width, 1024)); + assert.ok(decoded.hasAlpha, "WebP keeps the alpha channel JPEG would have to flatten"); + + const untouched = blocksOf(body)[1]; + assert.match(untouched.image_url, /^data:image\/png;base64,/, "the newest picture is untouched"); +}); diff --git a/test/proxy.test.mjs b/test/proxy.test.mjs index 3a730aa..17fb456 100644 --- a/test/proxy.test.mjs +++ b/test/proxy.test.mjs @@ -5,7 +5,7 @@ import test from "node:test"; import { once } from "node:events"; import { gzipSync, zstdCompressSync } from "node:zlib"; import { WebSocket, WebSocketServer } from "ws"; -import { buildDeepSeekBody, createProxyServer } from "../src/proxy.mjs"; +import { buildDeepSeekBody, compactImagesToBudget, createProxyServer } from "../src/proxy.mjs"; import { requestModel, routingHintModel, safeCloseCode, websocketTarget } from "../src/websocket-proxy.mjs"; const ROUTER_TOKEN = "A".repeat(43); @@ -1368,3 +1368,109 @@ test("falls back to the last good model list when ChatGPT is unreachable or fail const bare = await modelsProxy(t, { chatGptBaseUrl: upstreamUrl }); assert.equal((await fetch(route(bare, "/v1/models"))).status, 502); }); + +function imageBody(imageCount, bytesPerImage) { + const filler = "A".repeat(bytesPerImage); + return { + model: "deepseek/deepseek-flash", + input: Array.from({ length: imageCount }, (_, index) => ({ + type: "function_call_output", + call_id: `call-${index}`, + output: [{ + type: "input_image", + image_url: `data:image/png;base64,${index}${filler}`, + // Real Codex image blocks carry `detail`; a marker must not inherit it. + detail: "high", + }], + })), + }; +} + +function sizeOf(body) { + return Buffer.byteLength(JSON.stringify(body), "utf8"); +} + +test("image compaction leaves a body that already fits the budget untouched", () => { + const body = imageBody(3, 1024); + const before = JSON.stringify(body); + const result = compactImagesToBudget(body, 8 * 1024 * 1024); + assert.deepEqual(result, { before: sizeOf(body), after: sizeOf(body), dropped: 0, images: 0 }); + assert.equal(JSON.stringify(body), before); +}); + +test("image compaction drops the oldest images until the body fits", () => { + const body = imageBody(12, 512 * 1024); + const budget = 2 * 1024 * 1024; + const result = compactImagesToBudget(body, budget, { keepRecent: 3 }); + assert.ok(result.before > budget, `expected the fixture to start oversized, got ${result.before}`); + assert.ok(result.after <= budget, `expected compaction to fit the budget, got ${result.after}`); + assert.equal(result.images, 12); + assert.ok(result.dropped >= 9, `expected at least the 9 oldest images dropped, got ${result.dropped}`); + + const parts = body.input.flatMap((item) => item.output); + const kept = parts.filter((part) => part.type === "input_image"); + assert.equal(kept.length, 12 - result.dropped); + // The newest turn must survive: the tail of the history is what is being answered. + assert.equal(parts[parts.length - 1].type, "input_image"); + for (const part of parts.slice(0, result.dropped)) { + assert.equal(part.type, "input_text"); + assert.equal(part.image_url, undefined); + assert.match(part.text, /image omitted by DSCodex/); + } +}); + +// The direction only shows when the budget is met before the whole droppable +// range is consumed: dropping all of it looks identical either way. A previous +// loop walked the range backwards and kept the oldest picture while discarding +// the ones closest to the live window. +test("a partial compaction keeps the newest images, not the oldest", () => { + const imageCount = 12; + const keepRecent = 4; + const droppable = imageCount - keepRecent; + const body = imageBody(imageCount, 512 * 1024); + const perImage = sizeOf(body) / imageCount; + // Enough for 7 survivors, so 5 of the 8 droppable images have to go. + const budget = Math.floor(perImage * 7.5); + + const result = compactImagesToBudget(body, budget, { keepRecent }); + assert.ok(result.after <= budget, `expected compaction to fit the budget, got ${result.after}`); + assert.ok( + result.dropped > 0 && result.dropped < droppable, + `expected a partial drop of 1..${droppable - 1} images to make the direction observable, got ${result.dropped}`, + ); + + const parts = body.input.flatMap((item) => item.output); + const survivors = parts + .map((part, index) => (part.type === "input_image" ? index : -1)) + .filter((index) => index >= 0); + assert.deepEqual( + survivors, + Array.from({ length: imageCount - result.dropped }, (_, index) => index + result.dropped), + "the surviving images must be the newest tail of the history", + ); + assert.equal(parts[0].type, "input_text", "the oldest image must be the first to go"); + assert.equal(parts[parts.length - 1].type, "input_image"); +}); + +test("a replaced image leaves a clean self-describing record in its own slot", () => { + const body = imageBody(6, 512 * 1024); + const perImage = sizeOf(body) / 6; + const result = compactImagesToBudget(body, Math.floor(perImage * 4.5), { keepRecent: 4 }); + assert.equal(result.dropped, 2); + + const replaced = body.input[0].output[0]; + assert.deepEqual(Object.keys(replaced).sort(), ["text", "type"]); + assert.equal(replaced.type, "input_text"); + assert.match(replaced.text, /image omitted by DSCodex/); + assert.match(replaced.text, /transcript image 1 of 6/); + assert.match(replaced.text, /image\/png/); + + // The item keeps its identity and its output array, so the tool-call replay + // pairing that DeepSeek requires is untouched. + assert.equal(body.input[0].type, "function_call_output"); + assert.equal(body.input[0].call_id, "call-0"); + assert.ok(Array.isArray(body.input[0].output)); + assert.equal(body.input[1].output[0].type, "input_text"); + assert.match(body.input[1].output[0].text, /transcript image 2 of 6/); + assert.equal(body.input[2].output[0].type, "input_image"); +});