Integration batch V (#67, #69, #70)

API.md

@@ -97,6 +97,8 @@ import {
   isCodexUserPromptLine, isCodexStatusLine, isCodexIntermediateChromeLine,
   detectCodexApproval, isApprovalOverlayVisible,
   detectCodexTrustDialog, CODEX_TRUST_DIALOG_ACCEPT_KEYS,
+  CODEX_TRUST_DIALOG_DECLINE_KEYS, CODEX_TRUST_DIALOG_FOLDER_ACCESS_ACCEPT_KEYS,
+  CODEX_TRUST_DIALOG_FOLDER_ACCESS_DECLINE_KEYS,
   diffLines,
   // Transcript
   isCodexConversationEntry, isCodexResponseItem, isCodexEventMsg,
@@ -220,7 +222,7 @@ shared verbatim with `claude-code-headless` (see the header comment in
 | Transcript | `~/.claude/projects/<sanitized-cwd>/<uuid>.jsonl` (per-cwd) | `~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl` (date-bucketed globally) |
 | Assistant marker | `⏺` | `•` (or older `◦`) |
 | User marker | `❯` | `›` |
-| Trust prompt | "Accessing workspace" | "Do you trust the contents of this directory" |
+| Trust prompt | "Accessing workspace" | "Do you trust the contents of this directory" (≤ 0.149); "Folder access" / "Trust this folder?" (0.156+) |
 | Proxy | mitmproxy TLS interceptor | plain HTTP server behind `openai_base_url` |
 
 ---
@@ -421,9 +423,13 @@ also carries `ts: number` (epoch ms).
 
 Notes on the `trust_dialog` action callbacks:
 
-- `accept()` writes `CODEX_TRUST_DIALOG_ACCEPT_KEYS` (`'\r'`) —
-  confirms the pre-selected "Yes, continue".
-- `reject()` writes `'2\r'` — selects "No, quit".
+- `accept()` writes the matched layout's `acceptKeys`: `'1'` on the legacy
+  layout (selects "Yes, continue" at once); `'1\r'` on the 0.156+ layout,
+  where `1` only moves the highlight to option 1 and Enter confirms it.
+- `reject()` writes the layout's `declineKeys`, `'2'` in both layouts. It
+  selects option 2 at once, with no trailing Enter to leak into the next
+  screen. On 0.156+ option 2 is "Quit", or "Back to Agent Command Center"
+  when Codex is connected to its background server.
 
 The `event` flat surface only emits a `{ type: 'trust_dialog', … }`
 member when the dialog becomes **visible**; the simple `trust-dialog`
@@ -1138,7 +1144,7 @@ on-screen.
 | Field | Type | Description |
 | --- | --- | --- |
 | `state` | `CodexTrustDialogState` | The parsed trust dialog (§7.3). |
-| `actions` | `ConditionAction[]` | `accept` (Trust folder, writes `'\r'`), `reject` (Quit, writes `'2\r'`). |
+| `actions` | `ConditionAction[]` | `accept` and `reject`, writing the state's `acceptKeys` / `declineKeys` (above). Legacy labels are "Trust folder" / "Quit"; 0.156+ labels are the option texts painted on screen. |
 
 **`CodexApprovalCondition`** — `kind: 'codex.approval'`
 
@@ -1301,23 +1307,27 @@ title matching, returns `boolean`.
 detectCodexTrustDialog(screen: string): CodexTrustDialogState
 ```
 
-Detects Codex's first-launch-in-a-new-directory trust dialog. **All**
-required markers must be present (conservative — avoids
-false-positiving on assistant text that mentions "trust"):
-`Do you trust the contents of this directory`, `Yes, continue`,
-`No, quit`.
+Detects Codex's first-launch trust dialog in either upstream layout, **structurally**: the dialog's key hint must be the last painted row, with the adjacent option pair above it and the dialog's title line above that. A transcript that quotes the dialog, even verbatim, always has the live composer below it and never matches.
+
+- **Legacy (≤ 0.149.1):** `> You are in <path>`, `1. Yes, continue` / `2. No, quit`, then `Press enter to continue`.
+- **0.156+:** `Folder access` and the path, `1. Trust and continue` (or `Open restricted` / `Open existing task`), then `2. Quit` (or `Back to Agent Command Center`), then `enter continue · esc quit|back`. One wrap of the hint is accepted.
 
 `CodexTrustDialogState`:
 
 | Field | Type | Description |
 | --- | --- | --- |
 | `visible` | `boolean` | |
-| `workspace?` | `string` | The directory Codex asks to trust, parsed from a `> You are in <path>` line. |
-| `options?` | `Array<{ key: string; label: string }>` | The two options, hardcoded `{ '1', 'Yes, continue' }` / `{ '2', 'No, quit' }`. |
+| `workspace?` | `string` | The folder the dialog names. Hard-wrapped path rows are joined. |
+| `trustTarget?` | `string` | 0.156+ only: the Git repository root that trust applies to, when Codex is in a subdirectory. |
+| `options?` | `Array<{ key: string; label: string }>` | The two options, labels as painted. |
+| `layout?` | `'you-are-in' \| 'folder-access'` | Which layout matched. |
+| `acceptKeys?` / `declineKeys?` | `string` | The bytes that choose option 1 / option 2 on that layout. |
+
+Constants:
+- `CODEX_TRUST_DIALOG_ACCEPT_KEYS` = `'1'` and `CODEX_TRUST_DIALOG_DECLINE_KEYS` = `'2'` (legacy layout).
+- `CODEX_TRUST_DIALOG_FOLDER_ACCESS_ACCEPT_KEYS` = `'1\r'` and `CODEX_TRUST_DIALOG_FOLDER_ACCESS_DECLINE_KEYS` = `'2'` (0.156+).
 
-`CODEX_TRUST_DIALOG_ACCEPT_KEYS` = `'\r'` — confirms the pre-selected
-"Yes, continue". (Reject is `'2\r'`, not exported as a constant —
-see the trust-dialog condition's `reject` action.)
+Prefer the state's `acceptKeys` / `declineKeys` over the constants.
 
 ### 7.4 Line diff — `LineDiff.ts`
 
@@ -1601,7 +1611,7 @@ static ResponsesProxy.create(options?: {
 | --- | --- | --- | --- |
 | `authMode` | `CodexAuthMode` (`'apikey' \| 'chatgpt'`) | auto-detected | Which auth path Codex uses. Auto-detection reads `~/.codex/auth.json`; absent → `'apikey'`. |
 | `upstreamBaseUrl` | `string` | derived from `authMode` | The real upstream. Defaults: `apikey` → `https://api.openai.com/v1`, `chatgpt` → `https://chatgpt.com/backend-api/codex`. |
-| `eventsFile` | `string` | unset | If set, every emitted `event` is also written as a JSON line to this file (forensic mirror; `Buffer` payloads are inlined as `{ _buffer_b64 }`. Dumps written before agent-code#372's fix hold `{"type":"Buffer","data":[…]}` byte arrays instead, so a reader of older files must accept both). Written asynchronously through one ordered stream, never on the emitting call. When a line would push the total unwritten bytes (across the live and any rotated-out file) past 16 MiB, it is dropped and counted. A `{kind:'mirror-dropped', droppedEvents, droppedBytes}` line precedes the next written line, or is written at `stop()` if no line follows. Markers obey the same bounds: with a queue cap smaller than one marker (only ever a test setting), the marker is not written and the drops stay counted in the mirror's stats. Call `flushMirror()` to wait for the queue; `stop()` flushes and closes it. |
+| `eventsFile` | `string` | unset | If set, every emitted `event` is also written as a JSON line to this file (forensic mirror; `Buffer` payloads are inlined as `{ _buffer_b64 }`. Dumps written before agent-code#372's fix hold `{"type":"Buffer","data":[…]}` byte arrays instead, so a reader of older files must accept both). Written asynchronously through one ordered stream, never on the emitting call. When a line would push the total unwritten bytes (across the live and any rotated-out file) past 16 MiB, it is dropped and counted. A `{kind:'mirror-dropped', droppedEvents, droppedBytes}` line precedes the next written line, or is written at `stop()` if no line follows. Markers obey the same bounds: with a queue cap smaller than one marker (only ever a test setting), the marker is not written and the drops stay counted in the mirror's stats. Also keeps the newest main-turn `responses*` request body in `latest-request-body.json` beside the file: one line, `{kind:'request-body-latest', requestId, endpoint, body_b64}` (raw on-wire bytes, zstd on current Codex), the name and kind of the Claude addon's sidecar. Requests with an output schema (`request_shape.has_output_schema`, e.g. title generation), subagent requests (`x-openai-subagent` other than `compact`), and bodiless requests never replace it. A newer body removes the old file at once. A write superseded by a newer body never lands, and a body over 16 MiB leaves no file, so the file is never an older prompt. Writes are asynchronous, encoded in 768 KiB slices, and keep at most one pending body. Call `flushMirror()` to wait for the queue and the sidecar; `stop()` flushes and closes them. |
 | `eventsFileMaxBytes` | `number` | 64 MiB | When the next line would pass this size, the file is renamed to `<name>.1.jsonl` (replacing any earlier one) and a fresh file starts with `{kind:'mirror-rotated', rotatedBytes, rotations, droppedEvents, droppedBytes}`. Every file stays within the cap, markers included, so a run holds at most two files of this size. A line that cannot fit even in a fresh file is dropped and counted. A restart that appends to a file already at the cap rotates on its first event. |
 
 `create()` starts listening before resolving. The resolved instance

SECURITY.md

@@ -42,6 +42,13 @@ While the proxy is in use:
   `Authorization` header** so a leaked bundle cannot expose bearer
   tokens — but it **does** record request bodies (your prompts, base64,
   capped at 2 MiB) and response bytes. Treat that file as sensitive.
+- With `eventsFile` set, the proxy also keeps the newest main-turn Responses
+  request body (up to 16 MiB) in `latest-request-body.json` next to it, so a
+  debug bundle has the prompt even after the events tail has moved past it
+  (agent-code#1336). A body between 2 MiB and 16 MiB is persisted **only**
+  there: the mirror omits it. A crash between writing and renaming can also
+  leave a `latest-request-body.json.<pid>.<n>.tmp` copy beside it. Treat all
+  of these as sensitive.
 
 The proxy is **opt-in**. If you don't construct it, no interception
 happens — Codex talks to the upstream directly and the package observes

docs/plans/2026-09-27-latest-request-body.md

@@ -0,0 +1,58 @@
+# Keep the newest Responses request body beside the events file (agent-code#1336)
+
+## Evidence
+- **What the bundle carries.** Agent Code's debug bundle includes the last 5 MiB of `proxy-events.jsonl` (plus `.1`).
+- **Why the prompt falls out.** On Codex the response chunks are the bulk of that file. So after a long stream, or a run of `/v1/models` refreshes, the `request` event with its `body_b64` is no longer in the tail, and the bundle has no prompt.
+- **Measured.** Review C of agent-code#1332 found the last inline body more than 5 MiB before EOF in 40 of 65 Codex files: 10 of 65 after the base64 fix (#53).
+- **The Claude precedent.** Claude solved the same loss with `latest-request-body.json` (claude-code-headless#62, agent-code#1273). Agent Code's `readLatestRequestBody` already appends `<runDir>/latest-request-body.json` for any provider. Codex never writes one.
+
+## Change
+- **New module, `src/proxy/latestRequestBody.ts`:** `LatestRequestBodySidecar`. For each `responses*` request with a body, it replaces the sidecar with `{kind:'request-body-latest', requestId, endpoint, body_b64}`.
+  - **Ordered writes.** One ordered chain of async writes (temp file plus rename): no sync I/O on the main process, and an older body can never rename over a newer one.
+  - **Removal instead of staleness.** It removes the sidecar when the newest body is over 16 MiB (the Claude cap) or a write fails, so it never shows an older prompt as current.
+  - **Bodiless requests are skipped.** `/models` GETs never touch it.
+- **Wiring in `ResponsesProxy`.** It is created with the mirror (same `eventsFile` opt-in) and recorded right after the `request` event. `flushMirror()` and `stop()` also wait for it.
+- **Docs.** `API.md` and `SECURITY.md` document the file, including that it holds prompt text.
+
+## Invariant, and how it differs from Claude
+Claude's sidecar holds the newest body that is NOT in its log, because that log omits bodies past a budget. Codex never omits bodies up to 2 MiB; the loss is in the bundle's tail, which the proxy cannot see. So this sidecar always holds the newest Responses body, even when the log also has it. The cost is at most one body appearing twice in a bundle.
+
+## Tests (`responsesProxy.latestBody.test.ts`, real requests through the proxy to a local upstream)
+1. After two POSTs, the sidecar holds the second body, with its `requestId`, as one line.
+2. A `/models` GET after a prompt leaves the prompt in place.
+3. A body over 16 MiB removes an existing sidecar.
+
+All three are red with the wiring removed.
+
+## Then
+The app bump carries this to Agent Code. Its reader's WHY comment, which describes only the Claude invariant, gets the Codex one.
+
+## Review a (round 1)
+- **Stale "latest"**, after a crash between write and rename, a failed removal, or a read while a newer write was queued. Fixed with three rules (see the module header):
+  - `record()` unlinks the real-name file synchronously;
+  - a superseded write never renames: the generation check and `renameSync` run in one synchronous turn (steering q96: an awaited async rename after the check let `record(B)` unlink the file and A's late rename restore it);
+  - only the newest pending body is kept.
+  If the directory refuses the unlink, nothing can make the file current; that stays unreported, like every mirror failure.
+- **Title generation replaced the main prompt.** Codex 0.157 title turns carry an output schema (`text.format`, `codex_output_schema`). `request_shape` gains `has_output_schema`, and such requests are skipped. A body whose shape cannot be read is still kept.
+- **Main-process cost.** Base64 is encoded in 768 KiB slices, one per turn after a drain, instead of 21 MiB in one call. At most one body is pending.
+- **SECURITY.md accuracy.** Bodies between 2 MiB and 16 MiB are persisted only in the sidecar.
+- **Fresh sessions never reach the file (P1).** This is an app-side selection bug, not a package bug: the bundle asks for `resume-<providerSessionId>` while a fresh run lives under `shell-<paneId>`. It is fixed in the app PR that bumps this package, where it is reachable and testable.
+- **Tests:** compaction is recorded, the title turn is skipped, a newer record removes the file at once, a superseded write cannot resurrect an older body, and a multi-slice body round-trips.
+  - Mutations killed: the endpoint narrowing, the schema check, the generation check and the unlink.
+
+## Steering q96
+- **A late async rename restored an older body.** The first round checked the generation, then awaited an async `rename`. A `record(B)` in that gap unlinked the public file, and A's rename then restored A while B was still being written.
+- **Fix: commit in one turn.** The commit is now `renameSync` in the same turn as the check. It is one metadata call; body bytes are still written asynchronously in slices.
+- **Test: `latestRequestBody.commitFence.test.ts`.** It holds every async `fs/promises` rename at a gate and spies `renameSync`. It records B while A's commit is pending, releases A, and reads the public file before B commits: the file is absent or B, never A. It was red on `a54cfe7` (A restored).
+
+## Review b (round 1)
+- **Subagent calls replaced the main prompt.** Codex tags every non-main Responses call with `x-openai-subagent` (codex-api `requests/headers.rs`: `review`, `compact`, `thread_spawn`, `memory_consolidation`, or a label). They are now skipped, except `compact`, which carries the main conversation. There are tests for `thread_spawn` and `review`, and for a kept `compact`.
+- **16 MiB cap untested.** A body of exactly 16 MiB is now kept, so lowering the cap fails a test.
+- **Crash-left temp copies.** They are now listed in `SECURITY.md`.
+- **P1 (the app side), no package change.** Covered by agent-code#1399, plus a bump that follows #1366. This PR is "For", not "Fixes", #1336.
+
+## Review c
+- **The endpoint filter was unpinned, and my "mutations killed" claim for it was false.** The `/models` test is bodiless, so the filter was never consulted. A new test sends bodied `/memories/trace_summarize` and `/alpha/search` POSTs after a main prompt, and the prompt must stay. Replacing the filter with `true` now fails it.
+- **`has_output_schema` is a heuristic, not a title detector.** Output schemas are a per-turn option upstream. Agent Code sends none on main turns, and a structured main turn would leave the sidecar one turn behind. The comment now says so.
+- **Crash-left temp files were cleaned only after a failure.** They are now also swept after a successful commit, with a test.
+- **Subagent label list in the comment.** Corrected to upstream's (`collab_spawn`, `guardian`, …). The rule itself is label-agnostic.