From 943346d91c9826038179bd9dd314b4a9ac2c6773 Mon Sep 17 00:00:00 2001 From: James Chapman Date: Tue, 17 Nov 2026 09:04:16 +0000 Subject: [PATCH 1/2] feat: cancel every model request and interrupt CLI turns (D6) Version 0.34.0 closes the gaps ADR-0050 left open (ADR-0051): - complete and complete_with_tools take a CancelToken; all four adapter methods share one async transport checked every 25 ms, and reqwest's blocking feature is removed. A non-streamed body read in part is discarded. - Every chat turn creates one token before the compaction check and hands it over on every path. ChatHooks::cancel lets a front end supply it, and TurnRuntime::with_cancel_token passes it to the runtime. A token set before the turn records nothing. - compact_journal takes a token; a cancelled compaction records nothing. - omgcode chat and session compact install the platform interrupt handler through the new watch_interrupt, so Ctrl-C cancels the request and a started turn is recorded as cancelled. capabilities reports cancellable_adapters. - The desktop stops non-streamed turns and pre-turn compaction with no command or wire type change. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 11 +- CHANGELOG.md | 60 ++++ Cargo.lock | 12 +- Cargo.toml | 4 +- IMPLEMENTATION_PLAN.md | 37 ++- crates/omgcode-app/src/chat.rs | 45 ++- crates/omgcode-app/src/chat/preflight.rs | 12 +- crates/omgcode-app/src/chat/text.rs | 48 +-- crates/omgcode-app/src/chat/tools.rs | 14 +- crates/omgcode-app/src/context.rs | 22 +- crates/omgcode-app/src/desktop/chat.rs | 31 +- crates/omgcode-app/src/desktop/turns.rs | 16 +- crates/omgcode-app/src/error.rs | 6 +- crates/omgcode-app/tests/chat_turn.rs | 307 ++++++++++++++++-- crates/omgcode-app/tests/desktop_chat.rs | 100 ++++-- crates/omgcode-cli/src/main.rs | 48 ++- crates/omgcode-cli/tests/cli_workflow.rs | 202 ++++++++++++ crates/omgcode-core/src/provider.rs | 270 ++++++++------- crates/omgcode-core/src/run.rs | 2 +- crates/omgcode-core/src/turn.rs | 13 + crates/omgcode-core/tests/local_provider.rs | 148 ++++++++- crates/omgcode-core/tests/turn_runtime.rs | 34 ++ crates/omgcode-platform/src/interrupt.rs | 37 ++- ...-streamed-model-request-on-cancellation.md | 3 +- ...y-model-request-and-interrupt-cli-turns.md | 142 ++++++++ docs/adr/index.md | 3 +- docs/desktop-interface.md | 59 ++-- docs/development.md | 13 +- docs/provider-authentication.md | 47 ++- docs/session-storage.md | 9 +- docs/testing-standards.md | 15 +- 31 files changed, 1454 insertions(+), 316 deletions(-) create mode 100644 docs/adr/0051-cancel-every-model-request-and-interrupt-cli-turns.md diff --git a/AGENTS.md b/AGENTS.md index 090dd27..ec445f1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -57,10 +57,13 @@ CLI; on by default in the desktop shell, `--stream off` at launch). Streamed text is provisional, and the journals record only the validated reply. The shell pushes chat events to the main window through one caller-scoped channel that needs no `core:` permission ([ADR-0049](docs/adr/0049-stream-local-model-replies-and-push-chat-events.md)). -A cancel request interrupts a streamed model request and the turn is recorded as -cancelled; a non-streamed request, the compaction request, and a tool call in -progress are not interruptible -([ADR-0050](docs/adr/0050-interrupt-a-streamed-model-request-on-cancellation.md)). +A cancel request interrupts any model request, streamed or not, and the +compaction requests, and the turn is recorded as cancelled; a read, list, or +edit tool call in progress is not interruptible +([ADR-0050](docs/adr/0050-interrupt-a-streamed-model-request-on-cancellation.md), +[ADR-0051](docs/adr/0051-cancel-every-model-request-and-interrupt-cli-turns.md)). +On the CLI, Ctrl-C during `chat` or `session compact` cancels instead of ending +the process. The desktop shell has no settings interface, development panels, packaging, or recovery interface. Most other planned features are not implemented. diff --git a/CHANGELOG.md b/CHANGELOG.md index 271195f..73b8128 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,66 @@ Changes are recorded here before a release version is assigned. ### Added +- Version 0.34.0 lets a cancel request stop every model request, streamed or + not, and the compaction requests, and lets an interrupt cancel a CLI turn + (Phase D6). See ADR-0051 and + [cancelling every request](docs/provider-authentication.md#cancelling-every-request-0340). + Until now a non-streamed request and every compaction request ran to their + end or to the thirty-second deadline, a non-streamed text-only turn had no + cancel token, and Ctrl-C ended the CLI and left an open turn. It is a + prototype on branch `codex/request-cancellation`, stacked on + `codex/desktop-interface`; it is not merged. It adds no dependency and + removes reqwest's `blocking` feature, which drops `futures-channel` and + `futures-sink` from two dependency lists in `Cargo.lock`. + - **Adapter:** `complete` and `complete_with_tools` take a `CancelToken`, like + the streamed methods. All four methods share one transport: an async client + built inside a single-thread runtime created for the request, with the + settings and the thirty-second deadline of ADR-0050. The token is checked + before the request is sent, while the response head is awaited, and before + and while each piece of the body is awaited, every 25 ms. A set token drops + the request and returns `ProviderError::Cancelled`; a non-streamed body + read in part is discarded. The 1 MiB response bound is applied as pieces + arrive. The blocking client is removed. The wire request is unchanged. + - **Turn:** every chat turn creates one token after the session checks and + before the compaction check, and `ChatHooks::on_cancel_token` receives it + on every path, streamed or not. The new `ChatHooks::cancel` lets a front + end supply the token, and the new `TurnRuntime::with_cancel_token` hands it + to the runtime. A token already set when the turn would start ends it as + `cancelled` before the turn marker, recording nothing; the text-only path + now matches the runtime here. + - **Compaction:** `compact_journal` takes a token. A cancelled compaction + fails with the error kind `cancelled` and the message "Compaction was + cancelled; nothing was recorded", and the journal is unchanged. + - **CLI:** `omgcode chat` and `session compact` install the interrupt handler + of the trusted `run` command once the journal is open. Ctrl-C or + Ctrl-Break on Windows and `SIGINT` or `SIGTERM` on Linux set the token. The + command fails with the code `cancelled`; a chat turn that had started is + recorded as cancelled and the error carries the session locator. A handler + that cannot be installed fails the command with the new code `interrupt`. + `omgcode-platform` gains `watch_interrupt`, which runs a closure once on an + interrupt; `cancel_on_interrupt` wraps it. `capabilities` reports + `cancellable_adapters: ["lm_studio"]`. + - **Desktop:** no command or wire type changes. `chat_cancel` and exit now + stop a non-streamed request (`--stream off`) and a pre-turn compaction. A + turn cancelled before it handed over its token ends before it is recorded. + - **Tests:** core fixtures for a cancel of a non-streamed text and tool + request before sending, before the head, and with a body held open, and a + slow non-streamed reply under a live token; the runtime's caller token; app + tests for cancelled non-streamed text and tool turns, a cancelled pre-turn + compaction, a cancelled `compact_journal`, and a token set before the turn; + desktop tests for cancelling a non-streamed turn and for the shutdown grace + with a turn held in an event sink; Linux CLI tests that deliver `SIGINT` + during `chat` and `session compact`; and `watch_interrupt` in the platform + interrupt test. Two tests that asserted the old behaviour were rewritten: + `a_text_only_turn_..._calls_the_cancel_hook_once` (it skipped the hook) and + the desktop `cancelling_a_non_streamed_text_turn_stops_its_request` (it + asserted that cancelling changed nothing). Two others were replaced: + `a_non_streamed_text_turn_never_calls_the_cancel_hook_and_cannot_be_cancelled` + and the desktop shutdown-grace test's use of a held non-streamed reply, + which can now be cancelled. + - **Still not interruptible:** a read, list, or edit tool call in progress. + Not verified: a live LM Studio server, and an interrupt delivered to the + CLI on Windows. - Version 0.33.0 interrupts a streamed model request when a turn is cancelled (Phase D5). See ADR-0050 and [cancellation](docs/provider-authentication.md#cancellation-0330). Until now diff --git a/Cargo.lock b/Cargo.lock index 7d81eb3..7eb4467 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1118,7 +1118,6 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b1f9e3d69d39e4862ffed03ed071a76f9a13ba1d9109d355b0f0aa6b15e393c4" dependencies = [ "futures-core", - "futures-sink", ] [[package]] @@ -2523,7 +2522,7 @@ dependencies = [ [[package]] name = "omgcode-app" -version = "0.33.0" +version = "0.34.0" dependencies = [ "directories", "omgcode-core", @@ -2538,7 +2537,7 @@ dependencies = [ [[package]] name = "omgcode-cli" -version = "0.33.0" +version = "0.34.0" dependencies = [ "clap", "omgcode-app", @@ -2551,7 +2550,7 @@ dependencies = [ [[package]] name = "omgcode-core" -version = "0.33.0" +version = "0.34.0" dependencies = [ "libc", "omgcode-platform", @@ -2569,7 +2568,7 @@ dependencies = [ [[package]] name = "omgcode-desktop" -version = "0.33.0" +version = "0.34.0" dependencies = [ "omgcode-app", "serde", @@ -2581,7 +2580,7 @@ dependencies = [ [[package]] name = "omgcode-platform" -version = "0.33.0" +version = "0.34.0" dependencies = [ "libc", "rustix", @@ -3079,7 +3078,6 @@ checksum = "16a1cfa75cc186dd73d5818e510e042e40927bccc9c236b061cea97e1eb08029" dependencies = [ "base64 0.23.1", "bytes", - "futures-channel", "futures-core", "futures-util", "http", diff --git a/Cargo.toml b/Cargo.toml index 604c080..b578690 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -3,7 +3,7 @@ resolver = "3" members = ["crates/omgcode-core", "crates/omgcode-cli", "crates/omgcode-platform", "crates/omgcode-app", "crates/omgcode-desktop"] [workspace.package] -version = "0.33.0" +version = "0.34.0" edition = "2024" rust-version = "1.99.1" license = "Apache-2.0" @@ -17,7 +17,7 @@ uuid = { version = "1", features = ["v4", "serde"] } time = { version = "0.3", features = ["formatting", "parsing", "serde"] } thiserror = "2" tokio = { version = "1", default-features = false, features = ["rt", "time", "net", "fs", "io-util"] } -reqwest = { version = "0.13.5", default-features = false, features = ["blocking", "json", "native-tls"] } +reqwest = { version = "0.13.5", default-features = false, features = ["json", "native-tls"] } sha2 = "0.11.1" clap = { version = "4.6.7", features = ["derive"] } directories = "6" diff --git a/IMPLEMENTATION_PLAN.md b/IMPLEMENTATION_PLAN.md index ce11dec..ecdec74 100644 --- a/IMPLEMENTATION_PLAN.md +++ b/IMPLEMENTATION_PLAN.md @@ -1,6 +1,6 @@ # OMGCode implementation plan -Last verified: 2026-11-16. +Last verified: 2026-11-17. This guide is part of `main` after PR #9. Read it before selecting the next increment. Checkpoint preparation does not mean native replacement exists. @@ -99,6 +99,11 @@ This is a resume guide, not a replacement for the owning contracts in cancelled (ADR-0050). A cancelled turn is recorded with the failure reason `cancelled`, and the desktop exit no longer abandons a streamed turn. It is **not merged and has no pull request**. +- Branch `codex/request-cancellation`, stacked on `codex/desktop-interface` + (open PR #33): **Phase D6**, version **0.34.0**, which lets a cancel request + stop every model request, streamed or not, and the compaction requests, and + lets Ctrl-C cancel a CLI `chat` turn or `session compact` (ADR-0051). It is + **not committed, not merged, and has no pull request**. Implemented today: @@ -234,9 +239,9 @@ Not implemented: - Model create, delete, rename, multi-edit, network, Git, and MCP tools; concurrent-writer detection for any apply route; live provider tool-call qualification; and streaming against a live server, discovery, and - cancellation of a streamed request against a live provider. A non-streamed - request, the compaction request, and a tool call in progress cannot be - interrupted. + cancellation of any request against a live provider. A read, list, or edit + tool call in progress cannot be interrupted, and a CLI interrupt has not been + delivered on Windows. - A run network destination allowlist, runtime skills, memories, mid-turn and provider-native compaction, and caches. Nothing calls the unconfined process runner. @@ -1063,13 +1068,31 @@ login or entitlement route. journal record changes shape. A non-streamed request, the compaction request, and a tool call in progress are not interrupted, and the CLI has no cancel source. The cancel path was checked against local fixtures only. +- **D6 (version 0.34.0):** every model request can be cancelled, and an + interrupt cancels a CLI turn. See + [ADR-0051](docs/adr/0051-cancel-every-model-request-and-interrupt-cli-turns.md). + `complete` and `complete_with_tools` take a `CancelToken`; all four adapter + methods share the async transport of D5, and reqwest's `blocking` feature is + removed. A non-streamed body read in part is discarded. Every chat turn + creates one token before the compaction check and hands it to + `ChatHooks::on_cancel_token` on every path; `ChatHooks::cancel` lets a front + end supply it and `TurnRuntime::with_cancel_token` passes it to the runtime. + A token set before the turn starts records nothing. `compact_journal` takes + a token, and a cancelled compaction records nothing. `omgcode chat` and + `session compact` install the platform interrupt handler through the new + `watch_interrupt`; the command fails with the code `cancelled`, and a + started chat turn is recorded as cancelled. A handler that cannot be + installed fails the command with the code `interrupt`. `capabilities` + reports `cancellable_adapters: ["lm_studio"]`. The desktop stops a + non-streamed turn and a pre-turn compaction with no command change. Checked + against local fixtures; the CLI interrupt was delivered on Linux only. Still open in Phase D: cloud provider adapters, subscription and API-key login, token refresh, account lifecycle, a memory-only session credential, endpoint rules for non-loopback providers, HTTPS, and certificates, qualification of the stream rules and of cancellation against a live LM Studio -server, a cancel source for the CLI, interrupting non-streamed and compaction -requests, and a longer deadline for long +server, an interrupt delivered to the CLI on Windows, interrupting a read, +list, or edit tool call in progress, and a longer deadline for long generations (the thirty-second deadline covers the whole response). The frontend shows nothing about the credential source beyond the endpoint text it already displayed. @@ -1121,6 +1144,8 @@ not merged, and no pull request exists yet. - **D5 (ADR-0050):** `chat_cancel` interrupts a streamed turn, the main window shows the notice "Turn cancelled." decided by the error code `cancelled`, and exit cancels every running turn. No command or wire type changes. +- **D6 (ADR-0051):** `chat_cancel` and exit also stop a non-streamed turn + (`--stream off`) and a pre-turn compaction. No command or wire type changes. Still open for the desktop: diff --git a/crates/omgcode-app/src/chat.rs b/crates/omgcode-app/src/chat.rs index 27e4de0..8be502d 100644 --- a/crates/omgcode-app/src/chat.rs +++ b/crates/omgcode-app/src/chat.rs @@ -93,21 +93,28 @@ pub enum ChatStreamEvent<'a> { Text(&'a str), } -/// Optional callbacks a front end can supply to a turn. +/// Optional callbacks and cancellation source a front end can supply to a turn. /// -/// One hook delivers the cancellation token. Cancellation is cooperative: the -/// core checks it between rounds and tool calls, and a streamed model request -/// also watches it and stops within about 25 ms (ADR-0050). A request that is -/// not streamed cannot be interrupted. The other hook streams provisional -/// text. +/// Every turn has one cancel token, created before compaction and before any +/// model request (ADR-0051). Cancellation is cooperative: the core checks it +/// between rounds and tool calls, and every model request, streamed or not, +/// including the compaction requests before a turn, watches it and stops +/// within about 25 ms (ADR-0050, ADR-0051). A tool call already running is +/// not interrupted, except a command, whose run is cancelled. One hook +/// delivers the token, and the other streams provisional text. #[derive(Default)] pub struct ChatHooks<'a> { - /// Called once, before the first model request: on the tool-enabled path - /// right after the turn runtime is built, and on a text-only turn only when - /// it streams (`on_stream` is set). A text-only turn that does not stream - /// never calls it, because a non-streamed request cannot be interrupted. - /// The callback runs on the turn's thread and must not block; hand the - /// token to another thread and return. + /// The token the turn uses. When `None`, the turn creates its own. A front + /// end that must be able to cancel before the turn starts, such as an + /// interrupt watcher installed beforehand, supplies it here. A token + /// already set when the turn starts ends it as cancelled before any model + /// request. + pub cancel: Option, + /// Called once with a clone of the turn's token, after the session checks + /// and before the compaction check and the first model request, on both + /// the text-only and the tool-enabled path. The callback runs on the + /// turn's thread and must not block; hand the token to another thread and + /// return. pub on_cancel_token: Option<&'a mut dyn FnMut(CancelToken)>, /// When set, every model request of the turn is streamed and this callback /// receives [`ChatStreamEvent::RequestStarted`] before each request and @@ -167,6 +174,8 @@ struct Turn<'a> { settings: &'a ChatSettings, token: Option<&'a str>, data_root: &'a Path, + /// The turn's one cancel token, shared by compaction and every request. + cancel: &'a CancelToken, } /// Run one chat turn, building the clients from the settings first. @@ -193,6 +202,11 @@ pub fn run_chat_turn( /// provider request or runtime is recorded in the journal where the core can /// do so, and an uncertain storage failure is reported, never papered over. /// +/// A cancelled turn returns `cancelled` (ADR-0051). A cancellation during the +/// compaction before a turn records nothing, because neither the compaction +/// nor the turn was recorded yet; a cancellation after the turn started is +/// recorded with the failure reason `cancelled` and leaves no open turn. +/// /// The message and instructions are cleaned of the known token before use. The /// token is used only for redaction and is never part of any returned error. pub fn run_chat_turn_with_clients( @@ -205,12 +219,14 @@ pub fn run_chat_turn_with_clients( let settings = input.settings; let message = redact_known_secret(input.message, input.token); let instructions = redact_known_secret(&settings.instructions, input.token); + let cancel = hooks.cancel.clone().unwrap_or_default(); let turn = Turn { message: &message, instructions: &instructions, settings, token: input.token, data_root, + cancel: &cancel, }; if journal.snapshot().archived { return Err(AppError::new( @@ -224,6 +240,11 @@ pub fn run_chat_turn_with_clients( tools::ACTIVE_TURN_MESSAGE, )); } + // The front end can cancel from here on, which covers the compaction + // requests below as well as the turn's own requests. + if let Some(hook) = hooks.on_cancel_token.as_deref_mut() { + hook(cancel.clone()); + } let mode = select(data_root, journal)?; let tools_enabled = matches!(mode, ToolMode::Enabled { .. }); let tool_extra = if tools_enabled { diff --git a/crates/omgcode-app/src/chat/preflight.rs b/crates/omgcode-app/src/chat/preflight.rs index 5db2145..1ad5e08 100644 --- a/crates/omgcode-app/src/chat/preflight.rs +++ b/crates/omgcode-app/src/chat/preflight.rs @@ -19,7 +19,8 @@ use crate::tool_mode::tool_replay_allowed; /// auto-compaction is a `context` error that names the operator's options. /// With auto-compaction, the summary client compacts the journal and the /// estimate is recomputed from live settings. No control store is held across -/// the summarizer requests. +/// the summarizer requests. The turn's cancel token stops a summarizer +/// request; a cancelled compaction is `cancelled` and records nothing. pub(super) fn preflight_context( journal: &mut SessionJournal, turn: &Turn<'_>, @@ -68,7 +69,14 @@ pub(super) fn preflight_context( format!("{reason}. Use session compact or rerun chat with --auto-compact."), )); } - let outcome = compact_journal(journal, data_root, summary_client, token, summary_budget)?; + let outcome = compact_journal( + journal, + data_root, + summary_client, + token, + summary_budget, + turn.cancel, + )?; summarizer_calls = Some(outcome.calls); // Settings can change during the summarizer requests, so read the // replay authority again for the compacted history. diff --git a/crates/omgcode-app/src/chat/text.rs b/crates/omgcode-app/src/chat/text.rs index 7f53f28..d85cb34 100644 --- a/crates/omgcode-app/src/chat/text.rs +++ b/crates/omgcode-app/src/chat/text.rs @@ -1,13 +1,13 @@ //! The text-only chat turn. //! //! Text-only chat has no model file-read authority. The turn is one provider -//! request. A streamed request watches a cancel token and can be interrupted; a -//! request that is not streamed cannot be (ADR-0050). +//! request, which watches the turn's cancel token and can be interrupted +//! whether or not it streams (ADR-0050, ADR-0051). use omgcode_core::prompt::{self, ConversationMessage, ConversationRole}; use omgcode_core::provider::{LocalModelClient, ProviderError}; use omgcode_core::session::{MessageRole, RequestManifest, SessionError, SessionJournal}; -use omgcode_core::turn::{CancelToken, replay_active}; +use omgcode_core::turn::replay_active; use super::output::TextTurnOutput; use super::stream::StreamSink; @@ -47,6 +47,11 @@ pub(super) fn run( journal .preflight_request(turn.message, &manifest) .map_err(AppError::session)?; + let cancel = turn.cancel; + // As in the runtime, a turn cancelled before it starts records nothing. + if cancel.is_cancelled() { + return Err(AppError::new(ErrorKind::Cancelled, CANCELLED_MESSAGE)); + } let request_id = journal.start_turn().map_err(AppError::session)?; journal .record_message(MessageRole::User, turn.message) @@ -56,30 +61,25 @@ pub(super) fn run( .map_err(AppError::session)?; // A streamed request shows provisional redacted text. The journal is // computed below from the whole reply, exactly as without streaming. - let result = match hooks.on_stream.as_deref_mut() { - Some(on_stream) => { - // Only a streamed request can be interrupted, so only it gets a - // token. The front end receives its clone before the request. - let cancel = CancelToken::new(); - if let Some(hook) = hooks.on_cancel_token.as_deref_mut() { - hook(cancel.clone()); - } - let mut sink = StreamSink::new(on_stream, token); + let mut sink = hooks + .on_stream + .as_deref_mut() + .map(|on_stream| StreamSink::new(on_stream, token)); + let mut result = match sink.as_mut() { + Some(sink) => { sink.start_request(); - let mut result = - client.complete_streaming(&messages, &mut |piece: &str| sink.text(piece), &cancel); - // As in the runtime, a cancellation seen once the reply has - // arrived still ends the turn as cancelled, recording no reply. - if result.is_ok() && cancel.is_cancelled() { - result = Err(ProviderError::Cancelled); - } - if result.is_ok() { - sink.finish_request(); - } - result + client.complete_streaming(&messages, &mut |piece: &str| sink.text(piece), cancel) } - None => client.complete(&messages), + None => client.complete(&messages, cancel), }; + // As in the runtime, a cancellation seen once the reply has arrived still + // ends the turn as cancelled, recording no reply. + if result.is_ok() && cancel.is_cancelled() { + result = Err(ProviderError::Cancelled); + } + if let (Ok(_), Some(sink)) = (&result, sink.as_mut()) { + sink.finish_request(); + } let mut reply = match result { Ok(reply) => reply, Err(ProviderError::Cancelled) => { diff --git a/crates/omgcode-app/src/chat/tools.rs b/crates/omgcode-app/src/chat/tools.rs index e635384..39669cb 100644 --- a/crates/omgcode-app/src/chat/tools.rs +++ b/crates/omgcode-app/src/chat/tools.rs @@ -39,10 +39,8 @@ fn chat_limits() -> TurnLimits { } } -/// Run one tool-enabled turn through the core runtime. -/// -/// `hooks.on_cancel_token` is called once, right after the runtime is built -/// and before the first model request. +/// Run one tool-enabled turn through the core runtime, which uses the turn's +/// cancel token for its checkpoints and every model request. pub(super) fn run( journal: &mut SessionJournal, workspace_id: WorkspaceId, @@ -85,10 +83,10 @@ pub(super) fn run( let redact_tool_text = |text: &str| redact_known_secret(text, token); let report: TurnReport = TurnRuntime::new(journal, turn.data_root, workspace_id, chat_limits()) .and_then(|runtime| { - if let Some(hook) = hooks.on_cancel_token.as_deref_mut() { - hook(runtime.cancel_token()); - } - runtime.with_tool_text_filter(&redact_tool_text).run( + runtime + .with_cancel_token(turn.cancel.clone()) + .with_tool_text_filter(&redact_tool_text) + .run( &mut model, turn.message, turn.instructions, diff --git a/crates/omgcode-app/src/context.rs b/crates/omgcode-app/src/context.rs index 75c7da1..affee48 100644 --- a/crates/omgcode-app/src/context.rs +++ b/crates/omgcode-app/src/context.rs @@ -12,8 +12,9 @@ use omgcode_core::context::{ MAX_SUMMARY_BYTES, SummaryReply, compact, }; use omgcode_core::prompt::{self, ConversationMessage}; -use omgcode_core::provider::LocalModelClient; +use omgcode_core::provider::{LocalModelClient, ProviderError}; use omgcode_core::session::SessionJournal; +use omgcode_core::turn::CancelToken; use serde::Serialize; use crate::error::{AppError, ErrorKind}; @@ -36,14 +37,18 @@ pub fn summary_output_tokens(context_tokens: u64) -> u32 { /// settings for the session's workspace are read before the summarizer input /// is built and again before each request that carries tool-derived content; /// a withdrawal stops compaction with a `permission` error and records nothing. -/// Unreadable settings fail closed: no tool data reaches the summarizer. Other -/// failures are `context`. `token` is removed from summarizer input and output. +/// Unreadable settings fail closed: no tool data reaches the summarizer. +/// `cancel` stops a summarizer request in progress and prevents the next one +/// (ADR-0051); a cancelled compaction is `cancelled` and records nothing, +/// because the journal is written only after the last summary. Other failures +/// are `context`. `token` is removed from summarizer input and output. pub fn compact_journal( journal: &mut SessionJournal, data_root: &Path, client: &LocalModelClient, token: Option<&str>, summary_budget: ContextBudget, + cancel: &CancelToken, ) -> Result { let model = client.model().to_owned(); let workspace = journal.snapshot().workspace_id; @@ -67,7 +72,9 @@ pub fn compact_journal( .collect(); let messages = prompt::assemble(COMPACTION_INSTRUCTIONS, &redacted) .map_err(|_| ContextError::TooLarge)?; - let reply = client.complete(&messages).map_err(ContextError::Provider)?; + let reply = client + .complete(&messages, cancel) + .map_err(ContextError::Provider)?; let usage = reply.usage.as_ref(); Ok(SummaryReply { text: redact_known_secret(&reply.content, token), @@ -78,10 +85,17 @@ pub fn compact_journal( ) .map_err(|error| match error { ContextError::AuthorityWithdrawn => AppError::new(ErrorKind::Permission, error.to_string()), + // Only a set token makes this a cancellation, as in the runtime. + ContextError::Provider(ProviderError::Cancelled) if cancel.is_cancelled() => { + AppError::new(ErrorKind::Cancelled, COMPACTION_CANCELLED_MESSAGE) + } error => AppError::new(ErrorKind::Context, error.to_string()), }) } +/// The message of a cancelled compaction, which recorded nothing. +const COMPACTION_CANCELLED_MESSAGE: &str = "Compaction was cancelled; nothing was recorded"; + /// Visible, conservative context accounting for one chat turn. #[derive(Clone, Debug, Serialize)] pub struct ContextMeter { diff --git a/crates/omgcode-app/src/desktop/chat.rs b/crates/omgcode-app/src/desktop/chat.rs index 1ae1164..1357ee3 100644 --- a/crates/omgcode-app/src/desktop/chat.rs +++ b/crates/omgcode-app/src/desktop/chat.rs @@ -159,14 +159,15 @@ impl DesktopService { /// Request cooperative cancellation of a running turn. /// - /// A turn that streams, text-only or tool-enabled, hands its cancel token - /// to the slot. The request is then seen at once, the streamed model - /// request stops within about 25 ms, and the turn ends as `failed` with the - /// error code `cancelled` (ADR-0050). A tool-enabled turn also checks the - /// token between model rounds and tool calls. A turn whose request is not - /// streamed (`--stream off`) has no cancel token: the request is ignored, - /// the status stays `running`, and the request in flight is not - /// interrupted. `cancelling` is reported only when a cancel token exists. + /// Every turn, text-only or tool-enabled, streamed or not, hands its + /// cancel token to the slot before its first model request, including the + /// compaction requests before a turn (ADR-0051). A request is then seen at + /// once: the model or compaction request in flight stops within about + /// 25 ms, and the turn ends as `failed` with the error code `cancelled` + /// (ADR-0050). A tool-enabled turn also checks the token between model + /// rounds and tool calls. A cancel that arrives before the token is acted + /// on when the token is handed over, which ends the turn before it is + /// recorded. `cancelling` is reported only once a cancel token exists. /// /// # Errors /// @@ -203,12 +204,14 @@ impl DesktopService { /// Request cancellation of every running turn and wait up to `grace` for /// the turn threads to finish, polling for completion. /// - /// Returns without waiting past `grace`. A streamed request stops within - /// about 25 ms, so a streamed turn finishes inside the grace period and - /// leaves no open turn. A request that is not streamed cannot be - /// interrupted: such a turn is reported in `still_running` and leaves its - /// journal with an open turn, which the operator can close later with the - /// interrupt command. After shutdown, `chat_send` is refused. + /// Returns without waiting past `grace`. Every model and compaction + /// request stops within about 25 ms (ADR-0051), so a turn waiting on the + /// model finishes inside the grace period and leaves no open turn. A turn + /// in a step that cannot be interrupted, such as a read, list, or edit tool + /// call, can outlast the grace period: it is reported in `still_running` + /// and may leave its journal with an open turn, which the operator can + /// close later with the interrupt command. After shutdown, `chat_send` is + /// refused. pub fn shutdown(&self, grace: Duration) -> ShutdownReport { self.inner.shutdown_turns(grace) } diff --git a/crates/omgcode-app/src/desktop/turns.rs b/crates/omgcode-app/src/desktop/turns.rs index 23451e3..e4ec908 100644 --- a/crates/omgcode-app/src/desktop/turns.rs +++ b/crates/omgcode-app/src/desktop/turns.rs @@ -40,11 +40,12 @@ const STOPPED_UNEXPECTEDLY: &str = "The chat turn stopped unexpectedly."; pub struct ShutdownReport { /// Turn threads that were in flight at the start and have finished. pub finished: usize, - /// Turn threads still running when the grace period ended. A streamed - /// request is interrupted by the cancel request, so this is a turn whose - /// request is not streamed, or one in a step that cannot be interrupted - /// (a tool call, or compaction). Such a thread may still be waiting; its - /// journal is left with an open turn. + /// Turn threads still running when the grace period ended. Every model + /// request, streamed or not, and every compaction request is interrupted + /// by the cancel request (ADR-0051), so this is a turn in a step that + /// cannot be interrupted, such as a read, list, or edit tool call, or one + /// that has not yet handed over its token. Such a thread may still be + /// waiting; its journal may be left with an open turn. pub still_running: usize, } @@ -54,8 +55,8 @@ pub(super) enum SlotState { Running { /// A cancel was requested (it may have no token to act on yet). cancel_requested: bool, - /// The turn's cancel token, once it exists. A streamed turn has one; a - /// turn whose request is not streamed has none unless it uses tools. + /// The turn's cancel token, once the turn has handed it over. Every + /// turn hands one over before its first model request (ADR-0051). cancel: Option, }, /// The turn ended. @@ -408,6 +409,7 @@ fn execute(inner: &Inner, job: &TurnJob) -> Result { let mut hand_over = |cancel: CancelToken| inner.hand_over_cancel_token(session_id, cancel); let mut forward = |event: ChatStreamEvent<'_>| inner.events_stream(session_id, event); let mut hooks = ChatHooks { + cancel: None, on_cancel_token: Some(&mut hand_over), on_stream: if inner.stream { Some(&mut forward) diff --git a/crates/omgcode-app/src/error.rs b/crates/omgcode-app/src/error.rs index fb8f5b9..9b6978f 100644 --- a/crates/omgcode-app/src/error.rs +++ b/crates/omgcode-app/src/error.rs @@ -33,8 +33,10 @@ pub enum ErrorKind { ToolLimit, /// Any other controlled turn failure, such as a timeout. Turn, - /// The operator cancelled the turn. The journal records the failure - /// reason `cancelled` and no assistant message (ADR-0050). + /// The operator cancelled the turn or the compaction. A turn that had + /// started records the failure reason `cancelled` and no assistant + /// message; a turn or compaction cancelled before it was recorded records + /// nothing (ADR-0050, ADR-0051). Cancelled, /// A selected provider credential could not be used. The message names a /// stable cause code and never contains the secret. diff --git a/crates/omgcode-app/tests/chat_turn.rs b/crates/omgcode-app/tests/chat_turn.rs index 904c252..9dc0a3a 100644 --- a/crates/omgcode-app/tests/chat_turn.rs +++ b/crates/omgcode-app/tests/chat_turn.rs @@ -16,12 +16,14 @@ use std::time::{Duration, Instant}; mod common; use omgcode_app::{ - ChatHooks, ChatSettings, ChatStreamEvent, ChatTurnInput, ChatTurnOutput, create_session, - run_chat_turn, + ChatHooks, ChatSettings, ChatStreamEvent, ChatTurnInput, ChatTurnOutput, compact_journal, + create_session, run_chat_turn, }; +use omgcode_core::context::ContextBudget; use omgcode_core::control::ControlStore; use omgcode_core::policy::PermissionSet; -use omgcode_core::session::SessionJournal; +use omgcode_core::provider::{LocalModelClient, LocalModelConfig}; +use omgcode_core::session::{MessageRole, SessionJournal}; use omgcode_core::turn::CancelToken; use serde_json::{Value, json}; use tempfile::TempDir; @@ -447,12 +449,15 @@ fn a_transport_failure_is_recorded_and_never_mentions_the_token() { } #[test] -fn a_text_only_turn_serializes_the_published_object_and_skips_the_cancel_hook() { +fn a_text_only_turn_serializes_the_published_object_and_calls_the_cancel_hook_once() { let fixture = Fixture::new(false); let mut journal = fixture.journal(); let server = SequenceServer::start(vec![text(&format!("pong {SECRET}"))], None); let mut calls = 0_usize; - let mut on_cancel_token = |_token: CancelToken| calls += 1; + let mut on_cancel_token = |token: CancelToken| { + assert!(!token.is_cancelled()); + calls += 1; + }; let output = turn( &mut journal, &fixture.data, @@ -460,16 +465,15 @@ fn a_text_only_turn_serializes_the_published_object_and_skips_the_cancel_hook() Some(SECRET), &format!("ping {SECRET}"), &mut ChatHooks { + cancel: None, on_cancel_token: Some(&mut on_cancel_token), on_stream: None, }, ) .expect("text-only turn"); assert_eq!(server.finish(), 1); - assert_eq!( - calls, 0, - "a request that is not streamed cannot be interrupted" - ); + // A request that is not streamed can be interrupted too (ADR-0051). + assert_eq!(calls, 1); assert!(!output.tools_enabled()); assert_eq!(output.disabled_reason(), Some("settings_absent")); @@ -518,6 +522,7 @@ fn a_tool_turn_calls_the_cancel_hook_once_and_serializes_the_published_object() None, "hello", &mut ChatHooks { + cancel: None, on_cancel_token: Some(&mut on_cancel_token), on_stream: None, }, @@ -574,6 +579,7 @@ fn a_cancelled_token_ends_the_tool_turn_with_the_core_cancelled_failure() { None, "list the files", &mut ChatHooks { + cancel: None, on_cancel_token: Some(&mut on_cancel_token), on_stream: None, }, @@ -921,6 +927,7 @@ fn cancel_after_the_first_piece(tools_enabled: bool) -> CancelledTurn { None, "say something", &mut ChatHooks { + cancel: None, on_cancel_token: Some(&mut on_cancel_token), on_stream: Some(&mut on_stream), }, @@ -1033,6 +1040,7 @@ fn a_streamed_text_turn_that_is_not_cancelled_hands_over_a_live_token_and_comple None, "greet me", &mut ChatHooks { + cancel: None, on_cancel_token: Some(&mut on_cancel_token), on_stream: Some(&mut on_stream), }, @@ -1079,6 +1087,7 @@ fn a_cancel_seen_only_after_the_reply_arrived_still_ends_the_turn_as_cancelled() None, "say it", &mut ChatHooks { + cancel: None, on_cancel_token: Some(&mut on_cancel_token), on_stream: Some(&mut on_stream), }, @@ -1095,34 +1104,278 @@ fn a_cancel_seen_only_after_the_reply_arrived_still_ends_the_turn_as_cancelled() assert!(!has_assistant_message(&events)); } -#[test] -fn a_non_streamed_text_turn_never_calls_the_cancel_hook_and_cannot_be_cancelled() { - // Given a non-streamed text-only turn and a hook that would cancel. - let fixture = Fixture::new(false); +// ---- Cancelling a non-streamed turn and its compaction (ADR-0051) ---- + +/// What a turn did when its token was cancelled while the server held the +/// reply of its first request. +struct HeldTurn { + error: omgcode_app::AppError, + elapsed: Duration, + served: usize, + has_active_turn: bool, + events_before: usize, + events: Vec, +} + +/// Run a non-streamed turn whose token comes from the caller. The server +/// holds the reply of the first request, which is the compaction request when +/// `chat` compacts, on a gate nobody opens until the turn has returned; a +/// helper thread cancels the token once that request has arrived. +fn cancel_while_a_reply_is_held( + tools_enabled: bool, + chat: impl FnOnce(&str) -> ChatSettings, + prepare: impl FnOnce(&mut SessionJournal), +) -> HeldTurn { + let fixture = Fixture::new(tools_enabled); let mut journal = fixture.journal(); - let server = common::MockServer::start(vec![common::Step::reply(common::text("plain"))]); - let mut calls = 0_usize; - let mut on_cancel_token = |token: CancelToken| { - calls += 1; - token.cancel(); + prepare(&mut journal); + let path = journal.path().to_owned(); + let events_before = close_and_read(journal).len(); + let mut journal = SessionJournal::open(&path).expect("reopen the session"); + let (release, gate) = common::gate(); + let (arrived, arrival) = mpsc::channel(); + let server = common::MockServer::start(vec![ + common::Step::gated(common::text("too late"), gate).announcing(arrived), + ]); + let cancel = CancelToken::new(); + let canceller = { + let cancel = cancel.clone(); + thread::spawn(move || { + common::wait_arrived(&arrival); + cancel.cancel(); + }) }; - - // When the turn runs. - let output = turn( + let started = Instant::now(); + let result = turn( &mut journal, &fixture.data, - &settings(&server.endpoint), + &chat(&server.endpoint), None, "hello", &mut ChatHooks { - on_cancel_token: Some(&mut on_cancel_token), + cancel: Some(cancel), + on_cancel_token: None, on_stream: None, }, + ); + let elapsed = started.elapsed(); + canceller.join().expect("canceller thread"); + // Let the server finish; its held reply finds the connection closed. + drop(release); + let served = server.finish(); + let has_active_turn = journal.snapshot().active_turn.is_some(); + HeldTurn { + error: result.expect_err("the cancelled turn fails"), + elapsed, + served, + has_active_turn, + events_before, + events: close_and_read(journal), + } +} + +#[test] +fn a_non_streamed_text_turn_cancelled_while_waiting_is_recorded_as_cancelled() { + // Given a non-streamed text-only turn whose server holds the reply. + // When the token is cancelled once the request has arrived. + let ended = cancel_while_a_reply_is_held(false, settings, |_| {}); + + // Then the request stops far inside its 30-second deadline, and the + // journal holds a cancelled failure, no reply, and no open turn. + assert_eq!(ended.error.code(), "cancelled"); + assert_eq!(ended.error.message(), "turn was cancelled"); + assert!( + ended.elapsed < CANCEL_BOUND, + "cancellation took {:?}", + ended.elapsed + ); + assert_eq!(ended.served, 1); + assert_eq!( + last_failure_code(&ended.events).as_deref(), + Some("cancelled") + ); + assert!(!has_assistant_message(&ended.events)); + assert!(!ended.has_active_turn); + assert!(!format!("{:?}", ended.events).contains("too late")); +} + +#[test] +fn a_non_streamed_tool_turn_cancelled_while_waiting_runs_no_tool() { + // Given a non-streamed tool-enabled turn whose server holds the reply. + // When the token is cancelled once the request has arrived. + let ended = cancel_while_a_reply_is_held(true, settings, |_| {}); + + // Then the outcome is the one of the text-only path, and no tool ran. + assert_eq!(ended.error.code(), "cancelled"); + assert_eq!(ended.error.message(), "turn was cancelled"); + assert!( + ended.elapsed < CANCEL_BOUND, + "cancellation took {:?}", + ended.elapsed + ); + assert_eq!( + last_failure_code(&ended.events).as_deref(), + Some("cancelled") + ); + assert!(!has_assistant_message(&ended.events)); + assert!(!ended.has_active_turn); + assert!( + ended + .events + .iter() + .all(|event| !event["type"].as_str().unwrap_or("").starts_with("tool.")), + "no tool was requested or run" + ); +} + +/// Two completed exchanges, so that the older one can be compacted. +fn two_exchanges(journal: &mut SessionJournal) { + for index in 0..2 { + let request_id = journal.start_turn().expect("turn"); + journal + .record_message(MessageRole::User, &format!("question {index}")) + .expect("user message"); + journal + .record_message(MessageRole::Assistant, &format!("answer {index}")) + .expect("assistant message"); + journal.complete_turn(request_id).expect("complete turn"); + } +} + +/// Settings that compact before every turn. +fn compacting(endpoint: &str) -> ChatSettings { + let mut chat = settings(endpoint); + chat.compact_threshold = 1; + chat.auto_compact = true; + chat +} + +#[test] +fn cancelling_the_compaction_before_a_turn_records_nothing() { + // Given a session over the compaction trigger with auto-compaction, and a + // server that holds the summarizer's reply. + // When the token is cancelled once the summarizer request has arrived. + let ended = cancel_while_a_reply_is_held(false, compacting, two_exchanges); + + // Then the summarizer request stops promptly, and neither a compaction nor + // a turn was recorded: the journal is exactly as it was. + assert_eq!(ended.error.code(), "cancelled"); + assert_eq!( + ended.error.message(), + "Compaction was cancelled; nothing was recorded" + ); + assert!( + ended.elapsed < CANCEL_BOUND, + "cancellation took {:?}", + ended.elapsed + ); + assert_eq!(ended.served, 1); + assert!(!ended.has_active_turn); + assert_eq!(ended.events.len(), ended.events_before); +} + +#[test] +fn a_token_set_before_the_turn_records_nothing_and_sends_nothing() { + for tools_enabled in [false, true] { + // Given a token that is already cancelled and a server that must not + // be contacted. + let fixture = Fixture::new(tools_enabled); + let mut journal = fixture.journal(); + let server = common::NoCallServer::start(); + let cancel = CancelToken::new(); + cancel.cancel(); + let mut handed = 0_usize; + let mut on_cancel_token = |token: CancelToken| { + assert!(token.is_cancelled(), "the caller's token is handed over"); + handed += 1; + }; + + // When the turn runs. + let error = turn( + &mut journal, + &fixture.data, + &settings(&server.endpoint), + None, + "hello", + &mut ChatHooks { + cancel: Some(cancel), + on_cancel_token: Some(&mut on_cancel_token), + on_stream: None, + }, + ) + .expect_err("a cancelled turn fails"); + + // Then it is cancelled before it starts: no request and no turn. + assert_eq!(error.code(), "cancelled", "tools: {tools_enabled}"); + assert_eq!(handed, 1); + server.assert_untouched(); + assert!( + close_and_read(journal) + .iter() + .all(|event| event["type"] != "turn.started"), + "tools: {tools_enabled}" + ); + } +} + +#[test] +fn a_cancelled_compaction_through_the_command_path_records_nothing() { + // Given a session with two exchanges and a server that holds the summary. + let fixture = Fixture::new(false); + let mut journal = fixture.journal(); + two_exchanges(&mut journal); + let (release, gate) = common::gate(); + let (arrived, arrival) = mpsc::channel(); + let server = common::MockServer::start(vec![ + common::Step::gated(common::text("summary"), gate).announcing(arrived), + ]); + let client = LocalModelClient::new( + LocalModelConfig { + base_url: server.endpoint.clone(), + model: "fixture".to_owned(), + context_tokens: 131_072, + max_output_tokens: 256, + }, + None, + ) + .expect("summary client"); + let cancel = CancelToken::new(); + let canceller = { + let cancel = cancel.clone(); + thread::spawn(move || { + common::wait_arrived(&arrival); + cancel.cancel(); + }) + }; + + // When the token is cancelled while the summarizer request waits. + let started = Instant::now(); + let error = compact_journal( + &mut journal, + &fixture.data, + &client, + None, + ContextBudget { + threshold: u64::MAX, + context_tokens: 131_072, + max_output_tokens: 256, + }, + &cancel, ) - .expect("a request that is not streamed is not interrupted"); + .expect_err("the cancelled compaction fails"); + let elapsed = started.elapsed(); + canceller.join().expect("canceller thread"); + drop(release); server.finish(); - // Then no token was created and the turn completed. - assert_eq!(calls, 0); - assert_eq!(output.reply(), "plain"); + // Then it is cancelled promptly and no compaction was recorded. + assert_eq!(error.code(), "cancelled"); + assert!(elapsed < CANCEL_BOUND, "cancellation took {elapsed:?}"); + assert!(journal.snapshot().compaction.is_none()); + assert!( + close_and_read(journal) + .iter() + .all(|event| event["type"] != "context.compacted"), + "no compaction event" + ); } diff --git a/crates/omgcode-app/tests/desktop_chat.rs b/crates/omgcode-app/tests/desktop_chat.rs index 8c9b500..d012a61 100644 --- a/crates/omgcode-app/tests/desktop_chat.rs +++ b/crates/omgcode-app/tests/desktop_chat.rs @@ -4,7 +4,7 @@ use std::time::{Duration, Instant}; use omgcode_app::desktop::{ - ChatState, ChatStatus, DesktopError, DesktopService, SessionStatus, ToolsMode, + ChatEvent, ChatState, ChatStatus, DesktopError, DesktopService, SessionStatus, ToolsMode, }; use serde_json::json; use uuid::Uuid; @@ -440,7 +440,9 @@ fn cancelling_a_tool_turn_reports_cancelling_then_fails_with_the_core_error() { } #[test] -fn a_cancel_request_does_not_pretend_to_stop_a_text_only_turn() { +fn cancelling_a_non_streamed_text_turn_stops_its_request() { + // Given a text-only turn whose request is not streamed and whose reply the + // server holds until the test ends. let (release, hold) = gate(); let (announce, arrived) = std::sync::mpsc::channel(); let started = started( @@ -448,21 +450,40 @@ fn a_cancel_request_does_not_pretend_to_stop_a_text_only_turn() { vec![Step::gated(text("late answer"), hold).announcing(announce)], ); let service = &started.service; - let session = new_session(service, &started.workspace, "Uncancellable"); + let session = new_session(service, &started.workspace, "Cancellable"); send(service, session, "hi", &started.server.endpoint).expect("send"); wait_arrived(&arrived); + // When the turn is cancelled while the request waits. + let begun = Instant::now(); let after = service .chat_cancel(wire(session_request(session))) .expect("cancel"); - assert_eq!( - after.state, - ChatState::Running, - "no cancellation point exists" + + // Then the turn has a token to act on, its request stops without the + // reply, and the cancellation is recorded (ADR-0051). + assert_eq!(after.state, ChatState::Cancelling); + let done = wait_finished(service, session); + assert!( + begun.elapsed() < Duration::from_secs(5), + "cancellation took {:?}", + begun.elapsed() ); - assert_eq!(status(service, session).state, ChatState::Running); - release.send(()).expect("release the model reply"); - assert_eq!(wait_finished(service, session).state, ChatState::Completed); + assert_eq!(done.state, ChatState::Failed); + let error = done.error.expect("an error"); + assert_eq!(error.code(), "cancelled"); + let page = service + .session_transcript(wire( + json!({"session_id": session, "after_seq": null, "limit": 50}), + )) + .expect("transcript"); + let entries: Vec<_> = page.entries.iter().map(json_of).collect(); + let failure = entries + .iter() + .find(|entry| entry["kind"] == "turn_failed") + .expect("a failed turn is recorded"); + assert_eq!(failure["code"], "cancelled"); + assert!(!format!("{entries:?}").contains("late answer")); // Cancelling something that is not running or unknown is idle or unchanged. let unknown = service @@ -474,23 +495,44 @@ fn a_cancel_request_does_not_pretend_to_stop_a_text_only_turn() { .chat_cancel(wire(session_request(session))) .expect("cancel finished") .state, - ChatState::Completed + ChatState::Failed ); + drop(release); assert_eq!(started.server.finish(), 1); } #[test] fn shutdown_does_not_wait_past_its_grace_and_later_reports_the_turn_finished() { + // Every model request can now be cancelled (ADR-0051), so a turn that + // outlasts the grace period must be held in a step that cannot be: here + // the turn thread is held inside the event sink when its request starts, + // standing in for a tool call in progress. + let fixture = Fixture::new(); + let workspace = fixture.workspace("project", text_only()); + // The cancelled turn never sends its request. + let server = common::NoCallServer::start(); + let service = &fixture.service_streaming(&server.endpoint, common::no_token, true); let (release, hold) = gate(); - let (announce, arrived) = std::sync::mpsc::channel(); - let started = started( - text_only(), - vec![Step::gated(text("slow"), hold).announcing(announce)], - ); - let service = &started.service; - let session = new_session(service, &started.workspace, "Slow"); - send(service, session, "hi", &started.server.endpoint).expect("send"); - wait_arrived(&arrived); + let (held, holding) = std::sync::mpsc::channel(); + let hold = std::sync::Mutex::new(Some((hold, held))); + service + .chat_subscribe(Box::new(move |event| { + if matches!(event, ChatEvent::RequestStarted { .. }) + && let Some((hold, held)) = hold.lock().ok().and_then(|mut slot| slot.take()) + { + // The test may have failed and stopped listening. + held.send(()).ok(); + // A release and a dropped sender both let the turn go on. + hold.recv_timeout(Duration::from_secs(20)).ok(); + } + true + })) + .expect("subscribe"); + let session = new_session(service, &workspace, "Slow"); + send(service, session, "hi", &server.endpoint).expect("send"); + holding + .recv_timeout(Duration::from_secs(10)) + .expect("the turn reaches its request"); let grace = Duration::from_millis(150); let begun = Instant::now(); @@ -502,17 +544,21 @@ fn shutdown_does_not_wait_past_its_grace_and_later_reports_the_turn_finished() { elapsed < grace + Duration::from_secs(2), "did not block past the grace period: {elapsed:?}" ); - assert_eq!(status(service, session).state, ChatState::Running); - let after = send(service, session, "more", &started.server.endpoint) - .expect_err("no new turn after shutdown"); + // The turn holds a token, so it is reported as cancelling. + assert_eq!(status(service, session).state, ChatState::Cancelling); + let after = + send(service, session, "more", &server.endpoint).expect_err("no new turn after shutdown"); assert_eq!(after.code(), "shutting_down"); - // Once the model answers, the thread ends and shutdown reports it finished. - release.send(()).expect("release the model reply"); + // Once released, the thread sees the cancellation before it sends the + // request, ends, and shutdown reports it finished. + release.send(()).expect("release the turn"); let report = service.shutdown(Duration::from_secs(20)); assert_eq!((report.finished, report.still_running), (1, 0)); - assert_eq!(status(service, session).state, ChatState::Completed); - assert_eq!(started.server.finish(), 1); + let done = status(service, session); + assert_eq!(done.state, ChatState::Failed); + assert_eq!(done.error.expect("an error").code(), "cancelled"); + server.assert_untouched(); } #[test] diff --git a/crates/omgcode-cli/src/main.rs b/crates/omgcode-cli/src/main.rs index fd9822f..5229484 100644 --- a/crates/omgcode-cli/src/main.rs +++ b/crates/omgcode-cli/src/main.rs @@ -19,7 +19,9 @@ use omgcode_app::{ use omgcode_core::context::{ContextBudget, DEFAULT_COMPACT_THRESHOLD}; use omgcode_core::prompt::{self, BASE_PROMPT, BASE_PROMPT_VERSION}; use omgcode_core::provider::{LocalModelClient, LocalModelConfig}; +use omgcode_core::run::{InterruptWatch, watch_interrupt}; use omgcode_core::session::{NativePath, SessionJournal}; +use omgcode_core::turn::CancelToken; use serde::Serialize; use serde_json::json; use std::io::{self, Write}; @@ -311,6 +313,7 @@ fn run(cli: Cli) -> Result<(), CliError> { "provider_adapters": false, "stored_credential_adapters": ["lm_studio"], "streaming_adapters": ["lm_studio"], + "cancellable_adapters": ["lm_studio"], "provider_credentials": omgcode_core::control::CredentialBackend::native().is_some(), "credential_store": credential_cli::capability(), "approvals": true, @@ -445,17 +448,26 @@ fn chat(data_dir: Option, mut args: ChatArgs) -> Result<(), CliError> { settings: &settings, token: token_text, }; + // From here an interrupt cancels the turn instead of ending the process, + // so the journal records the cancellation and keeps no open turn. + let cancel = CancelToken::new(); + let interrupts = + cancel_on_interrupt(&cancel).map_err(|error| error.with_session(locator.clone()))?; // Provisional text goes to standard error through one printer, which locks // standard error only for each write, never across a model request. let mut printer = stream_cli::StreamPrinter::new(std::io::stderr()); let turn = { let mut on_stream = |event: ChatStreamEvent<'_>| printer.handle(event); - let mut hooks = ChatHooks::default(); + let mut hooks = ChatHooks { + cancel: Some(cancel), + ..ChatHooks::default() + }; if args.stream { hooks.on_stream = Some(&mut on_stream); } run_chat_turn_with_clients(&mut journal, &data_root, &input, &clients, &mut hooks) }; + stop_interrupts(interrupts); // End the last streamed line, also after a failed turn. printer.finish(); // Every post-open failure, including output failures, carries the locator. @@ -499,6 +511,10 @@ fn compact_session( let data_root = data_root(data_dir)?; let path = resolve_journal(locator.journal, locator.journal_native)?; let mut journal = SessionJournal::open(&path).map_err(session_error)?; + // An interrupt stops the summarizer request in flight; a cancelled + // compaction records nothing. + let cancel = CancelToken::new(); + let interrupts = cancel_on_interrupt(&cancel)?; // Live settings decide whether saved tool results may reach the summarizer. let outcome = compact_journal( &mut journal, @@ -510,7 +526,10 @@ fn compact_session( context_tokens, max_output_tokens: summary_tokens, }, - )?; + &cancel, + ); + stop_interrupts(interrupts); + let outcome = outcome?; let locator = journal_locator(&journal)?; print_json(&JournalOutput { locator, @@ -522,6 +541,31 @@ fn compact_session( }) } +/// Cancel `cancel` when the operator interrupts the command (ADR-0051). +/// +/// Ctrl-C and Ctrl-Break on Windows, and `SIGINT` and `SIGTERM` on Linux, are +/// absorbed from here to the end of the process and set the token instead, so +/// a model or compaction request stops within about 25 ms and the outcome is +/// recorded. A command that cannot install the handler stops before any model +/// request with the code `interrupt`. +fn cancel_on_interrupt(cancel: &CancelToken) -> Result { + let token = cancel.clone(); + watch_interrupt(move || token.cancel()).map_err(|error| { + CliError::new( + "interrupt", + format!("Interrupt handling could not be installed: {error}"), + ) + }) +} + +/// Stop the interrupt watcher once the turn or compaction has returned. +fn stop_interrupts(interrupts: InterruptWatch) { + // The watcher is joined on every path. Its only failure is a panic in + // the watcher thread, which cannot change the outcome that was already + // recorded, so it is not reported over the command's result. + let _ = interrupts.stop(); +} + /// Obtain the endpoint and server token for `chat` or `session compact`. /// /// Without a profile this is exactly what it always was: the command-line diff --git a/crates/omgcode-cli/tests/cli_workflow.rs b/crates/omgcode-cli/tests/cli_workflow.rs index bdaa70c..cf6bb5d 100644 --- a/crates/omgcode-cli/tests/cli_workflow.rs +++ b/crates/omgcode-cli/tests/cli_workflow.rs @@ -1117,3 +1117,205 @@ fn capabilities_report_the_streaming_adapters() { json!(["lm_studio"]) ); } + +#[test] +fn capabilities_report_the_cancellable_adapters() { + let output = run_cli(&["capabilities"]); + assert_success(&output); + assert_eq!( + output_json(&output)["cancellable_adapters"], + json!(["lm_studio"]) + ); +} + +// ---- An operator interrupt cancels a chat turn or a compaction (ADR-0051) ---- + +/// A server that reads one request, announces it, and then holds the +/// connection open without answering until the client closes it or a bound +/// passes. The worker returns whether the client closed the connection. +#[cfg(target_os = "linux")] +fn holding_server() -> (String, Receiver<()>, JoinHandle>) { + let listener = TcpListener::bind(("127.0.0.1", 0)).expect("bind private holding server"); + listener + .set_nonblocking(true) + .expect("set bounded accept mode"); + let address = listener.local_addr().expect("read holding server address"); + let (arrived, arrival) = mpsc::channel(); + let worker = thread::spawn(move || { + let mut stream = accept_with_timeout(&listener)?; + stream + .set_nonblocking(false) + .map_err(|error| error.to_string())?; + stream + .set_read_timeout(Some(MOCK_TIMEOUT)) + .map_err(|error| error.to_string())?; + read_http_body(&mut stream)?; + // The test may already have failed and stopped listening. + arrived.send(()).ok(); + let mut buffer = [0_u8; 64]; + match stream.read(&mut buffer) { + Ok(0) => Ok(true), + Ok(_) => Err("the CLI sent bytes after its request".to_owned()), + Err(error) + if matches!( + error.kind(), + io::ErrorKind::ConnectionReset | io::ErrorKind::ConnectionAborted + ) => + { + Ok(true) + } + Err(error) => Err(error.to_string()), + } + }); + (format!("http://{address}/v1"), arrival, worker) +} + +/// Start `command`, interrupt it with `SIGINT` once the held request has +/// arrived, and return its output and whether it closed the connection. +#[cfg(target_os = "linux")] +fn interrupt_once_the_request_waits(command: &mut Command) -> (Output, bool) { + let (endpoint, arrival, server) = holding_server(); + command + .args(["--endpoint", &endpoint]) + .stdin(Stdio::null()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()); + let child = spawn_gate::spawn(command).expect("spawn the CLI"); + arrival + .recv_timeout(MOCK_TIMEOUT) + .expect("the CLI sends its model request"); + let started = Instant::now(); + let status = Command::new("kill") + .args(["-INT", &child.id().to_string()]) + .status() + .expect("send SIGINT"); + assert!(status.success()); + let (done, finished) = mpsc::channel(); + thread::spawn(move || done.send(child.wait_with_output())); + let output = finished + .recv_timeout(Duration::from_secs(20)) + .expect("the interrupted CLI exits") + .expect("collect the CLI output"); + assert!( + started.elapsed() < Duration::from_secs(5), + "the interrupt took {:?}", + started.elapsed() + ); + let closed = server + .join() + .expect("holding server thread") + .expect("holding server exchange"); + (output, closed) +} + +#[cfg(target_os = "linux")] +fn error_json(output: &Output) -> Value { + serde_json::from_slice(&output.stderr).expect("CLI stderr is one JSON error") +} + +#[cfg(target_os = "linux")] +#[test] +fn an_interrupt_cancels_a_waiting_chat_turn_and_records_it() { + // Given a chat whose model request is held unanswered. + let temporary = TempDir::new().expect("create private test directory"); + let data_dir = temporary.path().join("application-data"); + let workspace = temporary.path().join("workspace"); + std::fs::create_dir(&workspace).expect("create private workspace"); + let mut command = cli(); + command.args([ + "--data-dir", + data_dir.to_str().expect("temporary path is Unicode"), + "chat", + "--workspace", + workspace.to_str().expect("temporary path is Unicode"), + "--model", + "fixture", + "--context-tokens", + "131072", + "--max-output-tokens", + "32", + "say something", + ]); + + // When the operator interrupts the CLI while it waits. + let (output, closed) = interrupt_once_the_request_waits(&mut command); + + // Then the request was dropped, the CLI fails with `cancelled` and the + // session locator, and the journal records a cancelled turn and no reply. + assert!(closed, "the CLI closed the held connection"); + assert!(!output.status.success()); + assert!(output.stdout.is_empty(), "no result document"); + let error = error_json(&output); + assert_eq!(error["error"]["code"], "cancelled"); + let journal = PathBuf::from( + error["error"]["session"]["journal"] + .as_str() + .expect("the error carries the journal path"), + ); + let events = read_events(&journal); + let failure = events.last().expect("a terminal event"); + assert_eq!(failure["type"], "turn.failed"); + assert_eq!(failure["payload"]["code"], "cancelled"); + assert!( + events + .iter() + .all(|event| event["payload"]["role"] != "assistant"), + "no assistant message" + ); + assert_eq!(read_snapshot(&journal).active_turn, None); +} + +#[cfg(target_os = "linux")] +#[test] +fn an_interrupt_cancels_a_waiting_compaction_and_records_nothing() { + // Given a session with two exchanges and a compaction whose summarizer + // request is held unanswered. + let temporary = TempDir::new().expect("create private test directory"); + let data_dir = temporary.path().join("application-data"); + let workspace = temporary.path().join("workspace"); + std::fs::create_dir(&workspace).expect("create private workspace"); + let journal = create_journal(&data_dir, &workspace, "Compaction fixture"); + without_child_spawns(|| { + let mut session = SessionJournal::open(&journal).expect("reopen fixture journal"); + for index in 0..2 { + let request_id = session.start_turn().expect("start fixture turn"); + session + .record_message(MessageRole::User, &format!("question {index}")) + .expect("record fixture question"); + session + .record_message(MessageRole::Assistant, &format!("answer {index}")) + .expect("record fixture answer"); + session + .complete_turn(request_id) + .expect("complete fixture turn"); + } + }); + let before = std::fs::read(&journal).expect("read the journal before"); + let mut command = cli(); + command.args([ + "--data-dir", + data_dir.to_str().expect("temporary path is Unicode"), + "session", + "compact", + journal.to_str().expect("temporary path is Unicode"), + "--model", + "fixture", + "--context-tokens", + "131072", + "--summary-tokens", + "256", + ]); + + // When the operator interrupts the CLI while it waits. + let (output, closed) = interrupt_once_the_request_waits(&mut command); + + // Then the request was dropped, the CLI fails with `cancelled`, and the + // journal is byte for byte what it was. + assert!(closed, "the CLI closed the held connection"); + assert!(!output.status.success()); + assert_eq!(error_json(&output)["error"]["code"], "cancelled"); + assert_eq!( + std::fs::read(&journal).expect("read the journal after"), + before + ); +} diff --git a/crates/omgcode-core/src/provider.rs b/crates/omgcode-core/src/provider.rs index 2b84f36..55c8fb6 100644 --- a/crates/omgcode-core/src/provider.rs +++ b/crates/omgcode-core/src/provider.rs @@ -5,15 +5,12 @@ use crate::storage::parse_strict_json; use crate::turn::{ CancelToken, ModelError, ToolDefinition, ToolModel, ToolModelReply, ToolModelRequest, }; -use reqwest::StatusCode; -use reqwest::Url; -use reqwest::blocking::{Client, Response}; use reqwest::header::{AUTHORIZATION, HeaderMap, HeaderValue}; use reqwest::redirect::Policy; +use reqwest::{Client, Response, StatusCode, Url}; use serde::{Deserialize, Serialize}; use serde_json::Value; use std::fmt; -use std::io::Read; use std::net::IpAddr; use std::time::Duration; use thiserror::Error; @@ -22,8 +19,8 @@ const MAX_REQUEST_BYTES: usize = 256 * 1024; const MAX_RESPONSE_BYTES: u64 = 1024 * 1024; const REQUEST_TIMEOUT: Duration = Duration::from_secs(30); const CONNECT_TIMEOUT: Duration = Duration::from_secs(5); -/// How often a streamed request looks at its cancel token while it waits. -/// It is also the longest a cancellation waits for the next wake-up. +/// How often a request looks at its cancel token while it waits. It is also +/// the longest a cancellation waits for the next wake-up. const CANCEL_POLL: Duration = Duration::from_millis(25); const MESSAGE_OVERHEAD_ESTIMATE: u64 = 256; const PROMPT_HEADROOM_ESTIMATE: u64 = 512; @@ -50,8 +47,11 @@ pub struct LocalModelConfig { } /// A validated local LM Studio Chat Completions client. +/// +/// It holds the validated settings only. Every request builds its own HTTP +/// client inside a single-thread runtime made for that request (ADR-0050, +/// ADR-0051), so no pooled connection outlives the runtime that drives it. pub struct LocalModelClient { - client: Client, endpoint: Url, model: String, context_tokens: u64, @@ -127,8 +127,8 @@ pub enum ProviderError { /// A streamed response ended before its `[DONE]` marker. #[error("local model stream ended before completion")] IncompleteStream, - /// The caller's cancel token stopped a streamed request. Text already - /// delivered stays delivered and no reply exists. + /// The caller's cancel token stopped a request. Text a streamed request + /// already delivered stays delivered, and no reply exists. #[error("local model request was cancelled")] Cancelled, } @@ -200,17 +200,7 @@ impl LocalModelClient { }) .transpose()?; - let client = Client::builder() - .no_proxy() - .redirect(Policy::none()) - .retry(reqwest::retry::never()) - .connect_timeout(CONNECT_TIMEOUT) - .timeout(request_timeout) - .build() - .map_err(|_| ProviderError::Transport)?; - Ok(Self { - client, endpoint, model: config.model, context_tokens: config.context_tokens, @@ -231,9 +221,32 @@ impl LocalModelClient { /// `max_tokens`. It never sends tool definitions or hosted execution fields. /// Model output remains untrusted plain text and is never interpreted as an /// action by this adapter. - pub fn complete(&self, messages: &[ModelMessage]) -> Result { + /// + /// `cancel` stops the request exactly as it stops + /// [`Self::complete_streaming`] (ADR-0051): it is checked before anything + /// is sent, while the response head is awaited, and while each piece of + /// the body is awaited, at least every 25 ms. A set token drops the + /// request, closing the connection, and the result is `Cancelled`; a body + /// read in part is discarded. The method must not be called from inside + /// another async runtime. + /// + /// # Errors + /// + /// A request refused before sending is `InvalidConfiguration`, + /// `ContextLimit`, or `RequestTooLarge`. A connection, read, or deadline + /// failure is `Transport`, a non-success status is `HttpStatus`, a body + /// over 1 MiB is `ResponseTooLarge`, a reply outside the supported schema + /// is `MalformedResponse` or `UnsupportedToolCall`, and a set `cancel` is + /// `Cancelled`. + pub fn complete( + &self, + messages: &[ModelMessage], + cancel: &CancelToken, + ) -> Result { let body = self.text_body(messages, false)?; - parse_response(self.send(body)?) + let bytes = + self.on_request_runtime(body.len(), cancel, self.whole_exchange(body, cancel))?; + parse_response(&bytes) } /// Send one bounded streaming request and pass assistant text to `on_text`. @@ -267,7 +280,11 @@ impl LocalModelClient { cancel: &CancelToken, ) -> Result { let body = self.text_body(messages, true)?; - let outcome = self.stream_request(body, stream::StreamMode::Text, on_text, cancel)?; + let outcome = self.on_request_runtime( + body.len(), + cancel, + self.stream_exchange(body, stream::StreamMode::Text, on_text, cancel), + )?; Ok(ModelReply { content: outcome.content, finish_reason: outcome.finish_reason, @@ -298,14 +315,18 @@ impl LocalModelClient { /// keys, legacy `function_call` output, truncated tool calls, and more than /// 64 calls are refused. Returned tool calls stay raw and unvalidated; the /// caller must validate each one before any dispatch. This method executes - /// nothing and never trusts provider metadata or role labels. + /// nothing and never trusts provider metadata or role labels. `cancel` + /// stops the request as it does for [`Self::complete`]. pub fn complete_with_tools( &self, messages: &[ModelMessage], tools: &[ToolDefinition], + cancel: &CancelToken, ) -> Result { let body = self.tool_body(messages, tools, false)?; - parse_tool_response(self.send(body)?) + let bytes = + self.on_request_runtime(body.len(), cancel, self.whole_exchange(body, cancel))?; + parse_tool_response(&bytes) } /// Send one bounded streaming request that advertises the supplied tools. @@ -326,7 +347,11 @@ impl LocalModelClient { cancel: &CancelToken, ) -> Result { let body = self.tool_body(messages, tools, true)?; - let outcome = self.stream_request(body, stream::StreamMode::Tools, on_text, cancel)?; + let outcome = self.on_request_runtime( + body.len(), + cancel, + self.stream_exchange(body, stream::StreamMode::Tools, on_text, cancel), + )?; Ok(ToolModelReply { content: outcome.content, tool_calls: outcome.tool_calls, @@ -403,55 +428,117 @@ impl LocalModelClient { Ok(()) } - /// Post one encoded non-streaming request body to the validated loopback - /// endpoint with the blocking client. - fn send(&self, body: Vec) -> Result { - if body.len() > MAX_REQUEST_BYTES { + /// Drive one request on a single-thread runtime made for this call. + /// + /// The request size is checked first, and nothing is created, let alone + /// connected, for a token already set. The runtime is dropped on return; + /// the exchange builds its HTTP client inside it, so a pooled connection + /// never outlives the runtime that drives it. A runtime that cannot be + /// built is `Transport`. + fn on_request_runtime( + &self, + body_len: usize, + cancel: &CancelToken, + exchange: impl Future>, + ) -> Result { + if body_len > MAX_REQUEST_BYTES { return Err(ProviderError::RequestTooLarge); } + if cancel.is_cancelled() { + return Err(ProviderError::Cancelled); + } + let runtime = tokio::runtime::Builder::new_current_thread() + .enable_io() + .enable_time() + .build() + .map_err(|_| ProviderError::Transport)?; + runtime.block_on(exchange) + } + + /// Build the HTTP client for one request. Call it only inside the + /// request's runtime. + /// + /// No proxy, no redirect, no retry, the connect timeout, and the request + /// deadline. The async client's `timeout` is a total deadline from the + /// start of the request until the body has finished (reqwest 0.13.5, + /// `async_impl/client.rs`). + fn http_client(&self) -> Result { + Client::builder() + .no_proxy() + .redirect(Policy::none()) + .retry(reqwest::retry::never()) + .connect_timeout(CONNECT_TIMEOUT) + .timeout(self.request_timeout) + .build() + .map_err(|_| ProviderError::Transport) + } - let mut request = self - .client + /// Post one encoded body to the validated loopback endpoint and wait for + /// the response head while watching `cancel`. + async fn send( + &self, + client: &Client, + body: Vec, + streaming: bool, + cancel: &CancelToken, + ) -> Result { + let mut request = client .post(self.endpoint.clone()) .timeout(self.request_timeout) .header(reqwest::header::CONTENT_TYPE, "application/json"); + if streaming { + request = request.header(reqwest::header::ACCEPT, "text/event-stream"); + } if let Some(authorization) = &self.authorization { request = request.header(AUTHORIZATION, authorization.clone()); } - request - .body(body) - .send() + until_cancelled(request.body(body).send(), cancel) + .await? .map_err(|_| ProviderError::Transport) } - /// Run one streamed request on a single-thread runtime made for this call. + /// Send a non-streaming request and read its whole body within 1 MiB. /// - /// The runtime and the async client are built here and dropped here: a - /// pooled connection must not outlive the runtime that drives it, so no - /// client is kept. A runtime that cannot be built is `Transport`. - fn stream_request( + /// The token is checked before each piece of the body as well as while + /// each is awaited. A non-success status is refused before the body is + /// read. Dropping the response or the pending request on any return + /// closes the connection. + async fn whole_exchange( &self, body: Vec, - mode: stream::StreamMode, - on_text: &mut dyn FnMut(&str), cancel: &CancelToken, - ) -> Result { - if body.len() > MAX_REQUEST_BYTES { - return Err(ProviderError::RequestTooLarge); + ) -> Result, ProviderError> { + let client = self.http_client()?; + let mut response = self.send(&client, body, false, cancel).await?; + let status = response.status(); + if !status.is_success() { + return Err(ProviderError::HttpStatus(status.as_u16())); } - // Nothing is created, let alone connected, for a turn already cancelled. - if cancel.is_cancelled() { - return Err(ProviderError::Cancelled); + let mut bytes = Vec::new(); + loop { + if cancel.is_cancelled() { + return Err(ProviderError::Cancelled); + } + let chunk = until_cancelled(response.chunk(), cancel) + .await? + .map_err(|_| ProviderError::Transport)?; + let Some(chunk) = chunk else { + return Ok(bytes); + }; + let total = bytes + .len() + .checked_add(chunk.len()) + .and_then(|total| u64::try_from(total).ok()) + .ok_or(ProviderError::ResponseTooLarge)?; + if total > MAX_RESPONSE_BYTES { + return Err(ProviderError::ResponseTooLarge); + } + bytes.extend_from_slice(&chunk); } - let runtime = tokio::runtime::Builder::new_current_thread() - .enable_io() - .enable_time() - .build() - .map_err(|_| ProviderError::Transport)?; - runtime.block_on(self.stream_exchange(body, mode, on_text, cancel)) } - /// Send the request and feed the response body to the stream parser. + /// Send a streaming request and feed the response body to the stream + /// parser. /// /// Every wait goes through [`until_cancelled`], and the token is also /// checked before each piece of the body, so a fast stream cannot starve @@ -464,29 +551,8 @@ impl LocalModelClient { on_text: &mut dyn FnMut(&str), cancel: &CancelToken, ) -> Result { - // The settings equal the blocking client's. The async client's - // `timeout` is a total deadline from the start of the request until the - // body has finished (reqwest 0.13.5, `async_impl/client.rs`). - let client = reqwest::Client::builder() - .no_proxy() - .redirect(Policy::none()) - .retry(reqwest::retry::never()) - .connect_timeout(CONNECT_TIMEOUT) - .timeout(self.request_timeout) - .build() - .map_err(|_| ProviderError::Transport)?; - let mut request = client - .post(self.endpoint.clone()) - .timeout(self.request_timeout) - .header(reqwest::header::CONTENT_TYPE, "application/json") - .header(reqwest::header::ACCEPT, "text/event-stream"); - if let Some(authorization) = &self.authorization { - request = request.header(AUTHORIZATION, authorization.clone()); - } - - let mut response = until_cancelled(request.body(body).send(), cancel) - .await? - .map_err(|_| ProviderError::Transport)?; + let client = self.http_client()?; + let mut response = self.send(&client, body, true, cancel).await?; check_streaming_head(response.status(), response.headers())?; let mut parser = stream::StreamParser::new(mode); @@ -582,9 +648,10 @@ impl ToolModel for LocalToolModel<'_> { &mut **on_text, request.cancel, ), - None => self - .client - .complete_with_tools(request.messages, request.tools), + None => { + self.client + .complete_with_tools(request.messages, request.tools, request.cancel) + } }; result.map_err(|error| { self.last_error = Some(error); @@ -753,34 +820,14 @@ fn check_streaming_head(status: StatusCode, headers: &HeaderMap) -> Result<(), P } } -/// Read a successful response body within the byte limit. -fn read_body(response: Response) -> Result, ProviderError> { - let status = response.status(); - if !status.is_success() { - return Err(ProviderError::HttpStatus(status.as_u16())); - } - - let mut bytes = Vec::new(); - response - .take(MAX_RESPONSE_BYTES + 1) - .read_to_end(&mut bytes) - .map_err(|_| ProviderError::Transport)?; - if u64::try_from(bytes.len()).map_err(|_| ProviderError::ResponseTooLarge)? > MAX_RESPONSE_BYTES - { - return Err(ProviderError::ResponseTooLarge); - } - Ok(bytes) -} - /// Parse one tool-capable reply strictly, leaving each call unvalidated. /// /// Duplicate keys at any depth are refused before any value is inspected, so a /// call cannot carry two names or two argument strings. Content may be `null` /// only when the reply contains tool calls. A reply cut short by the token /// limit cannot carry tool calls, because their arguments may be truncated. -fn parse_tool_response(response: Response) -> Result { - let bytes = read_body(response)?; - let value = parse_strict_json(&bytes).map_err(|_| ProviderError::MalformedResponse)?; +fn parse_tool_response(bytes: &[u8]) -> Result { + let value = parse_strict_json(bytes).map_err(|_| ProviderError::MalformedResponse)?; let choices = value .get("choices") .and_then(Value::as_array) @@ -828,10 +875,9 @@ fn parse_tool_response(response: Response) -> Result Result { - let bytes = read_body(response)?; +fn parse_response(bytes: &[u8]) -> Result { let value: Value = - serde_json::from_slice(&bytes).map_err(|_| ProviderError::MalformedResponse)?; + serde_json::from_slice(bytes).map_err(|_| ProviderError::MalformedResponse)?; let choices = value .get("choices") .and_then(Value::as_array) @@ -941,7 +987,7 @@ pub(crate) const REFUSED_ENDPOINTS: &[&str] = &[ #[cfg(test)] mod tests { use super::*; - use std::io::Write; + use std::io::{Read, Write}; /// The client refuses every listed destination; the profile record tests /// run the same list through `validate_base_url`'s other caller. @@ -1001,7 +1047,11 @@ mod tests { assert_eq!(client.check_messages(&messages, 0), Ok(())); assert_eq!( client - .complete_with_tools(&messages, &crate::turn::tool_definitions()) + .complete_with_tools( + &messages, + &crate::turn::tool_definitions(), + &CancelToken::new() + ) .err(), Some(ProviderError::ContextLimit) ); @@ -1024,7 +1074,9 @@ mod tests { Err(ProviderError::RequestTooLarge) ); assert_eq!( - roomy.complete_with_tools(&messages, &[]).err(), + roomy + .complete_with_tools(&messages, &[], &CancelToken::new()) + .err(), Some(ProviderError::InvalidConfiguration) ); } @@ -1154,7 +1206,7 @@ mod tests { let result = if streamed { client.complete_streaming(&messages, &mut |_: &str| {}, &CancelToken::new()) } else { - client.complete(&messages) + client.complete(&messages, &CancelToken::new()) }; let elapsed = started.elapsed(); // Receiver disconnection also requests cleanup; either outcome is safe. diff --git a/crates/omgcode-core/src/run.rs b/crates/omgcode-core/src/run.rs index 7720d9d..b4366df 100644 --- a/crates/omgcode-core/src/run.rs +++ b/crates/omgcode-core/src/run.rs @@ -35,7 +35,7 @@ use crate::session::{NativePath, SessionError}; use crate::storage::{self, StorageError}; use crate::workspace::DirectoryRecord; -pub use omgcode_platform::interrupt::{InterruptWatch, cancel_on_interrupt}; +pub use omgcode_platform::interrupt::{InterruptWatch, cancel_on_interrupt, watch_interrupt}; pub use omgcode_platform::process::{CancelSignal, ProcessLimits, TreeLimits}; pub use omgcode_platform::process::{ MAX_CAPTURE_BYTES, MAX_DEADLINE, MAX_TREE_CPU_TIME, MAX_TREE_DISK_BYTES, MAX_TREE_MEMORY_BYTES, diff --git a/crates/omgcode-core/src/turn.rs b/crates/omgcode-core/src/turn.rs index 65655ec..943e000 100644 --- a/crates/omgcode-core/src/turn.rs +++ b/crates/omgcode-core/src/turn.rs @@ -943,6 +943,19 @@ impl<'a> TurnRuntime<'a> { self } + /// Use `cancel` as the turn's token instead of the one the runtime made. + /// + /// A caller whose cancellation must also reach work done before the + /// runtime exists, such as compaction before a turn, creates one token + /// and hands it here (ADR-0051). A token already set when [`Self::run`] + /// starts returns `Cancelled` before the turn marker, so nothing is + /// recorded. + #[must_use] + pub fn with_cancel_token(mut self, cancel: CancelToken) -> Self { + self.cancel = cancel; + self + } + /// Return a token which requests cancellation at the next checkpoint. pub fn cancel_token(&self) -> CancelToken { self.cancel.clone() diff --git a/crates/omgcode-core/tests/local_provider.rs b/crates/omgcode-core/tests/local_provider.rs index 753958d..642b27c 100644 --- a/crates/omgcode-core/tests/local_provider.rs +++ b/crates/omgcode-core/tests/local_provider.rs @@ -211,7 +211,7 @@ fn assert_prompt_rejected_before_connect(messages: &[ModelMessage]) { } }); - let result = model.complete(messages); + let result = model.complete(messages, &CancelToken::new()); let unexpected_request = join_bounded(monitor, "bounded listener monitor completes"); assert_eq!(result, Err(ProviderError::InvalidConfiguration)); @@ -229,7 +229,9 @@ fn sends_prompt_roles_without_tools_and_parses_plain_reply() { Some("local-secret-token".to_owned()), ); - let reply = model.complete(&messages()).expect("valid completion"); + let reply = model + .complete(&messages(), &CancelToken::new()) + .expect("valid completion"); let (head, body) = fixture.finish(); let request: serde_json::Value = serde_json::from_slice(&body).expect("request JSON"); @@ -269,7 +271,9 @@ fn does_not_leak_token_on_authentication_failure() { fixture.address.clone(), Some("local-secret-token".to_owned()), ); - let error = model.complete(&messages()).expect_err("401 is an error"); + let error = model + .complete(&messages(), &CancelToken::new()) + .expect_err("401 is an error"); let (head, _) = fixture.finish(); assert_eq!(error, ProviderError::HttpStatus(401)); @@ -285,7 +289,7 @@ fn does_not_leak_token_on_authentication_failure() { fn rejects_invalid_response_schemas_and_tool_requests() { let malformed = Fixture::start("200 OK", br#"{"choices":[]}"#, ""); let error = client(malformed.address.clone(), None) - .complete(&messages()) + .complete(&messages(), &CancelToken::new()) .expect_err("empty choices are invalid"); malformed.finish(); assert_eq!(error, ProviderError::MalformedResponse); @@ -296,7 +300,7 @@ fn rejects_invalid_response_schemas_and_tool_requests() { "", ); let error = client(tool_call.address.clone(), None) - .complete(&messages()) + .complete(&messages(), &CancelToken::new()) .expect_err("tool requests are unsupported"); tool_call.finish(); assert_eq!(error, ProviderError::UnsupportedToolCall); @@ -308,7 +312,7 @@ fn accepts_empty_lm_studio_tool_call_array_as_no_action() { let fixture = Fixture::start("200 OK", response, ""); let model = client(fixture.address.clone(), None); - let result = model.complete(&messages()); + let result = model.complete(&messages(), &CancelToken::new()); fixture.finish(); assert_eq!( @@ -329,7 +333,7 @@ fn accepts_empty_lm_studio_tool_call_array_as_no_action() { fn rejects_oversized_response_body() { let fixture = Fixture::start("200 OK", &vec![b'x'; 1024 * 1024 + 1], ""); let error = client(fixture.address.clone(), None) - .complete(&messages()) + .complete(&messages(), &CancelToken::new()) .expect_err("oversized response is rejected"); fixture.finish(); assert_eq!(error, ProviderError::ResponseTooLarge); @@ -373,7 +377,8 @@ fn null_action_fields_are_absent_but_actual_or_malformed_calls_fail() { target.extend(fields.as_object().expect("field object").clone()); let encoded = serde_json::to_vec(&response).expect("fixture response JSON"); let fixture = Fixture::start("200 OK", &encoded, ""); - let result = client(fixture.address.clone(), None).complete(&messages()); + let result = + client(fixture.address.clone(), None).complete(&messages(), &CancelToken::new()); fixture.finish(); assert_eq!(result.err(), expected); } @@ -388,7 +393,7 @@ fn does_not_follow_redirects() { "Location: http://127.0.0.1:9/v1/chat/completions\r\n", ); let error = client(fixture.address.clone(), None) - .complete(&messages()) + .complete(&messages(), &CancelToken::new()) .expect_err("redirect response is not followed"); fixture.finish(); assert_eq!(error, ProviderError::HttpStatus(302)); @@ -446,7 +451,7 @@ fn context_budget_fails_before_network_request() { .expect("valid limits"); assert_eq!( - model.complete(&messages()), + model.complete(&messages(), &CancelToken::new()), Err(ProviderError::ContextLimit) ); assert!( @@ -505,7 +510,7 @@ fn wire_size_limit_fails_before_network_request() { ]; assert_eq!( - model.complete(&oversized), + model.complete(&oversized, &CancelToken::new()), Err(ProviderError::RequestTooLarge) ); assert!( @@ -534,8 +539,11 @@ const TOOL_CALL_REPLY: &str = r#"{"choices":[{"message":{"role":"assistant","con /// Send one tool-enabled request to a fixture serving `body`. fn tool_exchange(body: &str) -> (Result, Value) { let fixture = Fixture::start("200 OK", body.as_bytes(), ""); - let result = - client(fixture.address.clone(), None).complete_with_tools(&messages(), &tool_definitions()); + let result = client(fixture.address.clone(), None).complete_with_tools( + &messages(), + &tool_definitions(), + &CancelToken::new(), + ); let (_, request) = fixture.finish(); ( result, @@ -749,7 +757,7 @@ fn non_streaming_requests_still_send_stream_false_and_no_accept_header() { // Given the non-streaming text and tool methods. let fixture = Fixture::start("200 OK", SUCCESS.as_bytes(), ""); client(fixture.address.clone(), None) - .complete(&messages()) + .complete(&messages(), &CancelToken::new()) .expect("reply"); // When their requests are captured. @@ -1166,7 +1174,9 @@ fn a_token_cancelled_before_the_call_makes_no_connection() { cancel.cancel(); let mut pieces = Vec::new(); - // When each streamed method is called. + // When each method, streamed or not, is called. + let plain = model.complete(&messages(), &cancel); + let plain_tools = model.complete_with_tools(&messages(), &tool_definitions(), &cancel); let text = model.complete_streaming( &messages(), &mut |piece: &str| pieces.push(piece.to_owned()), @@ -1179,7 +1189,9 @@ fn a_token_cancelled_before_the_call_makes_no_connection() { &cancel, ); - // Then both are cancelled, nothing was delivered, and nothing connected. + // Then all are cancelled, nothing was delivered, and nothing connected. + assert_eq!(plain, Err(ProviderError::Cancelled)); + assert_eq!(plain_tools.err(), Some(ProviderError::Cancelled)); assert_eq!(text, Err(ProviderError::Cancelled)); assert_eq!(tools.err(), Some(ProviderError::Cancelled)); assert!(pieces.is_empty()); @@ -1346,3 +1358,107 @@ fn a_token_that_is_never_cancelled_does_not_disturb_a_slow_stream() { assert_eq!(result.expect("a complete stream").content, "abc"); assert_eq!(pieces, ["a", "b", "c"]); } + +// ---- Cancellation of a non-streamed request (ADR-0051) ---- + +#[test] +fn cancelling_a_non_streamed_request_while_the_head_is_awaited_stops_it() { + // Given a server that accepts each request and never sends a head. + // When the token is cancelled once the server has read the request. + let text = held_call(None, None, |model, _on_text, cancel| { + model.complete(&messages(), cancel) + }); + let tools = held_call(None, None, |model, _on_text, cancel| { + model.complete_with_tools(&messages(), &tool_definitions(), cancel) + }); + + // Then each call is cancelled far inside the 30-second deadline, and its + // server saw the connection close. + assert_eq!(text.result, Err(ProviderError::Cancelled)); + assert_eq!(tools.result.err(), Some(ProviderError::Cancelled)); + for (elapsed, closed) in [(text.elapsed, text.closed), (tools.elapsed, tools.closed)] { + assert!(elapsed < CANCEL_BOUND, "cancellation took {elapsed:?}"); + assert!(closed, "the connection was closed on cancellation"); + } +} + +#[test] +fn cancelling_a_non_streamed_request_with_its_body_held_open_discards_the_part_read() { + // Given a server that sends a head announcing a longer body, the start + // of a valid reply, and then holds the connection open. + let partial = r#"{"choices":[{"message":{"role":"assistant","content":"partial"#; + let response = format!( + "HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{partial}", + SUCCESS.len() + ); + let (request_read, request_seen) = mpsc::channel::<()>(); + let (address, server) = holding_server(Some(response.into_bytes()), request_read); + let cancel = CancelToken::new(); + let canceller = { + let cancel = cancel.clone(); + thread::spawn(move || { + request_seen + .recv_timeout(Duration::from_secs(5)) + .expect("the server reads the request before the bound"); + // The head and the partial body are written right after this + // signal. Waiting here makes it likely that the client is reading + // the body when it is cancelled; the outcome asserted below is + // the same if it is still awaiting the head. + thread::sleep(Duration::from_millis(200)); + cancel.cancel(); + }) + }; + + // When the token is cancelled while the rest of the body is awaited. + let started = Instant::now(); + let result = client(address, None).complete(&messages(), &cancel); + let elapsed = started.elapsed(); + join_bounded(canceller, "canceller completes"); + let closed = join_bounded(server, "fixture completes"); + + // Then the call is cancelled promptly, nothing of the body is returned, + // and the connection was closed. + assert_eq!(result, Err(ProviderError::Cancelled)); + assert!(elapsed < CANCEL_BOUND, "cancellation took {elapsed:?}"); + assert!(closed, "the connection was closed on cancellation"); +} + +#[test] +fn a_token_that_is_never_cancelled_does_not_disturb_a_slow_non_streamed_reply() { + // Given a server that sends its head and body in pieces more than one + // poll interval apart. + let listener = TcpListener::bind("127.0.0.1:0").expect("bind local fixture"); + listener.set_nonblocking(true).expect("bounded accept"); + let address = format!("http://{}/v1", listener.local_addr().expect("address")); + let server = thread::spawn(move || { + let mut stream = accept_bounded(&listener); + stream + .set_nonblocking(false) + .expect("make accepted fixture stream blocking"); + stream + .set_read_timeout(Some(Duration::from_secs(3))) + .expect("set fixture read timeout"); + read_request(&mut stream).expect("read bounded request"); + thread::sleep(Duration::from_millis(80)); + let head = format!( + "HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n", + SUCCESS.len() + ); + stream.write_all(head.as_bytes()).expect("write head"); + stream.flush().expect("flush head"); + for piece in SUCCESS.as_bytes().chunks(SUCCESS.len().div_ceil(3)) { + thread::sleep(Duration::from_millis(80)); + stream.write_all(piece).expect("write piece"); + stream.flush().expect("flush piece"); + } + }); + + // When the request runs with a token nobody cancels. + let result = client(address, None).complete(&messages(), &CancelToken::new()); + join_bounded(server, "fixture completes"); + + // Then the polling did not restart or disturb the request. + let reply = result.expect("a complete reply"); + assert_eq!(reply.content, "A plain reply."); + assert_eq!(reply.finish_reason, "stop"); +} diff --git a/crates/omgcode-core/tests/turn_runtime.rs b/crates/omgcode-core/tests/turn_runtime.rs index a1d4cbb..28d9f26 100644 --- a/crates/omgcode-core/tests/turn_runtime.rs +++ b/crates/omgcode-core/tests/turn_runtime.rs @@ -1154,6 +1154,40 @@ fn a_cancelled_model_request_of_a_cancelled_turn_records_cancelled() { assert_eq!(fixture.last_failure_code().as_deref(), Some("cancelled")); } +#[test] +fn a_caller_token_is_the_turn_token_and_a_set_one_records_nothing() { + // Given a runtime given a caller's token, and a model that would reply. + let mut fixture = Fixture::new(); + let mut model = ScriptedModel::new(vec![text_reply("never sent")]); + let journal = fixture.journal.as_mut().expect("journal is open"); + let caller = CancelToken::new(); + let mut runtime = TurnRuntime::new( + journal, + &fixture.data, + fixture.workspace_id, + TurnLimits::default(), + ) + .expect("build runtime") + .with_cancel_token(caller.clone()); + + // When the caller cancels its token before the turn runs. + let handed = runtime.cancel_token(); + caller.cancel(); + let error = runtime + .run(&mut model, "read the file", "", manifest()) + .expect_err("a cancelled turn does not run"); + drop(runtime); + + // Then the runtime's token is the caller's, the turn ended before its + // marker, and the model was never asked. + assert!(handed.is_cancelled()); + assert!(matches!(error, TurnError::Cancelled), "{error:?}"); + assert!(model.requests.is_empty()); + assert_eq!(fixture.snapshot().active_turn, None); + fixture.close_journal(); + assert_eq!(fixture.last_failure_code(), None); +} + #[test] fn a_cancelled_model_error_without_a_cancelled_turn_is_a_model_error() { // Given a model that reports an interrupted request though nobody cancelled. diff --git a/crates/omgcode-platform/src/interrupt.rs b/crates/omgcode-platform/src/interrupt.rs index 175119c..9981758 100644 --- a/crates/omgcode-platform/src/interrupt.rs +++ b/crates/omgcode-platform/src/interrupt.rs @@ -5,6 +5,8 @@ //! handler installed here records the interrupt, and a watcher thread raises //! the run's [`CancelSignal`]. The runner then terminates the whole tree and //! reports `Cancelled`, so the caller can still record the outcome. +//! [`watch_interrupt`] calls any closure instead, so a CLI chat turn or +//! compaction can cancel its own token the same way (ADR-0051). //! //! Linux handles `SIGINT` and `SIGTERM`; Windows handles `CTRL_C_EVENT` and //! `CTRL_BREAK_EVENT`. The handler stays installed for the life of the @@ -35,8 +37,24 @@ const WATCH_INTERVAL: Duration = Duration::from_millis(20); /// started and the error is returned; the caller decides whether to run /// without interrupt handling. pub fn cancel_on_interrupt(cancel: &CancelSignal) -> io::Result { - install()?; let cancel = cancel.clone(); + watch_interrupt(move || cancel.cancel()) +} + +/// Install the interrupt handler and call `on_interrupt` once when an +/// interrupt arrives. +/// +/// This is [`cancel_on_interrupt`] for a caller whose cancellation is not a +/// [`CancelSignal`], such as a chat turn's cancel token (ADR-0051). +/// `on_interrupt` runs on the watcher thread, at most once, and must not +/// block. An interrupt that arrived before this call, while the handler was +/// already installed, is seen at once, because the recorded interrupt is +/// never cleared. The returned watch owns the thread and the closure; stop +/// it as described for [`cancel_on_interrupt`]. If the handler cannot be +/// installed or the thread cannot be started, `on_interrupt` is dropped +/// unused and the error is returned. +pub fn watch_interrupt(on_interrupt: impl FnOnce() + Send + 'static) -> io::Result { + install()?; let stop = Arc::new(AtomicBool::new(false)); let stopped = Arc::clone(&stop); let thread = thread::Builder::new() @@ -44,7 +62,7 @@ pub fn cancel_on_interrupt(cancel: &CancelSignal) -> io::Result .spawn(move || { while !stopped.load(Ordering::SeqCst) { if INTERRUPTED.load(Ordering::SeqCst) { - cancel.cancel(); + on_interrupt(); return; } thread::sleep(WATCH_INTERVAL); @@ -175,6 +193,21 @@ mod tests { assert!(cancel.is_cancelled(), "the watcher did not cancel the run"); watch.stop().expect("stop the finished watcher"); assert!(!idle.is_cancelled(), "a stopped watcher raised its signal"); + + // The recorded interrupt is never cleared, so a callback watcher + // started afterwards is called at once, exactly once. + let (called, calls) = std::sync::mpsc::channel(); + let watch = watch_interrupt(move || { + // The receiver outlives the bounded wait below; a failed send + // would show up there as a missing call. + let _ignored = called.send(()); + }) + .expect("start a callback watcher"); + calls + .recv_timeout(Duration::from_secs(5)) + .expect("the callback watcher saw the earlier interrupt"); + watch.stop().expect("stop the callback watcher"); + assert!(calls.try_recv().is_err(), "the callback ran more than once"); } #[cfg(unix)] diff --git a/docs/adr/0050-interrupt-a-streamed-model-request-on-cancellation.md b/docs/adr/0050-interrupt-a-streamed-model-request-on-cancellation.md index dcca19d..ecdc66d 100644 --- a/docs/adr/0050-interrupt-a-streamed-model-request-on-cancellation.md +++ b/docs/adr/0050-interrupt-a-streamed-model-request-on-cancellation.md @@ -1,6 +1,7 @@ # 0050: Interrupt a streamed model request when a turn is cancelled -- Status: Accepted +- Status: Accepted; amended by + [ADR-0051](0051-cancel-every-model-request-and-interrupt-cli-turns.md) - Date: 2026-11-16 ## Context diff --git a/docs/adr/0051-cancel-every-model-request-and-interrupt-cli-turns.md b/docs/adr/0051-cancel-every-model-request-and-interrupt-cli-turns.md new file mode 100644 index 0000000..aab35f4 --- /dev/null +++ b/docs/adr/0051-cancel-every-model-request-and-interrupt-cli-turns.md @@ -0,0 +1,142 @@ +# 0051: Cancel every model request and let an interrupt cancel a CLI turn + +- Status: Accepted +- Date: 2026-11-17 + +## Context + +ADR-0050 made a streamed model request watch the turn's cancel token. It left +three gaps, and named them as consequences: + +- A **non-streamed request** used the blocking HTTP client, so it could not be + interrupted. A text-only turn that did not stream had no cancel token at + all, and the desktop with `--stream off` ignored a cancel request. +- The **compaction requests** before a turn, and those of `session compact`, + had no token. A cancel during pre-turn compaction did nothing until the turn + itself started. +- The **CLI had no cancel source.** Ctrl-C ended the process, so a turn that + had started stayed open in its journal until the operator ran `session + interrupt`. + +ADR-0050 rejected moving the non-streaming requests to the async client +because nothing could cancel them and because that path had met a live +server. These constraints shape this decision: + +- **The blocking client is the async client.** reqwest's blocking client runs + the same async client on a background runtime thread. Moving a request to + the async client inside its own runtime changes how the caller waits, not + the request on the wire: the same body, headers, timeouts, redirect and + proxy settings. +- **The interrupt handler already exists.** The trusted `run` command absorbs + Ctrl-C and Ctrl-Break on Windows and `SIGINT` and `SIGTERM` on Linux, and a + watcher thread turns the recorded interrupt into a run cancellation + (ADR-0030). It accepts only the run's `CancelSignal`. +- **Compaction writes once.** Core records the compaction only after the last + summary is validated, so a compaction stopped earlier leaves the journal + unchanged. +- **The journal format must not change.** A cancelled turn already has a + defined record: a turn-failed record with the reason `cancelled` and no + assistant message. + +## Decision + +### Adapter: every request watches the token + +`complete` and `complete_with_tools` take a `CancelToken`, like the streamed +methods. All four methods share one transport. Each request builds its own +async HTTP client inside a single-thread runtime created for that request, with +the settings ADR-0050 gave the streamed request: no proxy, no redirect, no +retry, the connect timeout, and the thirty-second total deadline. The token is +checked before anything is sent, while the response head is awaited, and +before and while each piece of the body is awaited, at 25 ms intervals. A set +token drops the request, closing the connection, and returns +`ProviderError::Cancelled`. A non-streamed body read in part is discarded. The +1 MiB response bound is applied as the pieces arrive. + +The blocking client is removed, and with it reqwest's `blocking` feature. No +dependency is added. + +### Application: one token per turn, created before compaction + +Every chat turn has one cancel token. It is created after the session checks +and before the compaction check, and `ChatHooks::on_cancel_token` receives a +clone on both the text-only and the tool-enabled path, streamed or not. A +front end can supply the token itself through the new `ChatHooks::cancel`, so +that a cancellation source installed before the turn reaches it. The tool path +hands the token to the runtime through the new +`TurnRuntime::with_cancel_token`. + +The same token stops the pre-turn compaction requests. `compact_journal` takes +a token too. A cancelled compaction returns the error kind `cancelled` with the +message "Compaction was cancelled; nothing was recorded", and records nothing. + +A token already set when the turn would start ends it as `cancelled` before +the turn marker, so nothing is recorded. This was already the runtime's +behaviour, and the text-only path now matches it. A cancellation after the +turn started is recorded as before. + +### CLI: an interrupt cancels the turn or the compaction + +The platform interrupt module gains `watch_interrupt`, which runs any closure +once on an interrupt. `cancel_on_interrupt` is now a wrapper around it. +`omgcode chat` creates the turn's token, installs the watcher after the +journal is open and before the turn, and passes the token through +`ChatHooks::cancel`. `session compact` does the same around `compact_journal`. +An interrupt then stops the request in flight. The chat turn is recorded as +cancelled, and the command fails with the code `cancelled` and the session +locator. A cancelled compaction leaves the journal byte for byte unchanged. A +handler that cannot be installed fails the command with the code `interrupt` +before any model request. + +The handler stays installed until the process ends, as it does for `run`. +`capabilities` reports `cancellable_adapters: ["lm_studio"]`. + +### Desktop + +No command or wire type changes. Every turn now hands over its token before +its first model request, so `chat_cancel` and exit stop a non-streamed request +and a pre-turn compaction as they already stopped a streamed request. A cancel +that arrives before the token is acted on when it is handed over, which ends +the turn before it is recorded. + +## Alternatives considered + +- **Keep the blocking client for non-streamed requests.** It cannot be woken + while it waits, which is the case ADR-0050 was written for. +- **Add separate cancellable methods beside the old ones.** Two transports + would have to be kept equal, and a caller could still choose the one that + cannot be stopped. +- **Let the CLI receive the token through `on_cancel_token` and install the + watcher there.** The callback cannot fail the command, so a handler that + cannot be installed would be found only after the turn had started. With + the token supplied up front the failure happens before any request. +- **Exit with the conventional status 130 after an interrupt.** Every other + failure exits with status 1 and a JSON error. A caller tells a cancellation + apart by the code `cancelled`. +- **Bridge the run's `CancelSignal` to the turn's token with another thread.** + `watch_interrupt` is smaller and keeps one watcher per command. + +## Consequences + +- Every model request, streamed or not, and every compaction request can be + cancelled, and stops within about the check interval. +- Ctrl-C during `chat` or `session compact` no longer leaves an open turn. + Once the watcher is installed, a second interrupt does not end the process: + the command ends when the request has stopped. +- Still not interruptible: a read, list, or edit tool call already in + progress. A command tool call was already cancelled through its run. +- Each request now builds a client and a runtime. For a loopback server the + cost is small next to a model request. +- The wire request is unchanged, but the non-streamed path has not been run + against a live LM Studio server since this change. +- Not verified: an interrupt delivered to the CLI on Windows. The Windows + handler is unit-tested by calling it directly, and the cancellation it sets + is the one the Linux test delivers with `SIGINT`. + +## References + +- [Provider authentication](../provider-authentication.md) +- [Desktop interface](../desktop-interface.md) +- [ADR-0030](0030-run-trusted-commands-through-recorded-toolchain-profiles.md) +- [ADR-0041](0041-compact-active-context-with-readable-summaries.md) +- [ADR-0050](0050-interrupt-a-streamed-model-request-on-cancellation.md) diff --git a/docs/adr/index.md b/docs/adr/index.md index 19e973f..6a8549b 100644 --- a/docs/adr/index.md +++ b/docs/adr/index.md @@ -64,4 +64,5 @@ meaningful alternative does not need an ADR. | [0047: Read the local provider token from the credential store](0047-read-the-local-provider-token-from-the-credential-store.md) | Accepted; amended by ADR-0048 | 2026-11-16 | | [0048: Bind a stored credential to its profile's endpoint](0048-bind-a-stored-credential-to-its-profile-endpoint.md) | Accepted | 2026-11-16 | | [0049: Stream local model replies and push chat events to the main window](0049-stream-local-model-replies-and-push-chat-events.md) | Accepted | 2026-11-16 | -| [0050: Interrupt a streamed model request when a turn is cancelled](0050-interrupt-a-streamed-model-request-on-cancellation.md) | Accepted | 2026-11-16 | +| [0050: Interrupt a streamed model request when a turn is cancelled](0050-interrupt-a-streamed-model-request-on-cancellation.md) | Accepted; amended by ADR-0051 | 2026-11-16 | +| [0051: Cancel every model request and let an interrupt cancel a CLI turn](0051-cancel-every-model-request-and-interrupt-cli-turns.md) | Accepted | 2026-11-17 | diff --git a/docs/desktop-interface.md b/docs/desktop-interface.md index 414f817..499e340 100644 --- a/docs/desktop-interface.md +++ b/docs/desktop-interface.md @@ -44,12 +44,15 @@ merged to `main` at the time of writing): - Since version 0.33.0 (Phase D5, ADR-0050), cancelling a streamed turn interrupts its model request, and the turn ends as cancelled. Exit cancels every running streamed turn within the grace period. +- Since version 0.34.0 (Phase D6, ADR-0051), cancelling also interrupts a + non-streamed request (`--stream off`) and the compaction request before a + turn. No command or wire type changed. Not implemented: - Installers, signing, and any other packaging. `bundle.active` is `false`. -- Interrupting a request that is not streamed, the compaction request, or a - tool call in progress. Cancelling such a turn waits for the call to end. +- Interrupting a read, list, or edit tool call in progress. Cancelling such a + turn waits for the call to end. - A settings interface. Workspace settings change only through the trusted CLI ([workspace settings](workspace-permissions.md)). Chat settings are not saved between runs. @@ -376,25 +379,27 @@ estimate reached the compaction trigger. A context of 32,768 tokens worked. `running`. With `--stream off` no preview exists, and `finished` is still pushed. See [streamed replies](provider-authentication.md#streamed-replies-0320). -- **Cancellation.** Cooperative. A streamed turn, text-only or tool-enabled, - hands its cancel token to the service. `chat_cancel` sets it, the state - becomes `cancelling`, and the streamed model request stops within about 25 ms - (see [streamed replies](provider-authentication.md#cancellation-0330)). The - turn ends as `failed` with the error code `cancelled`, and `finished` is - pushed. The journal holds a turn-failed record with the reason `cancelled`, - no assistant message, and no open turn. The preview text is never recorded. - The window shows the plain notice "Turn cancelled." with no error styling. It - decides by the error code only. A tool-enabled turn also checks the token - between model rounds and tool calls. These are not interrupted: a request - that is not streamed (`--stream off`), the compaction request before a turn, - and a tool call already in progress. A non-streamed text-only turn has no - cancel token, so cancelling it changes nothing. The CLI has no cancel source. +- **Cancellation.** Cooperative. Every turn, text-only or tool-enabled, + streamed or not, hands its cancel token to the service before the + compaction check and its first model request (ADR-0051). `chat_cancel` sets + it, the state becomes `cancelling`, and the model or compaction request in + flight stops within about 25 ms (see + [cancellation](provider-authentication.md#cancelling-every-request-0340)). + The turn ends as `failed` with the error code `cancelled`, and `finished` is + pushed. A turn that had started leaves a turn-failed record with the reason + `cancelled`, no assistant message, and no open turn; a turn cancelled during + its compaction, or before its token was handed over, records nothing. The + preview text is never recorded. The window shows the plain notice "Turn + cancelled." with no error styling. It decides by the error code only. A + tool-enabled turn also checks the token between model rounds and tool + calls. A read, list, or edit tool call already in progress is not + interrupted. - **Archived or open sessions.** A chat request for an archived session fails with `archived`, and for a session with an open turn with `turn_open`. - **Exit.** Closing `main` asks every running turn to cancel and waits up to - five seconds. A streamed turn ends within that period and leaves no open - turn. A turn that cannot be interrupted, one waiting on a non-streamed - request, the compaction request, or a tool call, is abandoned if it is still + five seconds. A turn waiting on a model or compaction request ends within + that period and leaves no open turn. A turn in a step that cannot be + interrupted, a read, list, or edit tool call, is abandoned if it is still running when the period ends. Its journal keeps an open turn. The tree shows it as `turn_open`, and the operator can interrupt it. @@ -578,7 +583,7 @@ What it does not do: | Settings store retry | 2 seconds at 20 ms | | Index busy timeout | 2 seconds | | Shutdown grace | 5 seconds | -| Streamed request cancel check interval | 25 ms | +| Model request cancel check interval | 25 ms | | Approve button delay | 1 second | | Poll and check intervals | chat reconcile 5 seconds, chat fallback poll 750 ms, approvals 3 seconds | @@ -645,7 +650,8 @@ private store refuses, see the notes in - **Service.** Tests with a mock provider cover chat, approvals, review tokens, shutdown, and the index. `crates/omgcode-app/tests/desktop_events.rs` covers the event hub. The desktop chat and event tests also cover cancelling - a streamed turn and shutting down while one runs. + a streamed turn and shutting down while one runs, and cancelling a turn + whose request is not streamed. - **End to end.** The real Windows shell was driven over the WebView2 remote debugging port with real mouse and text input against a mock local model server, on 2026-11-16. A second run on the same day drove streaming and the @@ -673,18 +679,19 @@ private store refuses, see the notes in - Two tests that need file symbolic links print `SKIPPED link assertions` and return on a host without the privilege to create them. They did not run on the Windows development host; they ran on Linux. -- Not exercised: shutdown while a non-streamed provider request is in flight; - a live call that reaches the Rust `authorize` check. +- Not exercised: a live call that reaches the Rust `authorize` check. The + real shell's windows were not driven again for 0.34.0; the service tests + cover cancelling a non-streamed turn. - The CI workflow changes for the shell have not run. - The approvals polls (3 seconds) add a small constant IPC load while a window is visible, and the session tree does not update by push. - Streaming: the stream rules were checked against local fixtures only, and no live LM Studio server was used. No window has been opened on Linux, so the channel is unverified there. A streamed text reply has no usage counters - unless the server sends them. Cancellation interrupts only a streamed - request. It does not interrupt a non-streamed request, the compaction - request, or a tool call in progress, and the cancel path was checked against - local fixtures only, with no live server. The thirty-second request deadline + unless the server sends them. Cancellation interrupts every model and + compaction request but not a read, list, or edit tool call in progress, and + the cancel path was checked against local fixtures only, with no live + server. The thirty-second request deadline covers the whole response, so a long generation fails as before. While a turn runs, the transcript is not reloaded, so the message just sent is not shown above the live entry until the turn settles. diff --git a/docs/development.md b/docs/development.md index 42d4a16..4b211dc 100644 --- a/docs/development.md +++ b/docs/development.md @@ -1,6 +1,6 @@ # Build and run the core prototype -OMGCode 0.33.0 is a developer CLI prototype with workspace permission +OMGCode 0.34.0 is a developer CLI prototype with workspace permission settings and bounded native file operations through trusted operator commands. It can send a text conversation to an explicitly selected local LM Studio model and save that conversation. It can read/list files, exclusively create new @@ -582,6 +582,17 @@ further writes and does not fail the turn. `session compact` has no `--stream` flag, and compaction requests are never streamed. See [streamed replies](provider-authentication.md#streamed-replies-0320). +Since 0.34.0 an interrupt cancels `chat` and `session compact` (ADR-0051). Once +the journal is open, Ctrl-C or Ctrl-Break on Windows and `SIGINT` or `SIGTERM` +on Linux set the command's cancel token instead of ending the process. The +model or summarizer request in flight stops within about 25 ms, streamed or +not. The command then fails with the code `cancelled`: a chat turn that had +started is recorded as cancelled with no assistant message and no open turn, +and a cancelled compaction leaves the journal unchanged. A read, list, or edit +tool call already running finishes first. Before the journal is open, an +interrupt ends the process as before. A handler that cannot be installed fails +the command with the code `interrupt` before any model request. + ```text omgcode --data-dir C:\omgcode-data chat --stream --journal --model google/gemma-4-e4b --context-tokens 131072 "Summarise README.md." ``` diff --git a/docs/provider-authentication.md b/docs/provider-authentication.md index 2441b62..295ed08 100644 --- a/docs/provider-authentication.md +++ b/docs/provider-authentication.md @@ -368,8 +368,8 @@ unchanged and remain the default. LM Studio server was used. A streamed text reply has no usage counters unless the server sends them. A long generation still fails at the thirty-second deadline. Preview text can differ from the recorded reply. Since 0.33.0 a - cancelled turn interrupts a streamed request, as described below; a - non-streamed request still cannot be interrupted. + cancelled turn interrupts a streamed request, and since 0.34.0 any request, + as described below. ### Cancellation (0.33.0) @@ -400,12 +400,49 @@ are unchanged. runtime records a model error. A streamed text-only turn creates a token and ends as cancelled in the same way, including when the cancellation is seen after the reply arrived and before it is recorded. -- **Not interruptible.** The non-streaming methods keep the blocking client. A - non-streamed request, the compaction request before a turn, and a tool call - in progress are not interrupted. The CLI has no cancel source. +- **Not interruptible in 0.33.0.** The non-streaming methods kept the blocking + client. A non-streamed request, the compaction request before a turn, and a + tool call in progress were not interrupted, and the CLI had no cancel source. + Version 0.34.0 closes the first, second, and fourth gaps, below. - **Limits.** The cancel path was checked against local fixtures only. No live LM Studio server was used. +### Cancelling every request (0.34.0) + +[ADR-0051](adr/0051-cancel-every-model-request-and-interrupt-cli-turns.md) +applies the cancel rule above to every request. + +- **Transport.** `complete` and `complete_with_tools` take a `CancelToken` too. + All four methods share one transport: each request builds its own async + client inside a single-thread runtime created for that request, with the + settings and the deadline above. The blocking client and reqwest's `blocking` + feature are removed. The wire request is unchanged: reqwest's blocking client + was the same async client driven from a background thread. A non-streamed + body is read piece by piece within the 1 MiB bound; a body read in part when + the token is set is discarded. +- **One token per turn.** Every chat turn creates one token after the session + checks and before the compaction check, hands a clone to + `ChatHooks::on_cancel_token` on every path, and uses it for the pre-turn + compaction requests and for every model request. A front end can supply the + token through `ChatHooks::cancel`. A token already set when the turn would + start ends it as `cancelled` before the turn marker, recording nothing. +- **Compaction.** `compact_journal` takes a token. A cancelled compaction + returns the error kind `cancelled` ("Compaction was cancelled; nothing was + recorded"). The journal is written only after the last summary, so it is + unchanged. +- **CLI.** `omgcode chat` and `session compact` install the interrupt handler + of the trusted `run` command once the journal is open. Ctrl-C or Ctrl-Break + on Windows and `SIGINT` or `SIGTERM` on Linux then set the token instead of + ending the process. The command fails with the code `cancelled`; a chat turn + that had started is recorded as cancelled and the error carries the session + locator. A handler that cannot be installed fails the command with the code + `interrupt` before any model request. `capabilities` reports + `cancellable_adapters: ["lm_studio"]`. +- **Still not interruptible.** A read, list, or edit tool call already in + progress. A command tool call is cancelled through its run. +- **Limits.** Checked against local fixtures only. No live LM Studio server + was used, and the CLI interrupt was delivered by a test only on Linux. + LM Studio documents the request fields in [Chat Completions](https://lmstudio.ai/docs/developer/openai-compat/chat-completions) and tool calls arriving in pieces in diff --git a/docs/session-storage.md b/docs/session-storage.md index 9c0d307..172337e 100644 --- a/docs/session-storage.md +++ b/docs/session-storage.md @@ -215,9 +215,12 @@ has no streaming, artifact, index, compaction, migration, or memory events. Streamed model text (ADR-0049) is provisional and is never written to the journal: the recorded reply is the validated whole reply, redacted as a whole, and the journal does not say whether a reply was streamed. A turn cancelled -during a streamed request (ADR-0050) records a `turn.failed` event with the -code `cancelled` and no assistant message; its partial text is not recorded, -and no record changes shape. The CLI redacts the currently configured LM Studio token from +during a model request (ADR-0050, ADR-0051), streamed or not, records a +`turn.failed` event with the code `cancelled` and no assistant message; its +partial text is not recorded, and no record changes shape. A turn cancelled +before its `turn.started` event, including during the compaction before it, +records nothing, and a cancelled compaction writes no `context.compacted` +event. The CLI redacts the currently configured LM Studio token from new conversation text, preferences, titles, and replies before retention and output. It redacts that same token from historical context before inference. There is no general secret detector or automatic cleanup of older stored text. diff --git a/docs/testing-standards.md b/docs/testing-standards.md index bbe1f11..8b77b4d 100644 --- a/docs/testing-standards.md +++ b/docs/testing-standards.md @@ -216,7 +216,15 @@ the response head, and during the body; the stream parser under arbitrary chunking; the turn runtime's cancelled arm; the cancellation of streamed text-only and tool-enabled turns; the desktop cancel and shutdown cases; and the frontend notice, which is decided by the error code. They used the same kind of -local fixture. The adapter requires the +local fixture. The 0.34.0 fixtures (ADR-0051) add a cancel of a non-streamed +text and tool request before the request, before the response head, and with a +body held open; a slow non-streamed reply that a live token does not disturb; +the runtime's use of a caller's token; the cancellation of non-streamed text +and tool turns, of the compaction before a turn, and of `compact_journal`; a +token set before a turn; the desktop cancel of a non-streamed turn and the +shutdown grace with a turn held in an event sink; and, on Linux only, `SIGINT` +delivered to the CLI during `chat` and `session compact`. The platform +interrupt test also covers `watch_interrupt`. The adapter requires the compiled base prompt and accepts only a literal loopback HTTP endpoint. The adapter is not a permission boundary: the CLI selects tool mode from the live `model_tools` preference, and the runtime rechecks that preference before each @@ -232,8 +240,9 @@ own. It passed on Windows 11 build 26300 and WSL 6.6 (Ubuntu 24.04) on use a mock that returns the documented OpenAI-compatible tool-call shape; they do not establish live LM Studio tool-call compatibility. A single opt-in spot-check with LM Studio 0.4.28 and `google/gemma-4-e4b` on Windows is recorded in the implementation plan. Model discovery, streaming -against a live server, cancellation against a live server, interrupting a -non-streamed or compaction request, and live model compatibility remain future qualification work. The CLI can inspect +against a live server, cancellation against a live server, an interrupt +delivered to the CLI on Windows, and live model compatibility remain future +qualification work. The CLI can inspect and archive sessions, but it does not execute recorded tool actions or resume an incomplete turn. From c7d5a536e017c196f13a27b3db9601a28bc55ac4 Mon Sep 17 00:00:00 2001 From: James Chapman Date: Tue, 17 Nov 2026 09:45:00 +0000 Subject: [PATCH 2/2] fix: address review of the D6 cancellation change - Accept Cancelling or Failed for the status read right after chat_cancel: a non-streamed request now stops within about 25 ms, so the turn may already have ended. The terminal cancelled checks stay. - Replace the sleep-ordered body cancellation test with a unit test of the new read_whole_body helper. It cancels only after the response head has arrived and the reader is shown to be waiting for the held body. - Say that a cancelled turn records no final assistant reply, not no assistant message: messages recorded with an earlier round's tool calls stay. Note that a compaction finished before the cancel stays recorded. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 3 +- crates/omgcode-app/tests/desktop_chat.rs | 19 +- crates/omgcode-core/src/provider.rs | 291 +++++++++++++----- crates/omgcode-core/tests/local_provider.rs | 41 --- ...y-model-request-and-interrupt-cli-turns.md | 6 +- docs/desktop-interface.md | 6 +- docs/development.md | 2 +- docs/session-storage.md | 12 +- docs/testing-standards.md | 5 +- 9 files changed, 251 insertions(+), 134 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 73b8128..d48dfee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -49,7 +49,8 @@ Changes are recorded here before a release version is assigned. stop a non-streamed request (`--stream off`) and a pre-turn compaction. A turn cancelled before it handed over its token ends before it is recorded. - **Tests:** core fixtures for a cancel of a non-streamed text and tool - request before sending, before the head, and with a body held open, and a + request before sending and before the head, a unit test that cancels + while the rest of a held body is awaited once the head has arrived, and a slow non-streamed reply under a live token; the runtime's caller token; app tests for cancelled non-streamed text and tool turns, a cancelled pre-turn compaction, a cancelled `compact_journal`, and a token set before the turn; diff --git a/crates/omgcode-app/tests/desktop_chat.rs b/crates/omgcode-app/tests/desktop_chat.rs index d012a61..7de352b 100644 --- a/crates/omgcode-app/tests/desktop_chat.rs +++ b/crates/omgcode-app/tests/desktop_chat.rs @@ -393,6 +393,16 @@ fn the_server_token_never_reaches_a_response_even_when_the_model_echoes_it() { assert_eq!(server.finish(), 1); } +/// A cancel was taken by a running turn. The request in flight stops within +/// about 25 ms (ADR-0051), so the turn may already have ended as failed by the +/// time its status is read; the caller checks the terminal error separately. +fn assert_cancel_accepted(state: ChatState) { + assert!( + matches!(state, ChatState::Cancelling | ChatState::Failed), + "a cancel left the turn {state:?}" + ); +} + #[test] fn cancelling_a_tool_turn_reports_cancelling_then_fails_with_the_core_error() { let (release, hold) = gate(); @@ -415,9 +425,10 @@ fn cancelling_a_tool_turn_reports_cancelling_then_fails_with_the_core_error() { let cancelling = service .chat_cancel(wire(session_request(session))) .expect("cancel"); - assert_eq!(cancelling.state, ChatState::Cancelling); - assert_eq!(status(service, session).state, ChatState::Cancelling); - release.send(()).expect("release the model reply"); + assert_cancel_accepted(cancelling.state); + assert_cancel_accepted(status(service, session).state); + // The turn may already have ended, so nobody may be receiving. + let _ignored = release.send(()); let done = wait_finished(service, session); assert_eq!(done.state, ChatState::Failed); @@ -462,7 +473,7 @@ fn cancelling_a_non_streamed_text_turn_stops_its_request() { // Then the turn has a token to act on, its request stops without the // reply, and the cancellation is recorded (ADR-0051). - assert_eq!(after.state, ChatState::Cancelling); + assert_cancel_accepted(after.state); let done = wait_finished(service, session); assert!( begun.elapsed() < Duration::from_secs(5), diff --git a/crates/omgcode-core/src/provider.rs b/crates/omgcode-core/src/provider.rs index 55c8fb6..478b3e4 100644 --- a/crates/omgcode-core/src/provider.rs +++ b/crates/omgcode-core/src/provider.rs @@ -499,42 +499,16 @@ impl LocalModelClient { /// Send a non-streaming request and read its whole body within 1 MiB. /// - /// The token is checked before each piece of the body as well as while - /// each is awaited. A non-success status is refused before the body is - /// read. Dropping the response or the pending request on any return - /// closes the connection. + /// Dropping the response or the pending request on any return closes the + /// connection. async fn whole_exchange( &self, body: Vec, cancel: &CancelToken, ) -> Result, ProviderError> { let client = self.http_client()?; - let mut response = self.send(&client, body, false, cancel).await?; - let status = response.status(); - if !status.is_success() { - return Err(ProviderError::HttpStatus(status.as_u16())); - } - let mut bytes = Vec::new(); - loop { - if cancel.is_cancelled() { - return Err(ProviderError::Cancelled); - } - let chunk = until_cancelled(response.chunk(), cancel) - .await? - .map_err(|_| ProviderError::Transport)?; - let Some(chunk) = chunk else { - return Ok(bytes); - }; - let total = bytes - .len() - .checked_add(chunk.len()) - .and_then(|total| u64::try_from(total).ok()) - .ok_or(ProviderError::ResponseTooLarge)?; - if total > MAX_RESPONSE_BYTES { - return Err(ProviderError::ResponseTooLarge); - } - bytes.extend_from_slice(&chunk); - } + let response = self.send(&client, body, false, cancel).await?; + read_whole_body(response, cancel).await } /// Send a streaming request and feed the response body to the stream @@ -576,6 +550,43 @@ impl LocalModelClient { } } +/// Read the whole body of a non-streaming response within 1 MiB. +/// +/// A non-success status is refused before the body is read. The token is +/// checked before each piece of the body as well as while each is awaited; a +/// cancelled read discards what it read. Dropping the response on any return +/// closes the connection. +async fn read_whole_body( + mut response: Response, + cancel: &CancelToken, +) -> Result, ProviderError> { + let status = response.status(); + if !status.is_success() { + return Err(ProviderError::HttpStatus(status.as_u16())); + } + let mut bytes = Vec::new(); + loop { + if cancel.is_cancelled() { + return Err(ProviderError::Cancelled); + } + let chunk = until_cancelled(response.chunk(), cancel) + .await? + .map_err(|_| ProviderError::Transport)?; + let Some(chunk) = chunk else { + return Ok(bytes); + }; + let total = bytes + .len() + .checked_add(chunk.len()) + .and_then(|total| u64::try_from(total).ok()) + .ok_or(ProviderError::ResponseTooLarge)?; + if total > MAX_RESPONSE_BYTES { + return Err(ProviderError::ResponseTooLarge); + } + bytes.extend_from_slice(&chunk); + } +} + /// Await `future` while watching `cancel` every [`CANCEL_POLL`]. /// /// The future is pinned and polled through a timeout on `&mut`, so an elapsed @@ -1081,6 +1092,57 @@ mod tests { ); } + /// Accept one connection within `end`, read one whole bounded request, + /// and return the stream with 250 ms read and write deadlines. + fn accept_fixture_request(listener: &TcpListener, end: Instant) -> std::net::TcpStream { + let mut stream = loop { + match listener.accept() { + Ok((stream, _)) => break stream, + Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => { + assert!(Instant::now() < end, "fixture accept deadline"); + thread::sleep(Duration::from_millis(2)); + } + Err(error) => panic!("fixture accept: {error}"), + } + }; + stream + .set_nonblocking(false) + .expect("blocking accepted stream"); + stream + .set_read_timeout(Some(Duration::from_millis(250))) + .expect("read deadline"); + stream + .set_write_timeout(Some(Duration::from_millis(250))) + .expect("write deadline"); + let mut bytes = Vec::new(); + let mut buffer = [0_u8; 8192]; + loop { + assert!( + bytes.len() <= 256 * 1024 && Instant::now() < end, + "bounded request" + ); + if let Some(header_end) = bytes.windows(4).position(|window| window == b"\r\n\r\n") { + let headers = std::str::from_utf8(&bytes[..header_end]).expect("ASCII headers"); + let length = headers + .lines() + .find_map(|line| { + let (name, value) = line.split_once(':')?; + name.eq_ignore_ascii_case("content-length") + .then(|| value.trim().parse::().expect("body length")) + }) + .expect("Content-Length"); + assert!(length <= 256 * 1024, "bounded declared request"); + if bytes.len() >= header_end + 4 + length { + break; + } + } + let count = stream.read(&mut buffer).expect("request bytes"); + assert!(count > 0, "request must be complete"); + bytes.extend_from_slice(&buffer[..count]); + } + stream + } + /// Give the server its own deadline and cancellation path, independent of HTTP timeouts. fn slow_response( drip_body: bool, @@ -1092,52 +1154,7 @@ mod tests { let (cancel, cancelled) = mpsc::channel(); let task = thread::spawn(move || { let end = Instant::now() + Duration::from_secs(2); - let mut stream = loop { - match listener.accept() { - Ok((stream, _)) => break stream, - Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => { - assert!(Instant::now() < end, "fixture accept deadline"); - thread::sleep(Duration::from_millis(2)); - } - Err(error) => panic!("fixture accept: {error}"), - } - }; - stream - .set_nonblocking(false) - .expect("blocking accepted stream"); - stream - .set_read_timeout(Some(Duration::from_millis(250))) - .expect("read deadline"); - stream - .set_write_timeout(Some(Duration::from_millis(250))) - .expect("write deadline"); - let mut bytes = Vec::new(); - let mut buffer = [0_u8; 8192]; - loop { - assert!( - bytes.len() <= 256 * 1024 && Instant::now() < end, - "bounded request" - ); - if let Some(header_end) = bytes.windows(4).position(|window| window == b"\r\n\r\n") - { - let headers = std::str::from_utf8(&bytes[..header_end]).expect("ASCII headers"); - let length = headers - .lines() - .find_map(|line| { - let (name, value) = line.split_once(':')?; - name.eq_ignore_ascii_case("content-length") - .then(|| value.trim().parse::().expect("body length")) - }) - .expect("Content-Length"); - assert!(length <= 256 * 1024, "bounded declared request"); - if bytes.len() >= header_end + 4 + length { - break; - } - } - let count = stream.read(&mut buffer).expect("request bytes"); - assert!(count > 0, "request must be complete"); - bytes.extend_from_slice(&buffer[..count]); - } + let mut stream = accept_fixture_request(&listener, end); if drip_body { let body: &[u8] = if streamed { b"data: {\"choices\":[{\"delta\":{\"content\":\"completed slowly\"},\"finish_reason\":\"stop\"}]}\n\ndata: [DONE]\n\n" @@ -1248,4 +1265,126 @@ mod tests { fn request_deadline_bounds_a_dripping_streamed_body() { assert_bounded_transport_failure(true, true); } + + /// A server that answers with a head announcing a longer body and the + /// first part of that body, signals once both are written, and then holds + /// the connection open. The task returns whether the client closed it. + fn partial_body_server() -> ( + LocalModelClient, + mpsc::Receiver<()>, + thread::JoinHandle, + ) { + let listener = TcpListener::bind("127.0.0.1:0").expect("private fixture"); + listener.set_nonblocking(true).expect("bounded accept"); + let address = listener.local_addr().expect("address"); + let (sent, partial_sent) = mpsc::channel(); + let task = thread::spawn(move || { + let end = Instant::now() + Duration::from_secs(2); + let mut stream = accept_fixture_request(&listener, end); + let partial = br#"{"choices":[{"message":{"role":"assistant","content":"partial"#; + let head = format!( + "HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: {} +Connection: close + +", + partial.len() * 4 + ); + stream.write_all(head.as_bytes()).expect("write head"); + stream.write_all(partial).expect("write first part"); + stream.flush().expect("flush first part"); + // The test may already have failed and stopped listening. + let _ignored = sent.send(()); + stream + .set_read_timeout(Some(Duration::from_millis(25))) + .expect("poll deadline"); + let mut buffer = [0_u8; 64]; + loop { + match stream.read(&mut buffer) { + Ok(0) => return true, + Ok(_) => {} + Err(error) + if matches!( + error.kind(), + std::io::ErrorKind::WouldBlock | std::io::ErrorKind::TimedOut + ) => + { + if Instant::now() >= end { + return false; + } + } + Err(_) => return true, + } + } + }); + let client = LocalModelClient::new( + LocalModelConfig { + base_url: format!("http://{address}/v1"), + model: "fixture".to_owned(), + context_tokens: 131_072, + max_output_tokens: 32, + }, + None, + ) + .expect("client"); + (client, partial_sent, task) + } + + /// The body reader is reached only once the head has arrived, so the + /// cancel below is ordered after the head without any fixed sleep + /// (ADR-0051). + #[test] + fn cancelling_while_the_rest_of_a_body_is_awaited_discards_the_part_read() { + // Given a response whose head has arrived and whose body is held open + // after its first part, with the reader waiting for the rest. + let (client, partial_sent, task) = partial_body_server(); + let cancel = CancelToken::new(); + let runtime = tokio::runtime::Builder::new_current_thread() + .enable_io() + .enable_time() + .build() + .expect("request runtime"); + let (result, elapsed) = runtime.block_on(async { + let http = client.http_client().expect("request client"); + let response = client + .send(&http, b"{}".to_vec(), false, &cancel) + .await + .expect("the head arrives"); + partial_sent + .recv_timeout(Duration::from_secs(2)) + .expect("the server wrote the first part of the body"); + let mut read = std::pin::pin!(read_whole_body(response, &cancel)); + assert!( + tokio::time::timeout(Duration::from_millis(100), read.as_mut()) + .await + .is_err(), + "the reader is still waiting for the held body" + ); + + // When the token is set. + let started = Instant::now(); + cancel.cancel(); + let result = read.await; + (result, started.elapsed()) + }); + drop(runtime); + + // Then the read is cancelled promptly, nothing of the body is + // returned, and the connection was closed. + assert_eq!(result, Err(ProviderError::Cancelled)); + assert!( + elapsed < Duration::from_secs(1), + "cancellation took {elapsed:?}" + ); + let end = Instant::now() + Duration::from_secs(3); + while !task.is_finished() { + assert!(Instant::now() < end, "fixture cleanup deadline"); + thread::sleep(Duration::from_millis(2)); + } + assert!( + task.join().expect("fixture thread"), + "the connection was closed" + ); + } } diff --git a/crates/omgcode-core/tests/local_provider.rs b/crates/omgcode-core/tests/local_provider.rs index 642b27c..2cfb08b 100644 --- a/crates/omgcode-core/tests/local_provider.rs +++ b/crates/omgcode-core/tests/local_provider.rs @@ -1382,47 +1382,6 @@ fn cancelling_a_non_streamed_request_while_the_head_is_awaited_stops_it() { } } -#[test] -fn cancelling_a_non_streamed_request_with_its_body_held_open_discards_the_part_read() { - // Given a server that sends a head announcing a longer body, the start - // of a valid reply, and then holds the connection open. - let partial = r#"{"choices":[{"message":{"role":"assistant","content":"partial"#; - let response = format!( - "HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{partial}", - SUCCESS.len() - ); - let (request_read, request_seen) = mpsc::channel::<()>(); - let (address, server) = holding_server(Some(response.into_bytes()), request_read); - let cancel = CancelToken::new(); - let canceller = { - let cancel = cancel.clone(); - thread::spawn(move || { - request_seen - .recv_timeout(Duration::from_secs(5)) - .expect("the server reads the request before the bound"); - // The head and the partial body are written right after this - // signal. Waiting here makes it likely that the client is reading - // the body when it is cancelled; the outcome asserted below is - // the same if it is still awaiting the head. - thread::sleep(Duration::from_millis(200)); - cancel.cancel(); - }) - }; - - // When the token is cancelled while the rest of the body is awaited. - let started = Instant::now(); - let result = client(address, None).complete(&messages(), &cancel); - let elapsed = started.elapsed(); - join_bounded(canceller, "canceller completes"); - let closed = join_bounded(server, "fixture completes"); - - // Then the call is cancelled promptly, nothing of the body is returned, - // and the connection was closed. - assert_eq!(result, Err(ProviderError::Cancelled)); - assert!(elapsed < CANCEL_BOUND, "cancellation took {elapsed:?}"); - assert!(closed, "the connection was closed on cancellation"); -} - #[test] fn a_token_that_is_never_cancelled_does_not_disturb_a_slow_non_streamed_reply() { // Given a server that sends its head and body in pieces more than one diff --git a/docs/adr/0051-cancel-every-model-request-and-interrupt-cli-turns.md b/docs/adr/0051-cancel-every-model-request-and-interrupt-cli-turns.md index aab35f4..d6a6650 100644 --- a/docs/adr/0051-cancel-every-model-request-and-interrupt-cli-turns.md +++ b/docs/adr/0051-cancel-every-model-request-and-interrupt-cli-turns.md @@ -36,7 +36,8 @@ server. These constraints shape this decision: unchanged. - **The journal format must not change.** A cancelled turn already has a defined record: a turn-failed record with the reason `cancelled` and no - assistant message. + final assistant reply. An assistant message recorded with the tool calls of + an earlier round stays in the journal. ## Decision @@ -71,7 +72,8 @@ a token too. A cancelled compaction returns the error kind `cancelled` with the message "Compaction was cancelled; nothing was recorded", and records nothing. A token already set when the turn would start ends it as `cancelled` before -the turn marker, so nothing is recorded. This was already the runtime's +the turn marker, so the turn records nothing. A compaction that finished +before the cancellation stays recorded. This was already the runtime's behaviour, and the text-only path now matches it. A cancellation after the turn started is recorded as before. diff --git a/docs/desktop-interface.md b/docs/desktop-interface.md index 499e340..e9ebbcb 100644 --- a/docs/desktop-interface.md +++ b/docs/desktop-interface.md @@ -387,8 +387,10 @@ estimate reached the compaction trigger. A context of 32,768 tokens worked. [cancellation](provider-authentication.md#cancelling-every-request-0340)). The turn ends as `failed` with the error code `cancelled`, and `finished` is pushed. A turn that had started leaves a turn-failed record with the reason - `cancelled`, no assistant message, and no open turn; a turn cancelled during - its compaction, or before its token was handed over, records nothing. The + `cancelled`, no final assistant reply, and no open turn; an assistant + message recorded with an earlier round's tool calls stays. A turn cancelled + during its compaction, or before its token was handed over, records nothing; + a compaction that had already finished stays recorded. The preview text is never recorded. The window shows the plain notice "Turn cancelled." with no error styling. It decides by the error code only. A tool-enabled turn also checks the token between model rounds and tool diff --git a/docs/development.md b/docs/development.md index 4b211dc..f919314 100644 --- a/docs/development.md +++ b/docs/development.md @@ -587,7 +587,7 @@ the journal is open, Ctrl-C or Ctrl-Break on Windows and `SIGINT` or `SIGTERM` on Linux set the command's cancel token instead of ending the process. The model or summarizer request in flight stops within about 25 ms, streamed or not. The command then fails with the code `cancelled`: a chat turn that had -started is recorded as cancelled with no assistant message and no open turn, +started is recorded as cancelled with no final assistant reply and no open turn, and a cancelled compaction leaves the journal unchanged. A read, list, or edit tool call already running finishes first. Before the journal is open, an interrupt ends the process as before. A handler that cannot be installed fails diff --git a/docs/session-storage.md b/docs/session-storage.md index 172337e..d0349ca 100644 --- a/docs/session-storage.md +++ b/docs/session-storage.md @@ -216,11 +216,13 @@ Streamed model text (ADR-0049) is provisional and is never written to the journal: the recorded reply is the validated whole reply, redacted as a whole, and the journal does not say whether a reply was streamed. A turn cancelled during a model request (ADR-0050, ADR-0051), streamed or not, records a -`turn.failed` event with the code `cancelled` and no assistant message; its -partial text is not recorded, and no record changes shape. A turn cancelled -before its `turn.started` event, including during the compaction before it, -records nothing, and a cancelled compaction writes no `context.compacted` -event. The CLI redacts the currently configured LM Studio token from +`turn.failed` event with the code `cancelled` and no final assistant reply; +its partial text is not recorded, and no record changes shape. An assistant +message recorded with the tool calls of an earlier round of the same turn +stays. A turn cancelled before its `turn.started` event records no turn +event, and a cancelled compaction writes no `context.compacted` event. A +compaction that finished before the cancellation keeps its +`context.compacted` event. The CLI redacts the currently configured LM Studio token from new conversation text, preferences, titles, and replies before retention and output. It redacts that same token from historical context before inference. There is no general secret detector or automatic cleanup of older stored text. diff --git a/docs/testing-standards.md b/docs/testing-standards.md index 8b77b4d..ed4e4bf 100644 --- a/docs/testing-standards.md +++ b/docs/testing-standards.md @@ -217,8 +217,9 @@ chunking; the turn runtime's cancelled arm; the cancellation of streamed text-only and tool-enabled turns; the desktop cancel and shutdown cases; and the frontend notice, which is decided by the error code. They used the same kind of local fixture. The 0.34.0 fixtures (ADR-0051) add a cancel of a non-streamed -text and tool request before the request, before the response head, and with a -body held open; a slow non-streamed reply that a live token does not disturb; +text and tool request before the request and before the response head; a +unit test, ordered after the head arrived, that cancels while the rest of a +held body is awaited; a slow non-streamed reply that a live token does not disturb; the runtime's use of a caller's token; the cancellation of non-streamed text and tool turns, of the compaction before a turn, and of `compact_journal`; a token set before a turn; the desktop cancel of a non-streamed turn and the