Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -888,6 +888,20 @@ Changes are recorded here before a release version is assigned.

### Fixed

- The six cancellation tests in `crates/omgcode-app/tests/chat_turn.rs` now
time the interval from the moment the token is cancelled until the turn or
compaction returns, as the desktop cancellation tests already did. They
timed the whole turn, which also covers reading settings and the synced
journal appends made before the request. On a slow Windows runner,
`a_streamed_text_turn_cancelled_mid_stream_is_recorded_as_cancelled`
failed with "cancellation took 5.9s". In the same job, two `omgcode-app`
unit tests that only open a service on a temporary directory took 4 to 5
seconds each, against milliseconds on other runs. Local timing probes put
the cancel-to-return step under 1 ms, also under full CPU load and under
disk-flush contention. A 6-second delay before the response head
reproduced the failure with the old timing and passes with the new one. A
slowed-down cancellation (a 6-second poll) still fails the new check. The
5-second bound and the adapter are unchanged.
- Findings from the first Sonar analysis of the desktop frontend:
- Both workflows now install the frontend's dependencies with
`npm ci --ignore-scripts`. `ui/.npmrc` already set this; the flag makes
Expand Down
75 changes: 50 additions & 25 deletions crates/omgcode-app/tests/chat_turn.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
//! an ephemeral port. No process is spawned on Unix; on Windows the private
//! data root is prepared with `icacls`, which holds no store handle.

use std::cell::RefCell;
use std::cell::{Cell, RefCell};
use std::fs;
use std::io::{Read, Write};
use std::net::{TcpListener, TcpStream};
Expand Down Expand Up @@ -876,15 +876,23 @@ fn a_turn_without_a_stream_hook_sends_stream_false() {

// ---- Cancelling a streamed turn (ADR-0050) ----

/// The slowest acceptable cancellation. The request deadline is 30 seconds.
/// The slowest acceptable cancellation, timed from the moment the token is
/// cancelled until the turn or compaction returns. The request deadline is 30
/// seconds.
///
/// The interval starts at the cancel, as in the desktop cancellation tests.
/// Before it, a turn reads its settings and makes several synced journal
/// appends. Their duration depends on the host's storage, not on cancellation:
/// on a loaded Windows runner they once took 5.9 seconds.
const CANCEL_BOUND: Duration = Duration::from_secs(5);

/// What a streamed turn did when its token was cancelled by the first piece.
struct CancelledTurn {
error: omgcode_app::AppError,
log: Vec<String>,
handed: usize,
elapsed: Duration,
/// From the stream hook's cancel until the turn returned.
after_cancel: Duration,
served: usize,
has_active_turn: bool,
events: Vec<Value>,
Expand All @@ -903,6 +911,7 @@ fn cancel_after_the_first_piece(tools_enabled: bool) -> CancelledTurn {
common::sse_text(&["second-piece "]),
)]);
let slot: RefCell<Option<CancelToken>> = RefCell::new(None);
let cancelled_at: Cell<Option<Instant>> = Cell::new(None);
let mut handed = 0_usize;
let mut log = Vec::new();
let mut on_cancel_token = |token: CancelToken| {
Expand All @@ -914,12 +923,15 @@ fn cancel_after_the_first_piece(tools_enabled: bool) -> CancelledTurn {
ChatStreamEvent::Text(text) => {
log.push(format!("text:{text}"));
if let Some(token) = slot.borrow().as_ref() {
// The first cancel starts the measured interval.
if cancelled_at.get().is_none() {
cancelled_at.set(Some(Instant::now()));
}
token.cancel();
}
}
};

let started = Instant::now();
let result = turn(
&mut journal,
&fixture.data,
Expand All @@ -932,7 +944,12 @@ fn cancel_after_the_first_piece(tools_enabled: bool) -> CancelledTurn {
on_stream: Some(&mut on_stream),
},
);
let elapsed = started.elapsed();
let returned = Instant::now();
let after_cancel = returned.saturating_duration_since(
cancelled_at
.get()
.expect("the stream hook cancelled the turn's token"),
);
// Let the server finish; it never saw the rest of its body read.
drop(release);
let served = server.finish();
Expand All @@ -941,7 +958,7 @@ fn cancel_after_the_first_piece(tools_enabled: bool) -> CancelledTurn {
error: result.expect_err("the cancelled turn fails"),
log,
handed,
elapsed,
after_cancel,
served,
has_active_turn,
events: close_and_read(journal),
Expand All @@ -968,9 +985,9 @@ fn a_streamed_text_turn_cancelled_mid_stream_is_recorded_as_cancelled() {
assert_eq!(ended.handed, 1);
assert_eq!(ended.log, ["request_started", "text:one "]);
assert!(
ended.elapsed < CANCEL_BOUND,
ended.after_cancel < CANCEL_BOUND,
"cancellation took {:?}",
ended.elapsed
ended.after_cancel
);

// And the journal holds a cancelled failure, no reply, and no open turn.
Expand All @@ -996,9 +1013,9 @@ fn a_streamed_tool_turn_cancelled_in_its_first_request_runs_no_tool() {
assert_eq!(ended.log, ["request_started", "text:one "]);
assert_eq!(ended.served, 1);
assert!(
ended.elapsed < CANCEL_BOUND,
ended.after_cancel < CANCEL_BOUND,
"cancellation took {:?}",
ended.elapsed
ended.after_cancel
);
assert_eq!(
last_failure_code(&ended.events).as_deref(),
Expand Down Expand Up @@ -1110,7 +1127,8 @@ fn a_cancel_seen_only_after_the_reply_arrived_still_ends_the_turn_as_cancelled()
/// reply of its first request.
struct HeldTurn {
error: omgcode_app::AppError,
elapsed: Duration,
/// From the helper thread's cancel until the turn returned.
after_cancel: Duration,
served: usize,
has_active_turn: bool,
events_before: usize,
Expand Down Expand Up @@ -1142,10 +1160,11 @@ fn cancel_while_a_reply_is_held(
let cancel = cancel.clone();
thread::spawn(move || {
common::wait_arrived(&arrival);
let cancelled_at = Instant::now();
cancel.cancel();
cancelled_at
})
};
let started = Instant::now();
let result = turn(
&mut journal,
&fixture.data,
Expand All @@ -1158,15 +1177,16 @@ fn cancel_while_a_reply_is_held(
on_stream: None,
},
);
let elapsed = started.elapsed();
canceller.join().expect("canceller thread");
let returned = Instant::now();
let cancelled_at = canceller.join().expect("canceller thread");
let after_cancel = returned.saturating_duration_since(cancelled_at);
// Let the server finish; its held reply finds the connection closed.
drop(release);
let served = server.finish();
let has_active_turn = journal.snapshot().active_turn.is_some();
HeldTurn {
error: result.expect_err("the cancelled turn fails"),
elapsed,
after_cancel,
served,
has_active_turn,
events_before,
Expand All @@ -1185,9 +1205,9 @@ fn a_non_streamed_text_turn_cancelled_while_waiting_is_recorded_as_cancelled() {
assert_eq!(ended.error.code(), "cancelled");
assert_eq!(ended.error.message(), "turn was cancelled");
assert!(
ended.elapsed < CANCEL_BOUND,
ended.after_cancel < CANCEL_BOUND,
"cancellation took {:?}",
ended.elapsed
ended.after_cancel
);
assert_eq!(ended.served, 1);
assert_eq!(
Expand All @@ -1209,9 +1229,9 @@ fn a_non_streamed_tool_turn_cancelled_while_waiting_runs_no_tool() {
assert_eq!(ended.error.code(), "cancelled");
assert_eq!(ended.error.message(), "turn was cancelled");
assert!(
ended.elapsed < CANCEL_BOUND,
ended.after_cancel < CANCEL_BOUND,
"cancellation took {:?}",
ended.elapsed
ended.after_cancel
);
assert_eq!(
last_failure_code(&ended.events).as_deref(),
Expand Down Expand Up @@ -1265,9 +1285,9 @@ fn cancelling_the_compaction_before_a_turn_records_nothing() {
"Compaction was cancelled; nothing was recorded"
);
assert!(
ended.elapsed < CANCEL_BOUND,
ended.after_cancel < CANCEL_BOUND,
"cancellation took {:?}",
ended.elapsed
ended.after_cancel
);
assert_eq!(ended.served, 1);
assert!(!ended.has_active_turn);
Expand Down Expand Up @@ -1344,12 +1364,13 @@ fn a_cancelled_compaction_through_the_command_path_records_nothing() {
let cancel = cancel.clone();
thread::spawn(move || {
common::wait_arrived(&arrival);
let cancelled_at = Instant::now();
cancel.cancel();
cancelled_at
})
};

// When the token is cancelled while the summarizer request waits.
let started = Instant::now();
let error = compact_journal(
&mut journal,
&fixture.data,
Expand All @@ -1363,14 +1384,18 @@ fn a_cancelled_compaction_through_the_command_path_records_nothing() {
&cancel,
)
.expect_err("the cancelled compaction fails");
let elapsed = started.elapsed();
canceller.join().expect("canceller thread");
let returned = Instant::now();
let cancelled_at = canceller.join().expect("canceller thread");
let after_cancel = returned.saturating_duration_since(cancelled_at);
drop(release);
server.finish();

// Then it is cancelled promptly and no compaction was recorded.
assert_eq!(error.code(), "cancelled");
assert!(elapsed < CANCEL_BOUND, "cancellation took {elapsed:?}");
assert!(
after_cancel < CANCEL_BOUND,
"cancellation took {after_cancel:?}"
);
assert!(journal.snapshot().compaction.is_none());
assert!(
close_and_read(journal)
Expand Down
6 changes: 5 additions & 1 deletion docs/provider-authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -441,7 +441,11 @@ applies the cancel rule above to every request.
- **Still not interruptible.** A read, list, or edit tool call already in
progress. A command tool call is cancelled through its run.
- **Limits.** Checked against local fixtures only. No live LM Studio server
was used, and the CLI interrupt was delivered by a test only on Linux.
was used, and the CLI interrupt was delivered by a test only on Linux. The
fixtures bound the time from the cancel until the turn or compaction
returns, recording included, at five seconds. They do not bound the work
before the cancel: reading settings and the synced journal appends depend
on the host's storage.

LM Studio documents the request fields in
[Chat Completions](https://lmstudio.ai/docs/developer/openai-compat/chat-completions)
Expand Down
10 changes: 9 additions & 1 deletion docs/testing-standards.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,15 @@ and tool turns, of the compaction before a turn, and of `compact_journal`; a
token set before a turn; the desktop cancel of a non-streamed turn and the
shutdown grace with a turn held in an event sink; and, on Linux only, `SIGINT`
delivered to the CLI during `chat` and `session compact`. The platform
interrupt test also covers `watch_interrupt`. The adapter requires the
interrupt test also covers `watch_interrupt`. In `omgcode-app`, these turn and
desktop tests start the clock at the cancel or shutdown request and require
the turn or compaction to end within five seconds: six tests in
`chat_turn.rs`, the non-streamed cancel in `desktop_chat.rs`, and the streamed
cancel and shutdown in `desktop_events.rs`. The other `omgcode-app`
cancellation tests are not timed, and the `omgcode-core` adapter tests keep
their own bounds. In the listed tests, work before the cancel, such as synced
journal appends, is not timed: a slow Windows runner once spent 5.9 seconds
there. The adapter requires the
compiled base prompt and accepts only a literal loopback HTTP endpoint. The
adapter is not a permission boundary: the CLI selects tool mode from the live
`model_tools` preference, and the runtime rechecks that preference before each
Expand Down
Loading