core: honour bindWaitPolicy at switchBackend and reconnect, cancel the adapter's own completions, and document TimeoutScheduler::cancel (fixes #615, fixes #619, fixes #620) - #639
Conversation
Runner verification (landing sweep, 2026-09-20)Checked the one claim the branch's safety rests on, against the repository. The claim: for every 1. The evidence tests really are unchanged. 2. The mechanism holds, read at the source. The default auto [completion, promise] = Completion<ModelId>::makeSettleable(&cbExec);
try {
promise.resolve(bindModelBlocking(std::move(request)));
} catch (...) {
promise.reject(std::current_exception());
}
return std::move(completion);and So a backend that does not override 3. The knowingly-red mutation gate is real, and reproduced. Not taken on trust: The hint must move 1322 → 1412, exactly as reported. The lane was right not to edit that file — PR #635 holds it and restructures it substantially (#613 converts the Merge order for these two#635 lands first, then this PR rebases onto it and repoints the hint in #635's new structured shape. Doing it the other way means a conflict in Left for the next sweep
Noted, not acted onThe finding in the hand-off that the first mutation run of #615's timing cases passed and should not have — the test double's blocking verbs were bounded by the same #636 filed by the lane is correctly untriaged; Step 1 of the next sweep picks it up. 🤖 Generated with Claude Code |
…g callback (fixes #620) `cancel()`'s `@brief` said "immediately" and its body said "a no-op if the handle already fired or was already cancelled". Neither covers the third case, which is the one a caller reasoning about lifetime cares about: `run()` erases the entry *before* invoking the callback and drops `_mtx` across the invocation, so a `cancel()` for a callback that has started takes the not-found branch and returns while that callback is still running on the scheduler thread. "Already fired" silently covered both "finished" and "currently running". This is documentation plus one contract clarification, not a behaviour change. Making `cancel()` wait would be the wrong fix -- a callback that posts back to the cancelling thread would deadlock it, the family `docs/spec/concurrency_and_lifetimes.md` names and the reason `CallbackScope` has no block-until-drained either. What was missing is the obligation that non-waiting places on a caller, so `cancel()` now states it: every scheduled callback must stay safe to run after its own `cancel()`. - `cancel()` (threaded build) distinguishes "not started" from "already started", says `~TimeoutScheduler` is the only thing here that means "no callback in flight", and names why blocking is not the alternative. - The `@file` comment gains the third build difference. Only the browser build, being single-threaded, actually gives "no callback starts after `cancel()` returns"; the bullet list previously documented the strictly less interesting timer-not-cleared difference and omitted this one. - The browser `cancel()` says its "already fired" can only mean *finished*, and that a caller wanting both builds still gets the weaker guarantee. - Both shipped callers now say why a late callback is harmless rather than leaving it to be rediscovered: `Bridge::executeVia`'s deadline relies on `CompletionState`'s first-result-wins, `RemoteServer`'s `executeTimeout` on `complete`'s reply-exactly-once flag. - `docs/spec/core/completion.md` said "Two behavioural differences"; it is three now, and it names which one a cross-build caller must code against. Verification: two Catch2 cases added. "cancel() returns while the callback it names is still running" reproduces the finding (measured, GCC 16.2.1 Debug, Linux); its contrast case asserts `~TimeoutScheduler` *does* join, which is what makes the first a real distinction rather than a timing accident. Mutated to confirm the first case is not vacuous: running the callback under `_mtx` in `run()` (so `cancel()` blocks behind it) fails it at `REQUIRE_FALSE(finished.load())` with `!true`. Not verified: the Emscripten build, which no toolchain here can compile -- its stronger guarantee is inferred from being single-threaded, as the header's own `@warning` already says of everything in that branch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VptDWG2fKr2vBnLSJcgzgW
…ions it produced (fixes #619) `cancelPending` was a one-line forward to the wrapped backend. The two verbs this adapter exists to reshape do not produce completions the wrapped backend knows about: `bindModel`/`promoteModel` settle from a task posted to the adapter's own `_control` strand, holding a promise `_inner` has never seen. So the forward reached none of them, and a bind cancelled by `~Bridge` or by `switchBackend` went on to resolve **successfully** afterwards -- the exact opposite of the contract it overrides ("after this call, any later `setValue`/`setException` on those states is a no-op, so in-flight server replies cannot resurrect a cancelled completion"). The adapter now keeps a `weak_ptr` to each dispatched promise and rejects the live ones before forwarding, on the snapshot-then-deliver shape and the amortised compaction `LocalBackend::cancelPending`/`trackPending` already use (morph#528). An entry expires when its strand task is destroyed, so the success path erases nothing; tracking happens before the post, so a cancellation landing in the gap still finds the promise. Two limits are stated in the header rather than left to be rediscovered. A task that settles first wins -- its completion was not still pending, the same race `LocalBackend` has always had. And a task already queued on `_control` still runs its blocking control call against the wrapped backend after the cancellation: the caller is told the bind was cancelled while the registration may still go through. Stopping that needs the task to check before calling `op()`, not the promise to be settled after it, so it is a separate change, filed as morph#636 rather than folded in here. Verification: reproduced and then fixed, measured on GCC 16.2.1 Debug, Linux. The new case wraps a backend whose control call blocks until the test releases it, so the completion is provably still pending when `cancelPending` runs, and covers `bindModel` and `promoteModel` in two sections -- morph#619 had measured only the bind and inferred the promote. Mutated to confirm it is not vacuous: restoring the plain `_inner->cancelPending(exc)` forward fails it in both sections at `REQUIRE(waitUntil([&] { callerExec.runOnce(); return errRan.load() == 1; }))` with `false`, i.e. no rejection is ever delivered. Not verified: any production effect, because there is none to have -- `grep -rn SynchronousBackendAdapter` still finds no production call site, and morph#571 making this surface the default path is what would turn this from latent into live. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VptDWG2fKr2vBnLSJcgzgW
…et them ask bindWaitPolicy (fixes #615) After morph#568 and morph#593, `Bridge` had four places that acquire a model for a binding. Two went through the structural surface and honoured `IBackend::bindWaitPolicy()`; the other two -- `switchBackend`'s phase 1 and `installReconnectHandler`'s reconnect callback -- still called the blocking `registerModelShared`/`registerModelWithContext` directly, with no policy check and no structural path at all. So a backend that had just been given a way to say "do not block my caller" was blocked at both of them anyway: a WASM client that never blocks on its *first* registration still could on a reconnect or a backend swap, where the block is a nested `QEventLoop` on the very thread that has to deliver the reply. Both now dispatch through `bindModel` with the shape every other site already has -- dispatch, park an inline reply in an `AsyncDispatchHandoff`, then `awaitHandoff` or `claimHandoff` according to the policy. For every `kCallerMayBlock` backend in the tree that is behaviour-for-behaviour what it was: the default `bindModel` runs exactly the legacy verb the request shape names, settles inside the call, and the wait finds the outcome already there. What is new is what happens when the answer is `kCallerMustNotBlock`: - `switchBackend` defers those binds. The swap happens, each deferred binding's `currentId` is cleared, `registrationInFlight` is set so `whenBound()` can gate on the window, and the reply publishes its own id. Atomicity is now exactly as strong as the wait is, and the header and bridge.md both say so rather than continuing to promise all-or-nothing: "did every re-registration succeed" is not knowable without waiting, and waiting is the deadlock. A deferred bind that fails after the swap is logged and rejects that binding's waiters; it cannot roll the switch back. - The rollback keys on a rejected `Completion` instead of a thrown exception, which is what morph#615 asks for: the structural surface reports failure through the completion, and a bare `catch (...)` cannot see it. - Waiters are resolved, and the staging exception rethrown, only after `_mtx`/`_attachMtx` are released -- resolving a waiter runs consumer code that is free to re-enter the `Bridge`. - The reconnect handler clears `currentId` for a deferred binding: the id belonged to the connection that just dropped, so `executeVia`'s fast fail is the honest answer rather than dispatching a dangling id. A failing re-registration there no longer throws out of the handler onto the transport thread and abandons every binding after it; it is reported per binding and the loop continues. The three sites' continuation pair is now one `makeBindCallbacks` body rather than three copies of the same liveness gate, stale-backend check and waiter settlement -- they had three chances to solve it differently. The per-binding dispatch is likewise one `rebindThroughSurface`, and `switchBackend`'s two phases are `stageRebinds`/`rollbackStaged`/`commitRebinds`. That last split is not tidiness: with everything inline, clang-tidy's `readability-function-cognitive-complexity` scored `switchBackend` at 45 and `installReconnectHandler` at 46 against a threshold of 25 -- CI's clang-tidy-diff leg failing -- and it was right. A staging phase, a rollback, a commit and a settlement pass in one body is four concerns the reader has to hold at once. Also settles the ownership disagreement morph#615 reports as its second finding: docs/spec/core/backend.md's migration table assigned the reconnect half to morph#570, whose own body scopes itself to `examples/` and never mentions `bridge.hpp`, and assigned `switchBackend` to nobody. The table now has a morph#615 row and morph#570 keeps only what its body claims. Verification, measured on GCC 16.2.1 Debug, Linux: full suite 1547 cases / 22704 assertions, all passing (the single "failed as expected" case is test_replay_ledger's deliberate negative-conformance run). Three new cases, each run against a double whose `bindModel` never settles on its own and whose legacy verbs park the calling thread, so a site that goes back to a blocking verb does not merely fail an assertion -- it fails to return. Each runs the site on its own thread, records whether it came back inside the polling budget, releases the latch so the thread is joinable either way, and asserts afterwards, so a regression fails the case instead of wedging the suite. Also run: clang-tidy-diff over the whole branch diff against a configure-only clang-debug database (clang 22.1.8, matching CI's pinned CLANG_VERSION), which now reports nothing. Partial rather than equivalent to CI: that database is configured with tests only, where CI configures every optional feature, so a translation unit this run never analysed could still reach a changed line. Mutated twice. Re-introducing the blocking legacy call at both sites, leaving everything else intact, fails both timing cases at `REQUIRE(returnedPromptly)` with `false`. Building the new tests against this branch's base (`git show a8511aa:include/morph/core/bridge.hpp`) fails all three, including the rollback case at `REQUIRE_THROWS_AS(...)` with "no exception was thrown where one was expected" -- the old loop never called `bindModel`, so a rejection it never asked for could not reach it. Worth recording because it nearly slipped through: the *first* mutation run passed. The double's legacy verbs were bounded by `kDefaultWaitBudget`, the same two seconds the test's own `waitUntil` polls for, so the blocked thread came back at almost exactly the moment the measurement gave up. The bound is now 60 s -- a wedge guard, not a race partner -- and only then did the mutation fail the cases. A budget equal to the thing it measures is not a measurement. Not verified: any of this against a real `QtWebSocketBackend` with `asyncRegistrationEnabled`, or in a browser. No WASM toolchain is available here, and no shipped example sets that flag *and* relies on reconnect or `switchBackend`, so the WASM consequence remains inferred from the dispatch chain exactly as morph#615 states it -- what is measured is that neither site blocks a caller on a `kCallerMustNotBlock` backend, and that both bind it correctly when the reply lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VptDWG2fKr2vBnLSJcgzgW
…anch moved `scripts/branch_partial_allowlist.json` is keyed on `source` text, with `line` as a hint the gate audits — a drifted hint fails `check_branch_coverage.py`'s `resolve_allowlist_source_line` with "has moved to line N", by design (morph#349/#355/#419). Three of its entries sit below insertions this branch made: include/morph/core/backend.hpp:1324 -> 1414 include/morph/core/bridge.hpp:1542 -> 1551 include/morph/core/bridge.hpp:1654 -> 1673 The B11 entry's own prose cites the two textually-identical guards it is *not* about (`if (deadlineHandle && schedulerRef)` in the `.then`/`.onError` continuations); those moved 1678/1767 -> 1697/1786 and are repointed in the reason text too, since a citation inside a reason rots exactly like one in a field. Verified by resolving each entry's `source` against the working tree: all 22 entries match their hint exactly, and the three lines above were each read at their new number to confirm they are the same site the reason describes — the `catch` arm of `executeVia`'s dispatch guard for B11, not one of the two continuations. No entry's disposition changed. Not verified by a coverage run: no llvm-cov build was made here, so "still partial" is what the gate last measured, not something re-measured now. Separate commit from the three ticket commits on purpose: it is bookkeeping for insertions those commits made, and it must survive one of them being dropped. **Not repointed, and it will fail CI's mutation leg:** `scripts/mutation_survivors.json` needs `include/morph/core/backend.hpp:1322 -> 1412` (`aware.reserve(_changeAware.size());`), reported by `python3 scripts/check_mutation_survivors.py`: scripts/mutation_survivors.json: 1 citation(s) no longer match the code they name. - include/morph/core/backend.hpp:1322 has moved to line 1412. The text still matches, so nothing is wrong with the disposition -- update the `line` hint. That file is held by PR #635, so this lane does not touch it. Whoever lands these two branches applies that one-line hint change in whichever merges second. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VptDWG2fKr2vBnLSJcgzgW
This branch adds 97 lines to `include/morph/core/backend.hpp`, which moves
every `side_channel_metrics` citation in it. The gate reported all three:
- include/morph/core/backend.hpp:1322 has moved to line 1412. The text still
matches, so nothing is wrong with the disposition -- update the `line` hint.
- include/morph/core/backend.hpp:1215 is allowlisted by a source line that
appears 2 times (lines [1305, 1324]), and none of them is 1215, so which
one is meant is not decidable. Make the entry unambiguous.
- include/morph/core/backend.hpp:1379 is allowlisted by a source line that
appears 2 times (lines [1469, 1514]), and none of them is 1379, so which
one is meant is not decidable. Make the entry unambiguous.
The first resolves mechanically (`aware.reserve(_changeAware.size());`,
1322 -> 1412). The other two are the pair whose entries record in advance that
a drift here is reported as ambiguous rather than auto-corrected, because each
cites a statement appearing twice verbatim, so each was resolved by reading the
code at both candidates:
1305 is the `registerCount` emission inside `registerModel`; 1324 is the
`registerModelShared` arm. The entry names `registerModel`.
1469 is the `executeInFlight` increment -- `fetch_add`, `inFlightAfterInc`,
outside the posted task; 1514 is the decrement twin inside it. The entry
names the increment side.
No `source` text and no `reason` is touched, so no triage is re-stated that
nobody performed. After: `python3 scripts/check_mutation_survivors.py` exits 0.
This became possible only once #635 landed: that PR held this file, so the
repoint could not ride the commits that caused it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VptDWG2fKr2vBnLSJcgzgW
1ca161d to
024d5e7
Compare
Runner: rebased onto
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Three tickets on the core threading/cancellation tree, one commit each, plus one clearly-labelled bookkeeping commit. Nothing was rejected.
481c3c1TimeoutScheduler::cancel()documentation + contract clarification38370feSynchronousBackendAdapter::cancelPendingcancels its own completionsf944e26switchBackendand the reconnect handler ontobindModel/bindWaitPolicy1ca161dbranch_partial_allowlist.jsonline hints this branch moved#620 —
TimeoutScheduler::cancel()and an already-firing callbackConclusion asked for by the ticket: prose, plus one contract clarification — not a behaviour change.
run()erases the entry before invoking the callback and drops_mtxacross the invocation, so acancel()for a callback that has started takes the not-found branch and returns while that callback is still running. The header's "already fired" silently covered both "finished" and "currently running".Making
cancel()wait would be the wrong fix, not a missing feature: a callback that posts back to the cancelling thread would deadlock it — the familydocs/spec/concurrency_and_lifetimes.mdnames and the reasonCallbackScopehas no block-until-drained either. What was missing is the obligation that not-waiting puts on a caller, socancel()now states it: every scheduled callback must stay safe to run after its owncancel(). Both shipped callers now say why they are (first-result-wins inBridge::executeVia, reply-exactly-once inRemoteServer), and the@filecomment gains the third build difference — only the single-threaded browser build actually gives "no callback starts aftercancel()returns".Two Catch2 cases. The finding one is mutation-checked: running the callback under
_mtxinrun(), socancel()blocks behind it, fails it atREQUIRE_FALSE(finished.load())with!true.#619 — the adapter's own pending completions
cancelPendingwas_inner->cancelPending(exc);. The two verbs the adapter reshapes settle from a task on its own_controlstrand, holding a promise the wrapped backend has never seen — so the forward reached none of them and a cancelled bind went on to resolve successfully. The adapter now tracks each dispatched promise weakly and rejects the live ones before forwarding, on the snapshot-then-deliver shape and amortised compactionLocalBackendalready uses.Reproduced and fixed, with
bindModelandpromoteModelcovered — #619 measured only the bind and inferred the promote. Mutation: restoring the plain forward fails both sections atREQUIRE(waitUntil([&] { callerExec.runOnce(); return errRan.load() == 1; }))withfalse.One thing this deliberately does not fix, and it is filed rather than folded: a task already queued on
_controlstill runs its blocking control call against the wrapped backend after the cancellation, so the caller is told the bind was cancelled while the registration may still go through. That needs the task to check before callingop(), not the promise to be settled after it — #636.#615 — the two sites that never asked
bindWaitPolicy()switchBackend's phase 1 andinstallReconnectHandler's callback still called the blockingregisterModelShared/registerModelWithContextdirectly. Both now go throughbindModelwith the shape every other site already has — dispatch, park an inline reply in anAsyncDispatchHandoff, thenawaitHandofforclaimHandoffby policy.For every
kCallerMayBlockbackend this is behaviour-for-behaviour what it was: the defaultbindModelruns exactly the legacy verb the request shape names and settles inside the call. What is new iskCallerMustNotBlock:switchBackenddefers those binds — the swap happens,currentIdis cleared,registrationInFlightis set sowhenBound()can gate the window, and the reply binds it. Atomicity is now exactly as strong as the wait is, and the header andbridge.mdboth say so rather than continuing to promise all-or-nothing. A deferred bind that fails after the swap is logged and rejects that binding's waiters; it cannot roll the switch back, and an instance it created on a backend whose switch later threw is not deregistered. Both are consequences of not waiting, stated rather than hidden.Completionrather than a thrown exception, which is what the ticket asks for.currentIdfor a deferred binding (the id belonged to the connection that dropped), and a failing re-registration no longer throws out of the handler onto the transport thread taking every later binding with it._mtx/_attachMtxare released — resolving one runs consumer code free to re-enter theBridge.It also settles the ticket's second finding:
docs/spec/core/backend.md's migration table assigned the reconnect half to #570, whose own body scopes itself toexamples/and never mentionsbridge.hpp, and assignedswitchBackendto nobody. The table now has a #615 row and #570 keeps only what its body claims.Review reasoning, inline (no
/code-review, no/simplify)Reading the change back, four things wanted deciding rather than assuming:
resolveRegistrationWaitersinvokes consumer callbacks. Both new sites had them inside the{_mtx, _attachMtx}region on the first pass; a waiter that re-enters theBridgewould self-deadlock. They are collected under the locks and settled after, and the staging exception is rethrown after that too, so the throw path settles its waiters rather than stranding them._mtx, whichswitchBackendholds for the whole staging + commit + swap. A reply that lands during that window blocks a transport thread until the swap is done, and then seespinned == loadBackend()— the outcome we want. It cannot be the calling thread, because an inline settle is parked byparkIfInFrameand handled in-frame.currentIdshould be. Leaving the old id would letexecuteViadispatch a dangling id at a backend that never issued it.0makes it fail fast, which is the documented "handler not bound" path, andwhenBound()makes the window waitable.switchBackend45 andinstallReconnectHandler46 against a 25 threshold — CI's clang-tidy-diff leg failing, and correctly: four concerns in one body. Split intorebindThroughSurface/stageRebinds/rollbackStaged/commitRebinds/reregisterLive, with the three sites' continuation pair collapsed into onemakeBindCallbacksinstead of three near-copies.Verification
Measured, GCC 16.2.1 Debug, Linux: 1547 cases / 22704 assertions, all passing (the one "failed as expected" is
test_replay_ledger's deliberate negative-conformance run). Also clean locally:clang-format,check_nolint_directives.sh,check_catch_test_names.sh,check_test_type_names.sh,check_deprecated_markers.sh,check_spec_citations.sh,check_spec_sync.sh(driven with the realorigin/master..HEADpath list: "10 sub-domain(s) classified; every touched header sub-domain has a matching spec change"), the Doxygendoctarget, andclang-tidy-diff.pyover the whole branch diff with clang 22.1.8 (CI's pinned version) — the last one against a tests-only configure, where CI configures every optional feature, so it is partial rather than equivalent.#615's three new cases were run against this branch's base (
git show a8511aa6:include/morph/core/bridge.hpp) and all three fail, including the rollback case atREQUIRE_THROWS_AS(...)with "no exception was thrown where one was expected". Re-introducing just the blocking legacy call at both sites fails the two timing cases atREQUIRE(returnedPromptly)withfalse.Worth flagging because it nearly slipped through. The first mutation run of the #615 timing cases passed. The test double's legacy verbs were bounded by
kDefaultWaitBudget— the same two seconds the test's ownwaitUntilpolls for — so the blocked thread came back at almost exactly the moment the measurement gave up. The bound is now 60 s, a wedge guard rather than a race partner, and only then did the mutation fail. A budget equal to the thing it measures is not a measurement; the comment in the double says so, with the observation.Not verified: any of this against a real
QtWebSocketBackendwithasyncRegistrationEnabled, or in a browser — no WASM toolchain here, and no shipped example sets that flag and relies on reconnect orswitchBackend. The WASM consequence stays inferred from the dispatch chain, exactly as #615 states it. What is measured is that neither site blocks a caller on akCallerMustNotBlockbackend and that both bind it when the reply lands. Also not re-measured: branch coverage (no llvm-cov build was made here), so the repointed allowlist entries are "still partial" on the gate's last measurement, not on a new one.One thing this branch knowingly leaves red
scripts/check_mutation_survivors.pyfails on this branch:scripts/mutation_survivors.jsonis held by PR #635, so this lane does not touch it. Whichever of the two merges second applies that one-line hint change (1322→1412,aware.reserve(_changeAware.size());).Issues filed
SynchronousBackendAdapter::cancelPendingsettles the completion but the queued control call still registers on the wrapped backend. Reproduced as a side observation of core: SynchronousBackendAdapter::cancelPending does not cancel the completions the adapter itself produced #619's own regression test; the assertion recording it is in this PR and is the one that would change if core: SynchronousBackendAdapter::cancelPending settles the completion but the queued control call still registers on the wrapped backend #636 is fixed.Closes #615, #619, #620.
🤖 Generated with Claude Code
https://claude.ai/code/session_01VptDWG2fKr2vBnLSJcgzgW