Repository navigation
fix(sdk): wrap all transport errors and stop replaying writes - #295
Conversation
Two coupled transport defects. Leaked exceptions (matrixorigin#269): both the sync and async paths caught (ConnectError, TimeoutException, NetworkError). httpx.RemoteProtocolError — "server disconnected without sending a response", a routine production event — descends from ProtocolError, a *sibling* of NetworkError, so it escaped the SDK exception hierarchy and applications handling MemoriaError/MemoriaConnectionError missed the failure entirely. The same held for LocalProtocolError and ProxyError. Catch the httpx.TransportError base class instead: every subclass means no response was produced, which is exactly what MemoriaConnectionError describes, and new httpx subclasses can no longer slip through. Unsafe write replay (matrixorigin#265): POST/PATCH were retried on 502/503/504. No status code proves the upstream skipped the write — a gateway can return 504 while the upstream keeps going and commits, and 502/503 can follow a commit if the upstream dies right after writing — so retrying without an idempotency key risks committing one logical write twice. Non-idempotent retries are now opt-in via retry_unsafe_writes=True; the same rule applies to the newly-caught transport errors, so wrapping them does not introduce replay. ConnectError is still retried for every method because the request never reached the server, and idempotent methods are unchanged. Retry semantics are now documented as a per-method table. Fixes matrixorigin#265 Fixes matrixorigin#269 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Reviewed head One medium-severity finding remains: [P2] Treat memory correction as an unsafe operation even though its HTTP method is PUT. The retry eligibility helpers classify every PUT as idempotent, so I reproduced two requests with the default options in both sync and async clients, for both a first-response 504 and a first-attempt Please make retry eligibility operation-aware (or explicitly mark the correction endpoint unsafe), apply the opt-in policy to ambiguous correction failures, and cover this PUT endpoint with sync/async regression tests for both status and transport errors. The POST/PATCH tests do not exercise it. Validation: existing Python suite passed, 159/159 tests. Additional Coverage: 6/6 changed files reviewed, including documentation and tests; 0 skipped (100%). |
Follow-up to review on this PR; reproduced before fixing.
Retry eligibility was decided purely by HTTP verb, and PUT is in the
idempotent set, so retry_unsafe_writes=False did not protect
memories.correct() — which sends PUT /v1/memories/{id}/correct. The
server looks the memory up, mints a new UUID-backed replacement and
supersedes the original, so a replay either 404s on the
already-superseded memory or creates a second replacement. Worse, the
ProtocolError retry branch added in this PR *expanded* the exposure:
a disconnect on that PUT previously escaped without replay and would
now have been reissued automatically.
Eligibility is now per operation: _request/_arequest take an optional
`idempotent` override that defaults to the method-based classification,
and the two correct() call sites pass idempotent=False. Verified one
attempt per logical correction for 504 and RemoteProtocolError, sync and
async; retry_unsafe_writes=True still replays; DELETE and other
genuinely idempotent methods keep their behavior.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
lgtm Re-reviewed Validation: reran the Python unit suite on this head, 167/167 passed, including the added correction gateway-error/disconnect regressions. Coverage: 7/7 changed files reviewed, including documentation and tests; 0 skipped (100%). |
Summary
Two coupled transport defects in the Python SDK. Grouped because #269's acceptance criteria explicitly depends on the retry decision in #265 ("POST may have committed before disconnection, so wrapping must not introduce unsafe replay") — and both live in the same
exceptclauses.Leaked exceptions — #269. Both paths caught
(ConnectError, TimeoutException, NetworkError).httpx.RemoteProtocolError— "server disconnected without sending a response", a routine production event — descends fromProtocolError, which is a sibling ofNetworkError, not a subclass:So it escaped the SDK hierarchy and applications handling
MemoriaError/MemoriaConnectionErrormissed the failure. Now thehttpx.TransportErrorbase class is caught: every subclass means no response was produced, which is exactly whatMemoriaConnectionErrordescribes, and future httpx subclasses can't slip through. The original exception is preserved as__cause__.Unsafe write replay — #265. POST/PATCH were retried on 502/503/504. No status code proves the upstream skipped the write — a gateway can return 504 while the upstream keeps going and commits, and 502/503 can follow a commit if the upstream dies right after writing — so retrying without an idempotency key risks committing one logical write twice. Non-idempotent retries are now opt-in via
retry_unsafe_writes=True(the lighter of the two options the issue proposes; a real idempotency key needs server-side dedup support).ConnectErroris still retried for every method, since the request never reached the server, and idempotent methods are unchanged.Worth noting for severity:
memories.storeis already protected server-side by near-duplicate detection, which supersedes rather than duplicates identical content — so the practical exposure was on the other write endpoints. That's recorded in the docs.Verification
Fault injection via
httpx.MockTransport, sync and async:RemoteProtocolErrorMemoriaConnectionErrorLocalProtocolError,ProxyErrorMemoriaConnectionErrorConnectError,ReadTimeout,ReadErrorCommit counting with a transport that records a commit then returns a gateway error:
retry_unsafe_writes=TrueRemoteProtocolErrorConnectErrorTest plan
pytest tests/unitpasses — 159 tests (121 baseline + newtest_transport_errors.pycovering all 8TransportErrorbranches × GET/POST/async, plus the replay matrix)retry_unsafe_writes=Trueso the opt-in path isn't untestedruff check src tests— no new findings (7 pre-existing, identical on unmodifiedmain, from a newer ruff than the project's>=0.4floor)Fixes #265
Fixes #269
🤖 Generated with Claude Code