Skip to content

Close state-machine gaps in reads, transactions and retries - #78

Merged
cgnl merged 4 commits into
mainfrom
fix/state-machine-hazards
Nov 7, 2026
Merged

cgnl merged 4 commits into
mainfrom
fix/state-machine-hazards

Conversation

@cgnl

@cgnl cgnl commented Nov 7, 2026

Copy link
Copy Markdown
Owner

These are targeted fixes for the implicit state machines in the shim that can end up in a wrong state. Each one has its own test where possible.

Hazard Fix
A read that fails while executing on the eager path (no streaming, e.g. LIMIT 1) returned an empty result instead of an error, with only a log line (eager_fetch.rs). Records the error and returns SQLITE_ERROR, like the streaming path, including retry semantics on a lost session (first_fetch_failure_state, now shared).
After a lost session mid-transaction, the tracked transaction stayed active. Every statement, including ROLLBACK, was refused until the handle was closed. ROLLBACK on a lost session clears the transaction, because the server already discarded it, and passes the session check at step entry. Recognising ROLLBACK uses the shim's original SQL, since a PG statement wraps a placeholder statement.
close_handle_connection and destroy_pool_connection left the per-address transaction state behind, so a new connection at the same address could inherit it. Both call transaction::forget_connection.
Thread-local STEP_PG_CONN_ERROR could remain set and make the next failing statement on that thread retry as a connection error. Cleared at the end of every step.
DDL_PENDING in the description cache was only cleared by transaction control. Autocommit DDL left the cache switched off, and an inner RELEASE ended the window too early. Cleared only when libpq reports the session idle: after transaction control, and after autocommit writes and exec.
A stream that had been read ahead was cleared and re-run from row 0 when stepped from another connection. should_clear_cross_thread_result skips streams that have been read ahead (new unit test).
publish-manual-docker.yml overwrote :latest unconditionally, so it could replace a certified image. Moves :latest only when the tag is the repository's latest GitHub release. This workflow is not in the pinned trust surface.

Not changed because the code is already safe:

  • Pipeline mode has no early return between enter and exit.
  • The retry gate checks the handle connection, where transactions are tracked.

The pinned release automation (staging concurrency, shadow sidecar) follows in a separate PR because it requires a trust re-bootstrap.

Tests

  • Clippy -D warnings is clean and all 763 unit tests pass.
  • Runtime E2E has a new case eager_query_errors. The reconnect case now also does ROLLBACK, BEGIN, INSERT and COMMIT after a lost session; that failed before this fix. All relevant cases pass locally. The FTS/rtree/REINDEX cases fail identically on main in my local environment because of its SQLite version.

🤖 Generated with Claude Code

- Eager (non-streaming) reads report execution errors instead of an
  empty result, like the streaming path.
- ROLLBACK on a session lost mid-transaction clears the tracked
  transaction (it was refused forever); ROLLBACK passes the step-entry
  session check. Connection teardown forgets transaction state so a
  connection allocated at the same address cannot inherit it.
- The per-thread connection-error flag is cleared at the end of every
  step instead of leaking into the next failing statement.
- The description cache leaves its DDL window when the transaction is
  really idle (autocommit DDL, not inner RELEASE).
- A read-ahead stream stepped from another connection keeps its rows
  instead of being requeried from row 0.
- publish-manual-docker moves :latest only for the latest release.

Runtime E2E: new eager_query_errors case; reconnect now rolls back and
starts a new transaction after a lost session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sander and others added 3 commits November 7, 2026 11:04
Codex review of the previous commit:
- A COMMIT whose session breaks has an unknown outcome; the following
  ROLLBACK still clears the transaction but reports that instead of
  claiming success.
- The description cache tracks uncommitted DDL per session (the session
  that executed it) instead of one global flag, so another session's
  commit or autocommit write cannot end the window and let an old
  description be cached.
- publish-manual-docker runs in the certified publisher's concurrency
  group and re-reads the latest release right before moving :latest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- publish-manual-docker keeps its own concurrency group: sharing the
  certified publisher's let a queued manual run replace a pending
  certified publication. The latest-release recheck before moving
  :latest stays, and a repository without releases no longer fails.
- sqlite3_exec batches mark DDL even when the batch fails: an explicit
  COMMIT inside the batch can persist DDL before a later statement
  fails.
- Every session reset (pool reset_conn, step read/write PQreset, handle
  session repair) ends that session's DDL window, so a reset cannot
  leave the description cache disabled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Third review round: the latest-release recheck between writing the
version tag and :latest still raced a certified publication, and
suppressed API errors turned into a silent partial success that could
not be rerun. The decision is now made before any tag is written,
errors fail the run, and :latest is left alone while a certified
publication is running.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cgnl

cgnl commented Nov 7, 2026

Copy link
Copy Markdown
Owner Author

Updated after three rounds of independent review (Codex, ChatGPT subscription):

  • Unknown COMMIT outcome: if the session breaks during COMMIT, the following ROLLBACK still clears the transaction but reports that the commit outcome is unknown, instead of claiming success.
  • DDL tracked per session: the description cache tracks uncommitted DDL per session (the session that executed it), not with one global flag. Another session's commit or autocommit write can no longer end the window. Failed sqlite3_exec batches still mark DDL, because a COMMIT inside the batch can persist it. Every session reset ends that session's window.
  • Manual image publication: publish-manual-docker decides about :latest before writing any tag. It only moves it for the latest release, and leaves it alone while a certified publication is running. API errors fail the run, so it can be rerun. It keeps its own concurrency group, because sharing the certified publisher's group could cancel a pending certified publication.

Clippy is clean, 764 unit tests pass, and runtime E2E (including eager_query_errors and the extended reconnect) passes locally.

@cgnl
cgnl merged commit ec877bb into main Nov 7, 2026
17 checks passed
@cgnl
cgnl deleted the fix/state-machine-hazards branch November 7, 2026 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant