fix: restore the SQLite busy wait after sync

Rails installs SQLite's busy wait as a Ruby busy handler through the sqlite3 timeout configuration. PRAGMA busy_timeout neither reports that handler nor preserves it: sqlite3_busy_timeout replaces any registered handler, so reading zero and writing zero back stripped the handler permanently. Every later writer on that pooled connection, inside or outside Solid Objects, then failed immediately with SQLite3::BusyException instead of waiting for the lock. Reinstall the configured handler when restoring, and stop the test helper from simulating contention the same broken way.

Author: cardmagic

CHANGELOG.md

@@ -2,6 +2,14 @@
 
 ## 0.5.1 - 2026-08-07
 
+- Restore the SQLite busy wait that a synchronous invocation suspends for its
+  deadline. Rails installs the busy wait as a Ruby busy handler through the
+  sqlite3 `timeout` configuration, which `PRAGMA busy_timeout` reports as zero
+  and silently replaces, so the previous save and restore left pooled
+  connections with no busy handler at all. Every later writer on that
+  connection, inside or outside Solid Objects, then failed immediately with
+  `SQLite3::BusyException` instead of waiting for the lock.
+
 - Run the doctor round-trip probe on a dedicated caller process, and accept an
   explicit process registry in `SynchronousInvocation`, so the probe can no
   longer stop and delete a shared application caller process, release its

docs/correctness.md

@@ -167,7 +167,13 @@ result. Adapter lock/query deadlines cover the durable enqueue, caller-process
 registration and heartbeat, activation coordination, and result observation.
 SQLite retries busy coordination operations only within the original call
 deadline and reports `waiting_on=database_contention` when the database cannot
-be inspected at timeout. If enqueue cannot commit, `SyncEnqueueTimeout` is
+be inspected at timeout. To keep those retries in Ruby, the SQLite adapter
+suspends the connection's busy wait for the duration of each deadline-bound
+transaction and restores it afterwards. Restoration reinstalls the Ruby busy
+handler Rails configures from the sqlite3 `timeout` setting, which
+`PRAGMA busy_timeout` neither reports nor preserves, so a synchronous call
+leaves the connection's lock waiting behaviour exactly as it found it for
+later writers inside and outside Solid Objects. If enqueue cannot commit, `SyncEnqueueTimeout` is
 raised and no message reference exists. MySQL lock waits have one-second InnoDB
 granularity. Ruby handlers that already started are not preempted.
 

lib/solid_objects/database_adapters/sqlite.rb

@@ -67,11 +67,46 @@ def with_lock_probe
       def with_transaction_deadline(connection)
         return yield unless SyncDeadline.active?
 
-        previous_timeout = connection.select_value("PRAGMA busy_timeout").to_i
+        busy_wait = suspend_busy_wait(connection)
+        begin
+          yield
+        ensure
+          restore_busy_wait(connection, busy_wait)
+        end
+      end
+
+      # @rbs (untyped) -> Hash[Symbol, untyped]
+      def suspend_busy_wait(connection)
+        busy_wait = {
+          pragma_timeout: connection.select_value("PRAGMA busy_timeout").to_i,
+          handler_timeout: configured_busy_handler_timeout(connection)
+        }
         connection.execute("PRAGMA busy_timeout = 0")
-        yield
-      ensure
-        connection.execute("PRAGMA busy_timeout = #{previous_timeout}") if previous_timeout
+        busy_wait
+      end
+
+      # @rbs (untyped, Hash[Symbol, untyped]) -> void
+      def restore_busy_wait(connection, busy_wait)
+        handler_timeout = busy_wait.fetch(:handler_timeout)
+        pragma_timeout = busy_wait.fetch(:pragma_timeout)
+        if handler_timeout && pragma_timeout.zero?
+          connection.raw_connection.busy_handler_timeout = handler_timeout
+          return
+        end
+
+        connection.execute("PRAGMA busy_timeout = #{pragma_timeout}")
+      end
+
+      # @rbs (untyped) -> Integer?
+      def configured_busy_handler_timeout(connection)
+        return nil unless connection.respond_to?(:raw_connection)
+        return nil unless connection.raw_connection.respond_to?(:busy_handler_timeout=)
+
+        pool = connection.respond_to?(:pool) ? connection.pool : nil
+        return nil unless pool.respond_to?(:db_config)
+
+        timeout = pool.db_config.configuration_hash[:timeout]
+        timeout&.to_i
       end
 
       # @rbs (Exception) -> bool

sig/generated/lib/solid_objects/database_adapters/sqlite.rbs

@@ -26,6 +26,15 @@ module SolidObjects
       # @rbs (untyped) { () -> untyped } -> untyped
       def with_transaction_deadline: (untyped) { () -> untyped } -> untyped
 
+      # @rbs (untyped) -> Hash[Symbol, untyped]
+      def suspend_busy_wait: (untyped) -> Hash[Symbol, untyped]
+
+      # @rbs (untyped, Hash[Symbol, untyped]) -> void
+      def restore_busy_wait: (untyped, Hash[Symbol, untyped]) -> void
+
+      # @rbs (untyped) -> Integer?
+      def configured_busy_handler_timeout: (untyped) -> Integer?
+
       # @rbs (Exception) -> bool
       def deadline_error?: (Exception) -> bool
 

test/integration/synchronous_invocation_test.rb

@@ -502,6 +502,18 @@ def wait(timeout:)
     release_sqlite_write_lock(lock) if lock
   end
 
+  test "sync restores the SQLite busy handler it suspended for the deadline" do
+    skip unless SolidObjects::Record.connection.adapter_name.match?(/sqlite/i)
+
+    SolidObjects::Record.connection_pool.with_connection do
+      CounterActor.ref("busy-handler").increment
+
+      assert_nothing_raised do
+        write_while_write_lock_is_briefly_held
+      end
+    end
+  end
+
   test "sync bounds SQLite contention while reusing and heartbeating its caller process" do
     skip unless SolidObjects::Record.connection.adapter_name.match?(/sqlite/i)
 
@@ -758,6 +770,32 @@ def release_sqlite_write_lock(lock)
     thread.join
   end
 
+  BRIEF_LOCK_HOLD = 0.2
+
+  def write_while_write_lock_is_briefly_held
+    lock = hold_sqlite_write_lock
+    releaser = Thread.new do
+      mutex = Thread::Mutex.new
+      mutex.synchronize { Thread::ConditionVariable.new.wait(mutex, BRIEF_LOCK_HOLD) }
+      release_sqlite_write_lock(lock)
+      lock = nil
+    end
+
+    SolidObjects::Process.create!(
+      id: SecureRandom.uuid,
+      kind: "busy-handler-probe",
+      hostname: "test-host",
+      pid: ::Process.pid,
+      started_at: Time.current,
+      last_heartbeat_at: Time.current,
+      metadata: {}
+    )
+    releaser.join
+  ensure
+    releaser&.join
+    release_sqlite_write_lock(lock) if lock
+  end
+
   def invoke_with_immediate_sqlite_lock_failure(message_reference)
     result = Queue.new
     invocation = Thread.new do
@@ -768,15 +806,14 @@ def invoke_with_immediate_sqlite_lock_failure(message_reference)
         attempts += 1 if process_write?(event.payload)
       end
       SolidObjects::Record.connection_pool.with_connection do |connection|
-        previous_timeout = connection.select_value("PRAGMA busy_timeout").to_i
-        connection.execute("PRAGMA busy_timeout = 0")
-        started_at = monotonic_now
-        error = capture_exception do
-          SolidObjects::SynchronousInvocation.new.call(message_reference, timeout: 0.1)
+        with_immediate_sqlite_lock_failure(connection) do
+          started_at = monotonic_now
+          error = capture_exception do
+            SolidObjects::SynchronousInvocation.new.call(message_reference, timeout: 0.1)
+          end
+          elapsed = monotonic_now - started_at
         end
-        elapsed = monotonic_now - started_at
       ensure
-        connection.execute("PRAGMA busy_timeout = #{previous_timeout}")
         ActiveSupport::Notifications.unsubscribe(subscription)
       end
       result << [ error, elapsed, attempts ]
@@ -786,6 +823,25 @@ def invoke_with_immediate_sqlite_lock_failure(message_reference)
     captured
   end
 
+  def with_immediate_sqlite_lock_failure(connection)
+    database_adapter = SolidObjects.database_adapter
+    database_adapter.define_singleton_method(:configured_busy_handler_timeout) { |_connection| 0 }
+    connection.raw_connection.busy_handler_timeout = 0
+    yield
+  ensure
+    database_adapter.singleton_class.send(:remove_method, :configured_busy_handler_timeout)
+    connection.raw_connection.busy_handler_timeout = configured_busy_handler_timeout
+  end
+
+  def configured_busy_handler_timeout
+    SolidObjects::Record
+      .connection_pool
+      .db_config
+      .configuration_hash
+      .fetch(:timeout, 5_000)
+      .to_i
+  end
+
   def process_write?(payload)
     payload.fetch(:sql).match?(/\A(?:INSERT|UPDATE)/) &&
       payload.fetch(:sql).include?(SolidObjects::Process.table_name)