Mock TTS emits inside a non-reentrant lock: a skill that speaks on playback parks the whole suite

[goldyfruit]

Running a 271-case end-to-end suite for a skill that speaks and schedules events, the pytest job hangs mid-suite. It reproduces only on a slow runner (GitHub Actions), never on a workstation where the same suite finishes in seconds. Two distinct pile-ups, both in ovoscope.

1. Thread per delivered message (≤ 1.5.0). patched_emit delivered each bus message on its own thread. Across the suite hundreds accumulated, all contending pyee's emitter lock, and the main thread stalled in remove_listener behind them. One job sat at "53%" for 79 minutes before it was cancelled. Upgrading to 1.8.1a1 — which spawns none — removed this, so this half looks already fixed; recording it for anyone still on 1.5.

2. _mock_tts emits while holding self._stop_lock (still in 1.8.1a1). In __init__.py:

• _mock_tts takes _stop_lock and calls bus.emit(... "recognizer_loop:audio_output_start") inside it.
• the _unduck timer takes the same lock and emits recognizer_loop:audio_output_end inside it.

threading.Lock is not reentrant, and these emits are synchronous: whatever a handler does downstream happens with the lock held. If any handler speaks — which a skill legitimately does when an event fires or a round advances — the resulting _mock_tts needs the same lock, and every pending unduck timer queues behind it. The thread dump shows exactly that shape: one thread inside emit at the bus.emit line, several parked at the with self._stop_lock line above it.

~~~ Stack of Thread-32 ~~~
  .../ovoscope/__init__.py", line 678, in _unduck     # with self._stop_lock
~~~ Stack of Thread-27 ~~~
  .../ovoscope/__init__.py", line 681, in _unduck     # bus.emit(...)
  .../ovoscope/__init__.py", line 283, in patched_emit

The lock is there so stop() and the emit cannot interleave, which is worth keeping. Two options that would keep that property without holding it across a delivery: take a snapshot of _stopped under the lock and emit outside it, or make _stop_lock an RLock so a nested speak on the same thread proceeds.

Environment: ovoscope 1.5.0 and 1.8.1a1, ovos-core 3.5.1a1, ovos-workshop 9.7.4a1, Python 3.13 and 3.14 on ubuntu-latest.

Worked around downstream by keeping the skill's scheduled event from firing during the suite, which removes our trigger but not the re-entrancy.

[goldyfruit]

Correction to the second half of the report, after reading 1.8.1a1 rather than 1.5.0: _stop_lock is already an RLock there (__init__.py:545), so re-entrancy on one thread is not the problem and that part of my description was wrong. Apologies.

What the dumps actually show, on 1.8.1a1, is a lock held across a synchronous delivery:

~~~ Stack of Thread-27 ~~~          # holds the lock
  ovoscope/__init__.py:681 in _unduck        # bus.emit(... audio_output_end)
  ovoscope/__init__.py:283 in patched_emit   # return original_emit(self, message)
~~~ Stack of Thread-31, Thread-32 ~~~   # queued behind it
  ovoscope/__init__.py:678 in _unduck        # with self._stop_lock

_unduck emits while holding _stop_lock, and FakeBus.emit dispatches handlers synchronously on the calling thread. So any handler that blocks — and one does here, in a skill whose converse() drives multi-turn rounds — holds the session's lock for as long as it blocks, and every other unduck timer stops. RLock does not help, because the threads waiting are different threads.

The suggestion stands in a narrower form: decide under the lock, emit outside it. Something like reading self._stopped into a local inside with self._stop_lock: and calling bus.emit(...) after the block would keep the stop/emit ordering guarantee without pinning the lock to an arbitrary handler's runtime.

Reproduces only on a slow runner: identical suite, 271 cases, passes in about 68 seconds on a workstation on 3.13 and 3.14, hangs mid-suite on GitHub Actions on one matrix leg per run, never the same leg twice.