Repository navigation
Quote a cut's words once they are transcribed - #449
Merged
Merged
Conversation
jserv
reviewed
Oct 23, 2026
jserv
reviewed
Oct 23, 2026
ColtenOuO
reviewed
Oct 23, 2026
jserv
requested changes
Oct 23, 2026
jserv
left a comment
Contributor
There was a problem hiding this comment.
Avoid the ambiguous reference Refs #188.
jserv
reviewed
Oct 23, 2026
Aatroxo1
force-pushed
the
cut-line-waits-for-words
branch
from
October 23, 2026 12:53
3d37e27 to
2b38301
Compare
Aatroxo1
force-pushed
the
cut-line-waits-for-words
branch
from
October 23, 2026 13:41
2b38301 to
2a2b154
Compare
jserv
reviewed
Oct 23, 2026
jserv
reviewed
Oct 23, 2026
Gemini sends Interrupted with the start of the candidate's activity, before the words behind it are transcribed, and the candidate turn that is still open then holds the answer the reply was answering. The line naming the cause therefore quoted that answer for every cut, and quoting only what followed the reply read speech as wordless. The cut is now logged where it happens, and a second line quotes the start of the turn that follows it once that turn closes, or when the room ends, with how long after the cut it settled. A close that lands while the candidate is still speaking leaves the cut waiting with the words said so far, so neither the late end of the cut reply nor a pause reads speech as wordless. Closes sysprog21#188
Aatroxo1
force-pushed
the
cut-line-waits-for-words
branch
from
October 24, 2026 05:15
2a2b154 to
1539ecb
Compare
Contributor
|
Thank @Aatroxo1 for contributing! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The line printed when Gemini cuts the interviewer quoted the candidate turn as it stood at the cut, and at that moment the turn still held the answer Jim was replying to, so every cut quoted that answer and a wordless cut could not be told from speech.
Quoting only what followed the reply, the first approach on #188, fails the other way:
Interruptedarrives with the start of the candidate's activity, before the words that caused it are transcribed, so speech then read as wordless too. The quote has to wait for the words.on_interruptionnow logs the cut where it happens (Gemini cut its own turn at=..., N.Ns of it unplayed), closes the answer's turn, and notes the cut inSpeakerTurns.close_turnsprints a second line when the next candidate turn closes:candidate audio since the cut at=..., settled N.Ns later:and the first 80 characters of that turn, or(nothing transcribed; ...)when it has no words. The start is quoted because the words that caused the cut come first, andsettledshows how late the quote is. The line is read before the turns close, since closing clears the words, andclose_turns_settling_cutkeeps that order testable without a room.A close that lands while the candidate is still speaking leaves the cut waiting, and keeps the words the turn holds, because that close clears them and they are the ones the quote exists to show. After
Interrupted, a late transcript of the cut reply makes itsTurnCompletereachclose_turnsbefore any of the candidate's words arrive, which aHarnesstest reproduces. ATurnCompleteafter the words never gets there, since their arrival sets the discard that swallows it; the closes that can land mid-speech are the ones the loop makes outside an event, a pause or a socket replacement, and aHarnesstest drivesclose_turnsas they do. A cut still waiting when the next one arrives is settled first.A cut still waiting when the room exits is settled right after the room loop returns in
run_room_with_slot. Everyreturn,?andbreakout of the loop lands there, so the exits that never callclose_turns(the candidate not returning, the billing exits, the event stream ending) no longer drop it, and neither will an exit added later.One limit remains: a cut whose turn closes before any words are transcribed, as when the interview ends or the tab closes while the candidate is speaking, still reads as wordless.
SpeakerTurn::tailis replaced byhead, its only caller being this line. It reads the cleaned turn, so the#hashtagtranscription artifact counts as no words. The two doc lines that sat abovetaildescribesegment_id, and now sit there.docs/livekit-connection-troubleshooting.mddescribes the two lines.Twelve unit tests in
tests/unit/livekit/session.rscover speech after a cut, no words, the artifact alone, a long reply, a close while the candidate speaks and the words it keeps, a second cut, a cut waiting at teardown, no cut at all, and three throughHarnessthat drivehandle_gemini_eventandclose_turns. Removing the speaking check makes four of them fail, not keeping the words three, and noting the cut before closing the answer's turn three. The call inrun_room_with_slotis out of reach ofcargo test; the live check below covers it.Live check with a headset for microphone and playback, on this change:
settled 8.7s laterquoted the wordssettled 31.5s laterquoted the first 80 characterssettled 14.0s laterLiveKit event stream ended,settled 28.6s laterread(nothing transcribed; ...), because the words had not arrivedIn one session I tried to cut Jim off with speech three times; two overlapped a reply, and the third began 2 ms after the reply ended (
kind=pause), so there was nothing to cut. On the previous revision of this change, a cough or tongue click over a reply (3 times) and a tongue click over the opening (3 times) produced no cut, and this revision only changes what is printed after one../scripts/test.shpasses on 82bbdc5 plus this change. Of ten full local runs, two failed on thefloor-capturebrowser tests (follower not ready, a 5 s startup wait) and one on theindentgate over a comment wrap, since fixed; that test file passes alone, five runs in a row on this branch and on a cleanupstream/main, and the other seven full runs passed. Not checked locally: Prettier, ruff, eslint, cargo-audit, actionlint, script-requires, and the browser tests that need Chromium or Java.Changes 2 and 3 from #188 (
prefixPaddingMs, and keeping the queued audio incut_unless_protecteduntil the interruption carries words) are not part of this pull request. They wait for a log from a build with this change that shows a wordless cut, and get their own issue then.Closes #188