Skip to content

fix(realtime): strip EOU language tags from the LLM context - #12750

Open
nexxtmobile wants to merge 2 commits into
mudler:masterfrom
nexxtmobile:fix/realtime-strip-eou-langtag
Open

nexxtmobile wants to merge 2 commits into
mudler:masterfrom
nexxtmobile:fix/realtime-strip-eou-langtag

Conversation

@nexxtmobile

Copy link
Copy Markdown
Contributor

Fixes #12748

ASR backends like Nemotron-3.5 append end-of-utterance language tags to
transcripts ("Hallo. <en-US>"). The committed transcript feeds the LLM
conversation verbatim (user item Transcript field), and the per-utterance
tag is unreliable for short/ambiguous utterances — the LLM treats it as a
language signal and switches the reply language against the session
instructions (observed: German "Hallo." tagged <en-US> → English reply in
a German call).

commitUtteranceWithTranscript now strips the EOU tags from the transcript
that goes to appendUserItem/generateResponse (the LLM context). The
emitted input_audio_transcription event keeps the raw transcript
(debugging / client display).

Includes regression specs: stripEOULanguageTags unit specs + a client-commit
spec asserting the emitted event keeps the raw tag while the LLM user message
is tag-free.

Tested: go test -race ./core/http/endpoints/openai/ green (Go 1.26).

@localai-org-maint-bot

Copy link
Copy Markdown
Collaborator

@mudler Code-review sign-off at 75b8bf975, subject to CI and human DCO certification of the documentation fix. The client receives the raw transcription before language-tag removal; the LLM receives the cleaned transcript, with the existing empty-transcript guard retained. I added the required documentation. Reviewed against master 48f358dd9; whitespace checks pass. Focused endpoint tests could not start because the dependency proxy denied LocalAGI; race testing also lacks CGO/a C compiler. Nonblocking caveat: the broad tag pattern can also remove literal short HTML tags such as <div> from transcription text.

ASR backends like Nemotron-3.5 append end-of-utterance language tags to
transcripts ("<de-DE>", "<en-US>", ...). The committed transcript feeds
the LLM conversation (the user item Transcript field), but the per-utterance
tag is unreliable for short/ambiguous utterances (a German "Hallo." tagged
<en-US>) and the LLM treats it as a language signal, switching the reply
language against the session instructions.

commitUtteranceWithTranscript now strips the EOU tags from the transcript
that goes to appendUserItem/generateResponse (the LLM context). The emitted
input_audio_transcription event keeps the raw transcript (debugging /
client display).

Regression tests: stripEOULanguageTags unit specs + a client-commit spec
asserting the emitted event keeps the raw tag while the LLM user message is
tag-free.

Signed-off-by: nexxtmobile.de <git@nexxtmobile.de>
@nexxtmobile
nexxtmobile force-pushed the fix/realtime-strip-eou-langtag branch from 75b8bf9 to dff4880 Compare October 19, 2026 13:16
Assisted-by: Codex:gpt-6
Signed-off-by: nexxtmobile.de <git@nexxtmobile.de>
@nexxtmobile

Copy link
Copy Markdown
Contributor Author

Update: rebased onto mudler:master e51747055 (the branch had gone stale / conflicting). go test -race ./core/http/endpoints/openai/ is green on the new base.

Note on the doc commit: a rebase force-push clobbered the maint-bot's doc commit (75b8bf975). I have re-applied the identical doc change (same file, same text) as 005a2d4ce with human DCO attestation, per the process used in #12622. Sorry for the churn.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Realtime: ASR EOU language tags leak into the LLM conversation (unintended reply-language switching)

2 participants