Realtime calls with OpenWeb UI?

[hsman2]

OpenWeb UI finally has realtime calling with the openai realtime endpoint. Has anyone gotten it to work yet? I've tried setting it up and it fails. But it also asks for more than just a url and api key. I've added a screenshot of what OpenWeb UI wants, if anyone can assist on getting this working.

[nicholas-greenwood]

could you share the localai yaml for Talk and the error you get when starting a call?
please please please leave any api keys out

the screenshot only shows the model name. would be great to see how Talk is configured before changing the openwebui settings

[hsman2]

[hsman2]

[nicholas-greenwood]

thanks, that shows talk is set up as a pipeline

the 401s are from the tts model/voice lookups. we still don't know whether they're connected to the call failing, and ValueError doesn't tell us why it closed

could you share the message shown in the call window, plus your openwebui/localai versions? current openwebui sends a more specific error to the browser and only logs its type. that would give us something more useful to go on before changing the yaml

[hsman2]

My open webui is version 0.12.3 and localai version is latest commit as of today 7f7b10f.

[nicholas-greenwood]

thanks for checking. annoyingly, 0.12.3 uses that message for a browser websocket error, so it doesn't tell us whether talk is the problem

if you can reproduce it in a desktop browser, i'd check /api/v1/audio/realtime in Network > WS. does it get 101 Switching Protocols? that would confirm the browser's websocket to openwebui opened. if it does, a received bridge.error message would help; otherwise, the status or console error should narrow it down

just the status/error text is enough