Send the LLM request before the silence is completely detected. For example if the silence seconds is 2 s, send an LLM request with all the available transcription after 1s of silence. If the silence was indeed for 2 seconds we'll save 1s on LLM request time, if there was no silence and the user started speaking again we lose money on some tokens.
Should be a param.
Send the LLM request before the silence is completely detected. For example if the silence seconds is 2 s, send an LLM request with all the available transcription after 1s of silence. If the silence was indeed for 2 seconds we'll save 1s on LLM request time, if there was no silence and the user started speaking again we lose money on some tokens.
Should be a param.