Feature Type
Would make my life easier
Feature Description
AMD (livekit.agents.AMD) deliberately defers all audio processing — the no-speech timer (#5848) and, as of 1.6.9, the detection timeout too (#6187 / #6580) — until sip.callStatus == "active". Both fixes treat all pre-answer audio uniformly as "ringback / carrier early media / dialtone" noise that should never influence classification, which is correct when early media is just a ringback tone.
We run outbound voice AI calls through several Latin American mobile carriers, and we've confirmed that at least a couple of them (seen so far in Peru) send the ringing tone, system announcements, or the voicemail greeting itself via early media — SIP 183 Session Progress with an SDP body — rather than only after a final 200 OK. For calls that reach voicemail on these carriers, the 200 OK either arrives well after the greeting has already played, or the leg never produces a clean "answered" transition the way AMD expects. Deferring to active in that situation doesn't protect the classifier from noise — it discards the only audio that was ever going to be available, so the call falls through as uncertain/no-answer instead of being correctly classified as machine-vm.
This is different from what #6187 fixed: that issue was about timer accounting (a detection_timeout armed at track-subscription burning down during a long ring before a late active answer arrives — the useful audio there was always post-answer). In our case the useful audio is the pre-answer early media itself.
We're not asking to change the default behavior — the ringback-only case #5848/#6187 fixed is the common case and shouldn't regress. We'd like an opt-in, e.g. a process_early_media: bool (default False) on AMD.__init__ or DetectionOptions, that — when set — starts start_listening() (timers + transcript processing) from track subscription instead of gating behind sip.callStatus == "active" via _wait_for_sip_answer. This can't be auto-detected safely (no way to tell "informative early media" from generic ringback without listening to it), so it needs to be an integrator-set flag per call/trunk rather than a new default heuristic — trunk/carrier knowledge of which ones do this lives on our side, not the SDK's.
Workarounds / Alternatives
Checked whether upgrading fixes this: it doesn't. Confirmed the same defer-to-active behavior is present through 1.6.9 (latest with release notes as of this report) — #6580 actually reinforces it further (anchors the detection timeout to active too, on top of the no-speech timer from #5848).
The only workaround we can see today is bypassing AMD entirely and reimplementing the classifier/timer orchestration ourselves outside the SDK so we can start listening from track subscription for specific trunks — a much larger lift than an opt-in flag, and it means losing the SDK's built-in AMD maintenance/improvements going forward.
Additional Context
- SIP mechanism: RFC 3261
183 Session Progress responses can include an SDP body and start early media before the call is answered — see LiveKit's own SIP handshake reference, step 2.2.
livekit-agents version: 1.6.4 in production; confirmed same behavior through 1.6.9.- Setup: outbound SIP calls via a self-hosted LiveKit server with a SIP proxy in front of direct carrier SBCs.
- Confirmed so far in Peru; we suspect the pattern isn't unique to one country/carrier given how common early-media announcements are across LATAM mobile networks, but we don't have hard confirmation elsewhere yet.
- Happy to provide a redacted SIP trace (signaling only, no RTP/audio) showing the
183+SDP → late/absent 200 OK timing if useful.
Feature Type
Would make my life easier
Feature Description
AMD (
livekit.agents.AMD) deliberately defers all audio processing — the no-speech timer (#5848) and, as of 1.6.9, the detection timeout too (#6187 / #6580) — untilsip.callStatus == "active". Both fixes treat all pre-answer audio uniformly as "ringback / carrier early media / dialtone" noise that should never influence classification, which is correct when early media is just a ringback tone.We run outbound voice AI calls through several Latin American mobile carriers, and we've confirmed that at least a couple of them (seen so far in Peru) send the ringing tone, system announcements, or the voicemail greeting itself via early media — SIP
183 Session Progresswith an SDP body — rather than only after a final200 OK. For calls that reach voicemail on these carriers, the200 OKeither arrives well after the greeting has already played, or the leg never produces a clean "answered" transition the way AMD expects. Deferring toactivein that situation doesn't protect the classifier from noise — it discards the only audio that was ever going to be available, so the call falls through asuncertain/no-answer instead of being correctly classified asmachine-vm.This is different from what #6187 fixed: that issue was about timer accounting (a
detection_timeoutarmed at track-subscription burning down during a long ring before a lateactiveanswer arrives — the useful audio there was always post-answer). In our case the useful audio is the pre-answer early media itself.We're not asking to change the default behavior — the ringback-only case #5848/#6187 fixed is the common case and shouldn't regress. We'd like an opt-in, e.g. a
process_early_media: bool(defaultFalse) onAMD.__init__orDetectionOptions, that — when set — startsstart_listening()(timers + transcript processing) from track subscription instead of gating behindsip.callStatus == "active"via_wait_for_sip_answer. This can't be auto-detected safely (no way to tell "informative early media" from generic ringback without listening to it), so it needs to be an integrator-set flag per call/trunk rather than a new default heuristic — trunk/carrier knowledge of which ones do this lives on our side, not the SDK's.Workarounds / Alternatives
Checked whether upgrading fixes this: it doesn't. Confirmed the same defer-to-
activebehavior is present through 1.6.9 (latest with release notes as of this report) — #6580 actually reinforces it further (anchors the detection timeout toactivetoo, on top of the no-speech timer from #5848).The only workaround we can see today is bypassing
AMDentirely and reimplementing the classifier/timer orchestration ourselves outside the SDK so we can start listening from track subscription for specific trunks — a much larger lift than an opt-in flag, and it means losing the SDK's built-in AMD maintenance/improvements going forward.Additional Context
183 Session Progressresponses can include an SDP body and start early media before the call is answered — see LiveKit's own SIP handshake reference, step 2.2.livekit-agentsversion: 1.6.4 in production; confirmed same behavior through 1.6.9.183+SDP → late/absent200 OKtiming if useful.