feat(mixer): add per-channel direct outputs

[wagenet]

Opened by Claude (AI) on Peter Wagenet's behalf.

The design here is flexible, and we're open to other approaches and to any feedback. The points we'd most like a view on are listed under "Open for discussion".

Problem

The mixer has no per-channel output. Anything outside it that needs a channel as it is actually mixed can only tap the channel's source, before the mixer, and copy the operator's fader, mute and audio-follow-video (ch{N}_to_main) changes onto its own gain. That copying cannot be made correct from outside the mixer:

• Writes to the mixer and to the copy can be answered out of order, lost, or refused separately, so the two drift apart.
• Reading the mixer back to resynchronise returns live element values, not the operator's settings. A muted channel's fader reads 0, because mute ramps volume to 0. A channel whose source has stalled reads a stale value, because a controlled property only updates while buffers flow.

The case that prompted this is a guest return: a mix-minus (each guest hears everyone but themselves) built in a separate, low-latency flow, because every sink in the program flow runs at the program flow's latency. Eyevinn/open-live#386 builds it today by tapping each channel's source into a cross-flow audio sink and summing those in a liveaudiorouter, then mirroring every mixer change into the router's gains. The cross-flow hop can be interaudiosink or the adaptive bridge proposed in #844. This PR does not depend on #844, and the direct outs work with either.

Change

• New property and pads: an opt-in direct_outs property on builtin.mixer (Bool, default false, construction-time). When on, each channel gets an output pad direct_out_N (label DN). It carries exactly what that channel sends to Main: after its fader, mute and to-Main switch, before the Main sum and the Main processing.

  routing_tee_N → to_main_vol_N → direct_tee_N → to_main_queue_N → main audiomixer
                                  direct_tee_N → queue (leaky) → direct_out_tee_N → direct_out_N
  

• No change for existing flows: with the property off, the mixer builds exactly what it builds today.

• Nothing to mirror: the same volume elements shape Main and the direct out, so a mute or to-Main change fades identically on both. The router becomes a fixed mix-minus matrix.

• Consumers cannot block Main: the direct branch has a leaky queue holding 3 s. A consumer that stops pulling drops direct-out audio instead of blocking the channel and silencing it on Main. An unlinked direct out is ignored, like the other outputs.

• Mixer editor save fix: the fullscreen mixer editor replaces the block's properties wholesale on save, so it dropped construction-time settings it has no controls for. That includes dsp_backend, force_live, latency, internal_bus_latency, min_upstream_latency and sample_rate today, and would include direct_outs. A save reset each to its default: a mixer set to 44.1 kHz came back at 48 kHz. Strom then prunes links to pads that no longer exist, so a save would also quietly remove every direct-out link. The editor now carries those keys through saves.

• Docs: a section in the operator guide.

Open for discussion

• The name. direct_outs / direct_out_N follows console vocabulary. But this tap is not a hardware console's direct out (next point), so a different name may read better.
• Where the tap sits. On an X32 or similar, a direct out taps the channel at a selectable point (input, pre-EQ, post-EQ, pre-fader, post-fader) and ignores the channel's bus assignments. Here the to-Main switch applies, because that switch is how audio-follow-video works. As a result, a channel routed to a group but not to Main reaches program through the group, yet is silent on its direct out. It also limits the other common use of a direct out, recording each channel on its own track, which usually wants a pre-fader feed. This is documented for now. A direct_out_tap enum (post_route default, post_fader, pre_fader) could be added later without breaking anything.
• The leaky queue's size. It holds 3 s, with no buffer or byte limit. A consumer that syncs to the clock (the Inter Output block does by default) holds each buffer for the flow's latency, so the queue must hold at least that much. At the default limits (1 s, 200 buffers), a synced consumer in a flow at 1.5 s latency lost 330 of 384 10 ms slices, and one at 400 ms with 1 ms buffers lost 355 of 386. The cost is that a consumer that stalls and then resumes first receives up to 3 s of old audio. A smaller queue would bound that but would limit the flow latency a synced consumer can work at.
• A consumer that ends. The queue does not protect against a consumer that ends mid-stream (returns EOS), or one torn down while the flow runs (FLUSHING). Either result travels back through the tee to the channel's source and stops the channel on Main, immediately and without an error. Measured for both: the tone on Main stopped at once. Aux outs already do the same, about 300 ms to 1 s later. That is tracked for every mixer output in Audio mixer: an output whose consumer ends silences its channels on Main #933, not fixed here.
• A channel with no input. A direct out carries nothing until its channel has input, while the other mixer outputs carry silence (with force_live on, the default). An output that waits for its first audio before starting gets nothing until a source reaches that channel, and the flow reports PAUSED rather than PLAYING until then. Main and the other outputs keep playing: with interaudiosink on an empty direct out, an AES67 output on Main sent 2,021 packets in 2 s while the flow reported PAUSED, and the flow reached PLAYING about 65 ms after a source joined. Inter, AES67, Recorder, SRT, RTMP, WHEP and WHIP outputs do not wait. The NDI, DeckLink audio and TAMS outputs do (draft fix(outputs): start NDI, DeckLink audio and TAMS outputs without waiting for data #934 changes them), as does any raw sink left with async on. The property description and the operator guide say so. The alternative fix is to have the mixer send silence on an empty direct out. The simple way to do that adds a live mixer per channel, which adds delay to the return this PR is for.

Latency

Tapping the direct outs adds nothing measurable to the fast return. Main's 360 ms pipeline latency does not reach it.

• Setup: loopback, five synthetic WHIP seats (Opus + H.264), TRL-shaped program flow, conversation flow as in open-live#386 (bridge target 40 ms, router 15/10/5). One fresh server per trial, arms alternated, 4 trials each, load average 4–8.
• Result: mean tick delay into seat 0's fast feed was 184 ms through the direct outs and 190 ms tapped at the source. Trial means were 178–190 ms and 178–197 ms. Dropout was 0.12–0.21 % in both arms.
• Why nothing adds latency: with the default rust DSP backend, nothing in the channel strip holds audio. Of the lsp-rs elements only the limiter has lookahead, and it sits on the Main bus. The tap is upstream of the Main audiomixer.
• Build caveat: the rig needs WHIP fixes that upstream/main lacks (on plain upstream/main + feat(audio): adaptive audio bridge for the conversation return #844, every output was silent in both arms). It ran on our integration branch (main plus open PRs including feat(audio): adaptive audio bridge for the conversation return #844, and fix(mixer): run every bus at one sample rate so late inputs can link #919, whose fix has since merged as part of fix(audio): resample mixer and router inputs to one configurable sample rate #946) with the first commit on top. The later queue-size change cannot affect that measurement: the bridge does not wait on the clock, so the queue never fills.

Tests

New tests in mixer/tests.rs, each with its own block id:

Test	Covers	Revert that makes it fail
test_direct_outs_exist_only_when_enabled	Pads and elements only when the property is on; the tap is after the to-Main switch	Always add the pads
test_direct_out_carries_only_its_channel	A tone on input 1 appears on direct_out_1, silence on direct_out_2, resolved through the external pad mapping	Map every pad to channel 1's tee
test_direct_out_follows_mute_and_to_main_ramps	On a running mixer, a mute, an unmute and to-Main off, driven through VolumeRampManager (as the property API does). Each fades on the direct out and matches Main bucket by bucket	Tap before to-Main: differs 570 ms in a row. Tap pre-fader: differs 880 ms in a row
test_stalled_direct_out_consumer_does_not_starve_main	A consumer that stops pulling mid-stream leaves Main intact for 4.5 s, past the queue's 3 s	No queue: Main loses the tone for 4320 ms. Non-leaky queue: 1360 ms
test_direct_out_feeds_a_synced_consumer_at_high_latency	A clock-synced consumer in a flow at 1.5 s latency receives every buffer	Default queue limits: 292 of 306 10 ms slices lost

In frontend/src/mixer/state.rs, saves_keep_build_config_the_editor_does_not_edit fails when the carry-through is removed, or when sample_rate or internal_bus_latency is left out of it.

The ramp and stall tests judge by the longest contiguous run, not single 10 ms buckets. On a loaded host, Main's aggregator drops a bucket here and there with the feature off too.

Ran (macOS, GStreamer 1.28, STROM_REQUIRE_GST_PLUGINS=1 so a missing element fails instead of skipping):

• cargo fmt --all and cargo clippy --workspace --all-targets -- -D warnings at the head.
• cargo test --workspace at the head: 1,241 passed, 0 failed, 6 ignored, including openapi_test.
• Each revert in the table above, on the rebased branch: every one fails its test.
• The direct-out tests under taskpolicy -b (background priority, which mimics a slow CI runner), feeding 48 kHz against 44.1 kHz, alternated 8 times each: 16 of 16 passed, no difference between the two. In one earlier run at load average above 20, the ramp, stall and synced-consumer tests failed in 2 of 5 iterations; that did not recur in 22 runs at lower load.
• A scratch test (not added) with the mixer at 8, 44.1, 48, 96 and 192 kHz and inputs at another rate: each direct out carries the mixer's rate, like Main, and a 48 kHz consumer links to it.

Not run:

• The LV2 DSP backend (whether its gate or compressor add lookahead on the direct out).
• A real guest through Chrome.
• The mixer editor UI by hand.

🤖 Generated with Claude Code

[srperens]

Noted as a draft, so I am holding off on a full review until it is marked ready for review. If you want one before then, say so here or request me as a reviewer.

[wagenet]

Claude (AI), on behalf of @wagenet.

Rebased onto main. Conflicts were with #946 (one sample rate for every mixer bus):

• builder.rs: kept both sample_rate and direct_outs.
• tests.rs: kept fix(audio): resample mixer and router inputs to one configurable sample rate #946's tests and the direct-out tests. The direct-out feed helper fed 44.1 kHz because idle buses used to settle there. After fix(audio): resample mixer and router inputs to one configurable sample rate #946 the buses run at sample_rate, so it now feeds DEFAULT_AUDIO_SAMPLE_RATE and the channel resampler passes it through.
• The mixer editor's BUILD_CONFIG_KEYS now includes fix(audio): resample mixer and router inputs to one configurable sample rate #946's sample_rate. Without it, saving from the editor dropped the property and the mixer rebuilt at 48 kHz. The frontend test covers it.

Ran: cargo fmt --all -- --check, cargo clippy --workspace --all-targets -- -D warnings, wasm32 clippy on the frontend, cargo test --lib blocks::builtin::mixer (37 passed, 1 ignored), and the frontend mixer tests (saves_keep_build_config_the_editor_does_not_edit passed).

[srperens]

Verdict: Comment. I found no defect: the code, the tests and the queue override all hold up. What remains is a decision only a maintainer can make. The body asks for a view on the name and the tap point, and both become flow contract on merge: direct_outs and direct_out_N get persisted in saved flows and links.

For the maintainer to choose

1. Name / tap point. As built, the tap is post-route: after to-Main, so a group-only channel is silent on it. That is the right tap for this PR's purpose, a mix-minus that follows audio-follow-video. A console engineer reading "direct out" will expect a pre/post-fader tap that ignores routing. Either keep the name and later add a direct_out_tap enum (post_route default), which is additive, or rename now (e.g. main_sends / main_send_N) while nothing persists it. Renaming after merge is a breaking change to saved flows.
2. Consumer EOS (#933). With direct outs on, a consumer that ends silences its channel on Main at once, where today's aux outs take ~1 s. Disclosed; #933 owns the fix. Landing this first makes the failure faster on an opt-in path. Waiting for #933 is a legitimate alternative.

Claims

Claim	Verdict	Evidence
Off by default; with it off the graph is unchanged	CONFIRMED	pads only under backend/src/blocks/builtin/mixer/builder.rs:101 — if get_bool_prop(properties, "direct_outs", false) {. The else arm keeps the original to_main_vol → to_main_queue link
The tap is after to-Main, so Main and the direct out share one ramp	CONFIRMED	backend/src/blocks/builtin/mixer/builder.rs:1264 — ElementPadRef::pad(&direct_tee_id, "sink"), linked from to_main_vol's src
The queue override is justified	CONFIRMED	CLAUDE.md "GStreamer Queues" allows overrides with a documented latency requirement. The comment above backend/src/blocks/builtin/mixer/builder.rs:1237 — .property("max-size-time", 3 * gst::ClockTime::SECOND.nseconds()) documents one, and test_direct_out_feeds_a_synced_consumer_at_high_latency guards it
The editor save fix covers every build-time key it does not edit	CONFIRMED	frontend/src/mixer/state.rs:5 — const BUILD_CONFIG_KEYS: [&str; 7] = [, merged into both save paths (lines 375 and 638). Of the live: false keys in definition.rs, the rest are num_* (written by the editor) and per-channel label/aux-pre/HPF (loaded and written by the editor)

Diagnosis: Tapping inside the mixer is the only way to get "what this channel sends to Main" without mirroring, and the body shows why mirroring cannot converge. The leaky queue makes the loop OPEN on the healthy path: an unlinked or stalled branch drops, so Main costs nothing extra. The empty-channel case reports PAUSED for outputs that wait for their first audio. It is documented, and #934 removes it for NDI, DeckLink audio and TAMS.

Radius: LOCAL. The feature is opt-in in one block. The editor save fix changes every mixer save, but it only stops resetting keys that were previously dropped silently, and its test fails when the fix is removed.

Overlaps: #933 (above). #934 removes the empty-channel stall for three outputs. #898 drops input EOS at convert_N, upstream of this tap. A local trial merge of main + this + #898 is clean (not built). Once both land, a direct out on a force-live mixer never sees an ended input's EOS, which is consistent with that block not ending.

Tests & CI: Check (Linux) at b21fc6a ran all five test_direct_out* tests and saves_keep_build_config_the_editor_does_not_edit, all ok. It predates #971; the trial merge above includes it. The author saw the ramp/stall tests fail 2/5 at load above 20, so watch for flakes on CI. macOS/Windows skipped: nothing here is platform-specific.

Confidence: HIGH

[wagenet]

Claude (AI), on behalf of @wagenet.

Rebased onto main (dcccdd7) to clear the merge conflict. No behaviour change.

• mixer/tests.rs: both sides appended tests at the end of the file; kept both.
• Main added a feed(m, channel, live: bool) helper, which collided with ours. Ours is now feed_wave.
• The toolchain's new clippy::chunks_exact_to_as_chunks flagged the direct-out capture probe; it now uses as_chunks::<4>().

Both fixes are folded into the commits that introduced them, so each commit builds on its own (clippy -D warnings checked on the first two commits and the tip).

Ran: cargo fmt --all -- --check, cargo clippy --workspace --all-targets -- -D warnings, wasm32 clippy on the frontend, cargo test --lib blocks::builtin::mixer (43 passed, 2 ignored), cargo test --test openapi_test, and cargo test --workspace mixer (includes the frontend mixer tests). Not run: the full workspace test suite.

The open name and tap-point questions are untouched and still need a maintainer decision.

[srperens]

Verdict: Comment. This is a re-review at the new head, 61dd167. My review of 2026-10-05 at b21fc6a stands unchanged. The push only rebases the branch, and the maintainer decision that review asked for is still open.

What moved. git range-diff of the old and new branches shows the five commits unchanged apart from three things:

• the test helper feed is renamed to feed_wave, because main added its own feed;
• the capture probe uses as_chunks::<4>() for the new clippy lint;
• the operator guide's table context is updated.

No product code changed. The gate is still where it was: backend/src/blocks/builtin/mixer/builder.rs:101 — if get_bool_prop(properties, "direct_outs", false) {. So every claim row in my earlier review carries over without re-verification.

Still for the maintainer. Both choices from the earlier review are unanswered: the direct_outs name and its tap point, which become persisted flow contract on merge, and whether to wait for #933, which is still open.

Overlaps. #933 (open, owns the consumer-EOS behaviour), #934 and #898 are unchanged since the last review. The branch merges cleanly with current main (checked with git merge-tree).

Tests & CI. At 61dd167, Check (Linux) passes. All five direct-out tests are ok in that job's log, test_stalled_direct_out_consumer_does_not_starve_main among them, and so is saves_keep_build_config_the_editor_does_not_edit. The two ignored mixer tests are not part of this diff. Both Linux builds, WASM and API Contract pass. macOS and Windows were skipped, and nothing here is platform-specific.

Confidence: HIGH