fix(vision mixer): the PGM compositor no longer waits on the multiview to allocate (CPU)

[srperens]

Problem

vision_mixer_cpu_test::keyed_alpha::keyed_pads_keep_alpha_with_auto_output_format fails on the macOS runner, 1 to 3 times in 60 when the five keyed tests run in parallel: white program input never appeared in PGM within 20s. Seen on #1026's CI and in 180 runs from diagnostic branches. In isolation it passed 60 of 60.

A trace on the runner (GST_DEBUG=aggregator:5,…, kept only for the hung run) showed that the flow never starts. pgmsink stays READY with PAUSED pending and the pipeline never reaches PLAYING:

• The PGM compositor's src task sits in gst_aggregator_do_allocation from 0.39 s until teardown at 21 s (peer ALLOCATION query failed).
• tee_pgm forwards that query to every branch, including queue_pgm_mv into the multiview's PGM tile. There it waits until mv_comp's streaming thread takes it (gst_aggregator_default_sink_query_pre_queue: Waiting for query to be consumed).
• mv_comp's streaming thread is in its own second ALLOCATION query (a late input renegotiates it). That query waits in queue_mv_out, whose thread is held by the multiview sink. The sink holds a prerolled frame until the pipeline reaches PLAYING, and the pipeline only gets there once PGM has output.

Change

elements::answer_allocation_query answers ALLOCATION on queue_pgm_mv's sink pad (QUERY_DOWNSTREAM probe, Handled, nothing proposed). tee_pgm then takes its allocation from the PGM output branch, and the PGM compositor no longer waits on the multiview. The same queue already answers LATENCY itself (suppress_latency_query) for the same reason.

Software backend only. On the GPU backend the ALLOCATION query also carries the GL context and pools, and the hang was not seen there.

Rejected:

• Pinning the software compositors' output format for Auto. A first theory blamed the switch to GBRA_12LE. With AYUV pinned, mv_comp still renegotiated identical caps when a late input arrived, and the hang stayed (3 of 60).
• Changing the test's sources. That hides a start-up deadlock any flow can hit.

Evidence

• New guard pgm_allocation_does_not_wait_on_a_stalled_multiview: it holds mv_comp's streaming thread, renegotiates the PGM compositor, and requires PGM frames to keep coming. Without the probe it fails 3 of 3 (PGM stalled while the multiview was held: 1 frames in 2 s). With it, it passes.
• On the runner: 60 of 60 rounds of the keyed tests passed with the fix (run 37618452815), against 1 to 3 hangs in 60 without it.

Tests

Ran (macOS, GStreamer 1.28.6): cargo test --lib vision_mixer (37), vision_mixer_cpu_test (8), vision_mixer_underlay_upload_test, vision_mixer_fx_test, vision_mixer_overlay_upload_test, vision_mixer_source_resize_test, vision_mixer_cuda_*, vision_mixer_fx_passthrough_test, pipeline_lifecycle_test: all passed. cargo clippy --all-targets -D warnings clean.
Not run: Linux and Windows (CI).

For the reviewer

• The multiview's PGM tile no longer takes part in the PGM compositor's allocation, so PGM buffers are allocated for the output branch's needs only. The tile is fed through a leaky one-buffer queue and is composited by the CPU compositor, which converts what it gets.

🤖 Generated with Claude Code

[srperens]

Verdict: Comment. The diagnosis holds up and the probe is the right size. Two things are left for a human: a compile break with #935/#936 that neither diff shows, and a red Build (macOS) that I traced to the Media Player, not to this diff.

Requested changes

1. Set a landing order with #935 (and #936, which carries the same hunk). #935 changes build_cpu_manager to return a bare PipelineManager. This PR's new test destructures a tuple:
   backend/tests/vision_mixer_cpu_test.rs:529 — let (mut manager, _registry_file) = build_cpu_manager(&build_flow(block_id, ""));
   The two hunks are far apart, so git merges them cleanly and the test binary then fails to compile. Whichever PR lands second has to change that one line.

Claims

Claim	Verdict	Evidence
The probe answers only ALLOCATION, per query and never per buffer	CONFIRMED	QUERY_DOWNSTREAM probe on the sink pad; it captures nothing (see below)
The new test runs in CI	CONFIRMED	Check (Linux): keyed_alpha::pgm_allocation_does_not_wait_on_a_stalled_multiview ... ok, binary 8 passed in 4.31 s. Build (macOS): same test ok
The test fails 3 of 3 without the probe	UNVERIFIED	Author's local run. The mechanism agrees: the BLOCK probe on mv_comp's src stalls the only thread that could take the serialized query from queue_pgm_mv. But no run in this repo has reverted it
An empty, handled answer only drops the multiview branch from tee_pgm's allocation	EXTERNAL	Assumes tee merges each branch's answer and keeps only the metas every branch offers. An empty branch then removes GstVideoMeta and the rest from what the PGM compositor sees. The software compositor writes default strides, so I expect no correctness change. Any cost is downstream zero-copy only

backend/src/blocks/builtin/vision_mixer/elements.rs:274 — pad.add_probe(gst::PadProbeType::QUERY_DOWNSTREAM, |_pad, info| {
backend/src/blocks/builtin/vision_mixer/builder/pipeline_cpu.rs:139 — elements::answer_allocation_query(&queue_pgm_mv);

Diagnosis. The cycle (tee query → serialized query waiting in mv_comp's sink → mv_comp's own allocation stuck behind a prerolled sink → no PLAYING without PGM) explains a hang that appears only at start and only under parallel load. Breaking it at the branch that feeds an isolated tile is the right layer, and it mirrors suppress_latency_query on the same queue. Coverage is BOUNDED to the software backend. The GPU builder has the same tee_pgm → queue_pgm_mv → mv_comp shape without the probe:
backend/src/blocks/builtin/vision_mixer/builder/pipeline_gpu.rs:157 — elements::suppress_latency_query(&queue_pgm_mv);
SPECULATIVE (not verified): the same cycle can form on GL, where answering empty would also drop the GL context. A GL fix needs a different answer, not this probe.

Radius. LOCAL: one probe, on one queue, in one builder. The closure holds no pipeline object, and the probe lives and dies with the queue.

Overlaps. #935 and #936 own the test-helper signature (compile break above). #1010, #1020, #809 and #803 edit pipeline_cpu.rs in other hunks, and none touches queue_pgm_mv or tee_pgm.

Tests & CI. At 50a01a3: Linux, ARM64, WASM, API contract and JS are green. The red Build (macOS) job is media_player_control_test::a_storm_of_jumps_and_pauses_never_hangs_the_player ("the internal pipeline deadlocked (issue #963)"). That is the window #996 targets, and this diff does not touch the Media Player. Every vision-mixer binary in that job passed. A gh run rerun 37623484060 --failed would give a clean macOS row.

Confidence: HIGH