[wagenet]

What

Vision mixer changes from exercising the producer switching workflow:

1. actual_transition_type reports morph when a take moved or re-cropped a shared
   source instead of dissolving.
2. Vision mixer state carries input_media_age_ms, so a frozen input can be told apart
   from a motionless one.
3. A take with an unknown transition name is refused before it changes anything, on the
   PiP path too (400, as between plain inputs).

Why

The take reported a move as a dissolve. The PiP-aware path animates pads, not
pictures: a source in both compositions moves and re-crops into its new box, and only
sources exclusive to one side cross-fade. Taking a four-up to that participant full
frame reads as a zoom-in; taking a punch-in back to the plain source reads as a zoom-out.
That is deliberate, but the path reported fade for every non-cut take, so a caller
asking for a dissolve was told it got one. Behaviour is unchanged; only the report is
accurate. A slide_* request on this path is still downgraded, and now reports morph
rather than fade when a source is shared. morph is a report only, not a request type.
An effect take (glitch, flash, whip) out of a PiP lays its envelope over the pad move and
reported the move; it now reports the effect, as the plain-input path does. A
transition name the mixer does not know is now refused on this path with the same error
as between plain inputs; before, it ran as a dissolve and was reported as one. On both
paths the refusal now comes before the take cancels fade-to-black, which left a black
program marked as not faded, so the next FTB press did nothing visible. The bundled
control page no longer calls a fade that ran as a morph a downgrade; a slide, push
or wipe that ran as one still reads as downgraded. Known gap: a take between two buses
that both show the same PiP still reports fade.

A dropped participant stays on air as a still frame. The compositor repeats an
input's last frame indefinitely, so a lost connection looks exactly like someone sitting
motionless and persists after the ingest session is reaped. input_media_age_ms gives
milliseconds since each input last delivered a frame:

"input_media_age_ms": [ 24, 24, 7727, 24, null ]
//                       ^live ^live  ^^^^ frozen 7.7s   ^never published

Stamping at the mixer's own sink pads covers every input type. It measures arrival, not
display: when an input's timestamps jump backwards (an encoder restarting its clock), the
mixer discards frames older than the one it holds, so the tile freezes for the length of
the jump while its age reads live. The age has to be read against the source's own frame
interval: a source that only sends frames when its picture changes (a web graphic on a
static page) ages for seconds while healthy. Every input ageing together means the
mixer's output stalled, not that every source stopped: the mixer takes in frames only as
fast as it puts them out. The guide and the field doc say all three. On a five-seat WHIP rig, killing one publisher
produced 1.8 s → 29.9 s on that input while three live neighbours stayed under 30 ms, and
the age returned to 25 ms on rejoin. That rig run predates the take fix; the probe code is unchanged since.

Risk the reviewer should weigh

These are BUFFER probes, the hottest path in the pipeline. The callback reads the
monotonic clock and stores one relaxed atomic: no lock, no allocation, no formatting, no
state lookup. The overlay state is captured as an Arc, not looked up per buffer, and is
not a GStreamer object, so the closure forms no reference cycle with the pipeline. Five
inputs at 30 fps is 150 stores per second. The cost was not A/B measured.

Tests

Each guard below fails when its behaviour is reverted (the revert in brackets).

• take.rs unit tests: moved or re-cropped shared source → morph; disjoint, or shared
  and still → fade [classifier ignoring moves; ignoring crop].
• vision_mixer/tests.rs: age accessors, and the probe helper on a bare compositor
  [no stamp; wrong pad names].
• vision_mixer_signals_test (real CPU mixer): live / stalled / unlinked ages [CPU builder
  not installing probes]; punch-in to its plain source reports morph [response hardcoding
  fade]; morph requested out of a PiP is refused, buses and fade-to-black unchanged
  [unknown name run as a fade; FTB cancelled before the check]; the same refusal between
  plain inputs keeps fade-to-black [FTB cancelled before the check].
• vision_mixer_fx_test (real GPU mixer): glitch_cut out of a punch-in reports
  glitch_cut [effect name dropped]; live / stalled / unlinked ages, waiting up to 10 s
  for the stalled input to age [GPU builder not installing probes]. These render through GL; CI runs them on macOS and on Linux
  under Xvfb, both with STROM_REQUIRE_GL=1.
• backend/static/tests/vision_mixer_take_note.test.js: the page's take status note
  [every morph called a plain move; a punch-in described as a slide].

Run on macOS: cargo test --lib (690), vision_mixer_signals_test and
vision_mixer_fx_test with STROM_REQUIRE_GL=1 (5 parallel runs each under
taskpolicy -b), openapi_test, node --test backend/static/tests, cargo fmt --check, cargo clippy --all-targets -D warnings. The OpenAPI snapshot changed intentionally: one additive optional field and the
actual_transition_type description.

Not run locally: the rest of the integration suite. CI on Linux runs it.

Known gap, not from this PR: two flows whose mixers share a block id read each other's
state, including these ages (#887).

🤖 Generated with Claude Code

[srperens]

Noted as a draft, so I am holding off on a full review until it is marked ready for
review. If you want one before then, say so here or request me as a reviewer.

[srperens]

Verdict: Comment. The mechanism is right and the BUFFER probe meets the CLAUDE.md constraints. What holds this back from Approve is CI: this repo has never run the two GPU-path guards. Both skip green on Linux, and the macOS job that would render them did not run.

Requested changes

1. Maintainer: add the ci:macos label so that vision_mixer_fx_test renders at this head. The alternative is to merge #809 first, which makes the Linux job render GL on every PR, and then rebase this one. Either way, gpu_mixer_reports_media_age_per_input and master_fx_take_out_of_a_pip_reports_the_effect then execute instead of returning early.

Claims

Claim	Verdict	Evidence
The probe does no lock, allocation or formatting per buffer	CONFIRMED	The probe body (see Diagnosis) calls only backend/src/blocks/builtin/vision_mixer/overlay.rs:244 — let ms = self.instant_base.elapsed().as_millis() as u64; plus one relaxed store. It reads the clock through Instant, not SystemTime
The closure forms no reference cycle	CONFIRMED	It captures only an Arc<VisionMixerOverlayState>, and that struct has no gst:: field. The setup closure keeps the existing dist_weak.upgrade() pattern
An unknown name is refused before FTB is cancelled	CONFIRMED	backend/src/gst/pipeline/effects/take.rs:76 — `let trans_type = transition_type.parse::().map_err(
The GPU builder installs the probes, and a master-FX take out of a PiP reports the effect	UNVERIFIED	backend/tests/vision_mixer_fx_test.rs:712 — async fn gpu_mixer_reports_media_age_per_input() { and :626 both return early when there is no GL. In job 108793044875 the whole file reports 4 passed ... finished in 0.06s, which is the skip signature. .github/workflows/ci.yml:689 — STROM_REQUIRE_GL: 1 is set on the macOS job only, and Build (macOS) is skipping

Diagnosis. The probe is installed per input:

backend/src/blocks/builtin/vision_mixer/activity.rs:40 — pad.add_probe(gst::PadProbeType::BUFFER, move |_pad, _info| {

Coverage is ABSOLUTE for arrival: the stamp sits on the dist compositor's sink_i, so every input type is covered, including hidden ones. Frames the compositor discards after a backwards timestamp jump are the one blind spot, and the PR body and the field doc both state it.

Radius. SHARED. Every vision mixer in every flow now carries num_inputs BUFFER probes, on both builders. actual_transition_type has also gained a new value. A client that flags a downgrade whenever actual != requested will now flag every PiP fade that ran as morph. The bundled page was updated for this. External clients were not, and nothing in this repo can find them.

Overlaps

• #806 changes the same prologue of trigger_transition. It adds an index check that also runs before the FTB cancel, and it rewrites the old_pgm/from_input fallback right next to the parse block this PR removes. The two will conflict textually. #806 is older and its semantics are independent, so land it first and rebase this one.
• #809: see requested change 1.
• #887 (registry keyed by block id) is the known gap the body names. This PR does not settle it.

Tests & CI. At 2993a45, Check (Linux), both Build (Linux) jobs, WASM, API Contract Check and Static JS Tests are green. The CPU guards executed: media_age_separates_live_stalled_and_unlinked_inputs, punch_in_taken_to_plain_input_reports_morph, the four take::tests, and both vision_mixer::tests probe tests.

Confidence: HIGH

[srperens]

Supersedes my review of 2026-09-29 at 2993a45. That review's verdict was Comment. It is now Approve, because the blocker it raised has been resolved.

Verdict: Approve. At 8e02af3 this is a rebase onto dcccdd7 plus one test-only commit. That commit adapts the guards to the per-flow overlay registry (#1015) and to either trigger_transition index signature. The mechanism is unchanged from what I confirmed last time, and the GPU guards that I left UNVERIFIED have now executed in this repo's CI.

Claims

Claim	Verdict	Evidence
The probe does no lock, allocation or formatting per buffer	CONFIRMED	backend/src/blocks/builtin/vision_mixer/activity.rs:40 — pad.add_probe(gst::PadProbeType::BUFFER, move calls only backend/src/blocks/builtin/vision_mixer/overlay.rs:271 — let ms = self.instant_base.elapsed().as_millis() as u64; and one relaxed store. The captured Arc<VisionMixerOverlayState> still has no gst:: field, so it forms no reference cycle.
An unknown name is refused before FTB is cancelled	CONFIRMED	backend/src/gst/pipeline/effects/take.rs:76 — `let trans_type = transition_type.parse::().map_err(
The GPU builder installs the probes, and a master-FX take out of a PiP reports the effect	CONFIRMED	#977 made the Linux job render GL: .github/workflows/ci.yml:174 — STROM_REQUIRE_GL: 1. With that variable set, gl_available asserts instead of skipping: backend/tests/common/mod.rs:165 — strom_types::env::var_opt("STROM_REQUIRE_GL").is_none(),. In Check (Linux) job 113912231249, master_fx_take_out_of_a_pip_reports_the_effect ... ok and gpu_mixer_reports_media_age_per_input ... ok ran, and the file took finished in 4.97s instead of showing the old 0.06 s skip signature.

Radius. SHARED, unchanged from my last review. Every vision mixer gets num_inputs BUFFER probes, on both builders, and actual_transition_type gains the value morph. The bundled page handles morph. External clients cannot be found from this repo.

Overlaps

• #806: the idx() helper (backend/tests/vision_mixer_fx_test.rs:573 — fn idx<T: From<usize>>(i: usize) -> T {) compiles against either signature, and git merge-tree of the two heads is clean. Either can land first.
• #809, #1056: textual conflicts only. Against #809, pipeline_gpu.rs has adjacent let lines. Against #1056, vision-mixer.html has the version string. Whichever PR lands second resolves its conflict in one line.
• #887, which the body lists as a known gap, has been closed by #1015, and this head already tests against it. The PR body's last line is stale.

Tests & CI. At 8e02af3, Check (Linux), both Build (Linux) jobs, WASM, API Contract Check and Static JS Tests all pass. These guards executed: the four take::tests, both vision_mixer::tests probe tests, all four tests in vision_mixer_signals_test, and both new GPU tests. Build (macOS) and Build (Windows) were skipped, but the diff contains no platform-specific code. The base is 7 commits behind main. git merge-tree against origin/main is clean, and the only change main makes to openapi.json is the version line, which is disjoint from this diff's schema hunk.

Confidence: HIGH

[srperens]

Since this PR was opened, main gained shared test setup in backend/tests/common (#935, #936, #1083). After those, no test on main builds an AppState or a PipelineManager by hand except port_pool_test.

This PR adds three hand-built PipelineManagers with the same arguments as common::manager::build:

• backend/tests/vision_mixer_fx_test.rs:607 and :729. The rest of vision_mixer_fx_test.rs on main already uses the helper.
• backend/tests/vision_mixer_signals_test.rs:111

Each becomes let mut manager = common::manager::build(&flow).expect("build GPU pipeline"); (build(flow) in vision_mixer_signals_test, where flow is already a reference).

• common::manager::build(&flow) builds a PipelineManager the way start_flow does (EventBroadcaster::with_capacity(10), which equals new(10, false, false), media under the system temp dir). common::manager::build_with(&flow, events, media_path) takes your own broadcaster or media path. Neither calls detect_gpu_capabilities(), so keep that call where you have it. The registry temp file is not needed after construction.

Correction: I wrote here that this compiles against main as it is. It did not: #935 removed the imports those copies use, so vision_mixer_fx_test failed to build on a merge with main even though git reported no conflict. A maintainer has rebased this branch onto main and folded all three copies into common::manager::build in cf48e776, so nothing is left for you to do here.

Posted by Claude (AI) for the maintainers.