fix(efpsrt): carry embedded data end to end, with per-track stream routing, on efp v0.4.0

[srperens]

Repins gst-plugin-efp to v0.4.0, removes the data-track limit that #700 put
in as a stopgap, and fixes the embedded-data channel, which carried nothing in
a real flow. Follow-up to #700 and Eyevinn/efp#1.

The channel was dead in a real pipeline

Writing the wire round trip #691 asked for turned up a defect in the code #700
already merged. efpsrt_output's build linked data_input_<i> to the
embed_%u pad it had just requested. gst_bin_add drops any link whose peer is
outside the bin, and the pipeline builder adds every element a block returns
before it links anything (gst/pipeline/construction.rs:179, then
gst/pipeline/linking.rs), so that link was gone by the time data flowed and
the source feeding data_in_0 stopped with not-linked.

Plain GStreamer, nothing EFP-specific — funnel and identity reproduce it:

after link, before add:      mux:funnelpad0
after adding mux only:       NONE

Video and audio were never affected: they link from caps probes, by which time
everything is in the pipeline. Only the data chain linked eagerly, which is the
thing #700's own body flagged as worth review. Requesting the pad eagerly is
fine; linking eagerly is not.

The fix reports the link through internal_links and lets the pipeline builder
make it. The pad is still requested in build, so it exists with a
deterministic name for linking.rs's static_pad lookup to find.

Nothing caught this because every test in efpsrt_embedded_data_test.rs
inspects build's output.
output_block_requests_and_links_an_embed_pad_per_data_track asserted the src
pad's peer straight out of the builder — true, and meaningless, since it
described a link that existed only for the length of the call. It now asserts
the reported link, and its doc comment says why the old form passed while the
channel was dead.

What changed upstream

efpdemux now publishes one embedded_<stream-id> src pad per EFP stream that
carries data, with stream-id on the caps. At v0.3.0 it cached a single
embedded pad and returned it for every stream and data type, and its caps
carried no stream-id at all — the channel was write-only.

That was the whole reason #700 rejected num_data_tracks > 1 on the input
block: publishing data_out_1 would have advertised an output that no buffer
could ever reach. The limit is now obsolete, so it goes, along with the
get_external_pads clamp that hid the extra pads.

Track numbering: data_stream_ids

Data outputs were filled in arrival order, so which sender stream reached
data_out_0 could differ between runs of the same flow. data_stream_ids takes
one EFP stream ID per data track ("2,1"), so a track means the same thing every
time. Empty is the default and keeps arrival-order filling, so no existing flow
changes.

A list that does not name exactly one stream per track is rejected at build, as
are stream 0 (reserved, never carries media), duplicates, and anything that is
not a 0-255 integer. Ignoring a malformed list would put back the surprise the
property exists to remove. Pinned tracks match before unpinned ones, so a mixed
configuration cannot have an unpinned track swallow a pad another track was
configured to receive.

This is only possible because v0.4.0 puts stream-id on the embedded pad caps;
at v0.3.0 there was nothing to match on.

A description that had become wrong

#700's output-side description recorded v0.3.0 behaviour:

Both fields default to 0 when omitted rather than failing, and stream-id 0 is
reserved: data addressed to it […] is buffered by the muxer and never sent.

v0.4.0 rejects caps missing either field and drops data addressed to a stream
that carries no media, so that text would now actively mislead. Corrected rather
than extended.

This is a behaviour change for operators. A flow whose data source sends
incomplete application/x-efp-embedded caps used to get silence and an
unbounded buffer inside efpmux; it now gets a pipeline error. That is the point
of the upstream fix, but it is visible.

Tests

Ran locally on macOS against the newly tagged v0.4.0:

• cargo test --features efp --test efpsrt_embedded_data_test — 9 passed.
• cargo test --features efp — full backend suite green, 574 unit tests plus
  every integration target.
• cargo fmt --all --check and cargo clippy --features efp --all-targets -D warnings clean.

backend/tests/efpsrt_data_roundtrip_test.rs is new and is the guard for the
fix above. It drives EfpSrtOutputBuilder and EfpSrtInputBuilder, installs
each into a pipeline the way construction.rs does — add every element, then
apply internal_links — joins them over a real SRT connection on loopback, and
reads data_out_0. It asserts the bytes match and that the pad caps carry
data-type and stream-id.

It ran in CI on Linux, not skipped:

Running tests/efpsrt_data_roundtrip_test.rs
test embedded_data_survives_the_trip_between_the_blocks ... ok

Reverting only the fix, with the test untouched:

tx pipeline error: Internal data stream error.
  gst_base_src_loop (): /GstPipeline:tx/GstAppSrc:appsrc0:
  streaming stopped, reason not-linked (-1)

The two tests that guarded the removed limit are replaced by their opposites:
input_block_builds_a_data_output_per_track_beyond_the_first builds three data
outputs, and input_block_advertises_every_data_output_it_builds asserts all
three reach the flow graph. Both fail against v0.3.0's behaviour, which is
what makes the repin load-bearing rather than cosmetic.

data_stream_ids_pins_each_track_to_its_sender_stream is the guard for the
routing: two media streams each carrying their own embedded data, over SRT, with
a deliberately reversed list, so arrival order cannot produce the expected
result by luck. It does not depend on which media track the sender numbers 1 —
it asserts how the receiver routes streams to outputs, not how the sender
assigns them. Validation is covered through the real builder rather than the
parser in isolation.

Not verified: Windows and macOS. The macOS numbers are from a local run;
CI covers Linux only, as it does for every EFP change here.

Pin

Cargo.lock resolves v0.4.0 to d6fd3bdb05195a48fffab2f38ed88ec4037f373c,
the tagged commit.

🤖 Generated with Claude Code

[srperens]

The round trip #691 asked for is now written and run — and it found that the channel #700 shipped carried nothing in a real flow. Pushed as bd399dd on this branch.

The bug

efpsrt_output's build requested an embed_%u pad from efpmux and linked data_input_<i> to it right there. gst_bin_add drops any link whose peer is outside the bin, and the pipeline builder adds every element a block returns before it links anything (gst/pipeline/construction.rs:179, then gst/pipeline/linking.rs). So the link was gone by the time data flowed, and the source feeding data_in_0 stopped with not-linked.

It is plain GStreamer, not anything EFP-specific. With funnel and identity, linking to a request pad while both elements are parentless and then adding either one to a pipeline:

after link, before add:      mux:funnelpad0
after adding mux only:       NONE

Video and audio were never affected — they link from caps probes, by which time everything is already in the pipeline. Only the data chain linked eagerly, which is exactly the thing #700's body flagged as worth review ("the embed pad is requested eagerly in build(), not from a caps probe"). Requesting eagerly is fine. Linking eagerly is not.

Fix: report the link through internal_links and let the pipeline builder make it. The pad is still requested in build so it exists with a deterministic name, which is what linking.rs's static_pad lookup then finds.

Why nothing caught it

output_block_requests_and_links_an_embed_pad_per_data_track asserted the src pad's peer straight out of build. That was true, and meaningless — it described a link that existed for the length of the builder and no longer. Every other test in that file inspects build's output too.

This is the failure mode CLAUDE.md names: a test has to exercise the code it guards. Asserting graph structure at build time is not the same as asserting the graph the pipeline ends up with. The test is rewritten to assert the reported link, and says in its own doc comment why the old form passed while the channel was dead.

The new test

backend/tests/efpsrt_data_roundtrip_test.rs drives EfpSrtOutputBuilder and EfpSrtInputBuilder, installs each into a pipeline the way construction.rs does — add every element, then apply internal_links — joins them over a real SRT connection on loopback, and reads data_out_0:

test embedded_data_survives_the_trip_between_the_blocks ... ok

It asserts the bytes match and that the output pad caps carry data-type and stream-id, the field gst-plugin-efp v0.4.0 added.

Reverting just the fix, with the test untouched:

tx pipeline error: Internal data stream error.
  gst_base_src_loop (): /GstPipeline:tx/GstAppSrc:appsrc0:
  streaming stopped, reason not-linked (-1)

So it is a guard, not a demonstration.

Two things the test had to pin, because a caller cannot see them

Both are properties of efpmux that the block's shape does not reveal, and both are now constants with comments in the test:

• Embedded data is addressed to a media stream. IDs are allocated from 1 as sink pads are requested, so with no video and one audio track, audio is stream 1.
• It only leaves the muxer on a frame of that stream. Data addressed anywhere else is dropped by v0.4.0 — silently buffered forever by v0.3.0.

Full run

cargo test --features efp green: 574 unit tests plus every integration target, including the new one. fmt and clippy --features efp --all-targets -D warnings clean. macOS local; CI covers Linux.

@borisasadanin — this is the check you offered to run on #691, and it did not need a deployed instance after all. It also means the answer to "does the embedded channel work end to end" was no until this commit, on code that was already merged. Worth knowing before you build on it.