Skip to content

Number the samples between two labstream peers (seq/1) - #12

Draft
rednayan wants to merge 8 commits into
perf/quick-winsfrom
feat/seq
Draft

rednayan wants to merge 8 commits into
perf/quick-winsfrom
feat/seq

Conversation

@rednayan

Copy link
Copy Markdown
Owner

Stacked on #10 (which holds #9), with #11 merged in. The base is perf/quick-wins, so the diff shows #11 and the seq work. Merge order: #9, #10, #11, then this. After each merge, change the base of the next one.

Summary

The first labstream-only feature: seq/1, sequence numbers. Two labstream peers number their samples, and the inlet counts the samples that never arrived. liblsl ignores the feature on both sides. The spec is in docs/extensions.md.

  • Negotiation. The inlet offers seq/1 in the request header Labstream-Features. A labstream outlet answers it. liblsl reads only the header keys that it knows (src/tcp_server.cpp:605-633, src/data_receiver.cpp:215-253).
  • Numbers. One counter for the whole outlet. Each consumer queue keeps the number after its last sample, so its samples always have consecutive numbers.
  • Record. 9 bytes: tag 0xFE (not a sample tag, src/sample.h:19-20) and a little-endian u64. The outlet writes one record at the start of each connection, and one before each sample after a gap. A record does not count toward Max-Chunk-Length.
  • Count. The count stays across a reconnect to the same outlet, and starts again for a new outlet.
  • API (additive).
    • labstream: Inlet::lost() -> Option<u64>.
    • labstream-net: Inlet::samples_lost(), SampleQueue::with_first_number, SampleQueue::pop_many_numbered, Fanout::next_number, Config::features.
    • labstream-proto: the features and seq modules, FeedParams::to_wire_with, FeedClient::start_with, FeedClient::features, Accepted::to_wire_with.
    • None means unknown (for example a liblsl outlet), not zero.
  • Off switch. [labstream] Features = {} in the configuration file. On by default.

Tests

Live test Result
A full outlet queue 32 received + 99,968 lost = 100,000 pushed
A lost connection, continuous source received + lost = pushed, on each run
A lost connection, quiet stream 20 lost show at once, with no new sample
A new outlet after a restart the count starts again at 0
A request without the header plain samples, no record
  • The quiet-stream and new-outlet tests each fail without their fix.
  • Also: a model test of the queue numbers (39 seeds × 400 steps), records cut at each byte in both byte orders, the writer records, the configuration default, and each mix of offer and answer.
  • Gate: fmt, clippy -D warnings, doc -D warnings, 306 workspace tests.
  • Conformance is the same as main, with seq on and with Features = {}:
    • plain, --zero-stamps, --desc, --postproc 7 and selftest: 28/28;
    • --chunk 16: 26/28, and --sync: 24/28 (liblsl itself refuses the failing string cells);
    • refuse100, pylsl_check 5/5 and examples pass.
  • Header probe:
    • liblsl and main (the same code as v0.1.2 below crates/) give no answer line and a sample tag first;
    • seq answers seq/1 and starts the data with 0xFE;
    • with seq off, the answer is empty and the data starts with a sample tag.
  • Latency: no change. At 4000 Hz × 64, p50 is 73 µs on both builds, and CPU is 20.5 % against 20.7 %. docs/latency.md also notes that the order of the runs can move a result at 512 Hz.

For review

  • Config gets a new public field. docs/versioning.md counts only a removal or a signature change as breaking, so this PR treats the field as new API.
  • SPEC.md 8.7 still says that LSL has no loss signal. That is still true for plain LSL. SPEC.md changes only after this feature is accepted.

Not verified

  • CI (the account is locked for a billing problem), macOS, Windows and MSRV 1.75.
  • Two machines.

🤖 Generated with Claude Code

oracle/latency.c is one C program for the C ABI. It runs as an outlet or
as an inlet, and reads its own CLOCK_MONOTONIC. oracle/latency.py links
it to each library, runs each pair more than once, and makes a table.
Each row names the library that the loader picked. With --control, a
known delay of 1000 us must show in the table.

docs/latency.md gives the method and the first results for liblsl,
main, fix/network-robustness and perf/quick-wins. At 4000 Hz x 64
channels in chunks of 32, main uses 9 to 11 % CPU on each side. The
fix branch uses 2 to 3 %, near liblsl.
docs/extensions.md proposes the request header Labstream-Features. A
labstream outlet answers it, and liblsl ignores it on both sides. The
page gives the syntax, the rules, the byte where a feature can change
the framing, the first features (seq and shm), the rejected
alternatives, and the conformance plan. Nothing is implemented.

oracle/headerprobe.py sends a feed request with and without the header
to a real outlet. The answer headers and the test pattern must be the
same. It passes for liblsl and for labstream main. A probe with a known
header (Native-Byte-Order) shows a difference, so the probe can fail.
docs/extensions.md now gives the seq/1 feature in full: one counter for
the whole outlet, a 9-byte numbering record (tag 0xFE, little-endian
number), the count rules on the inlet, and what the inlet reports. The
configuration key labstream.Features turns a feature off.

labstream-proto gets two pure modules. features reads, writes, offers,
answers and agrees the Labstream-Features tokens. seq writes and reads
the record, decides when the outlet writes one, and counts the lost
samples. FeedParams::to_wire_with, FeedClient::start_with,
FeedClient::features and Accepted::to_wire_with add the header to the
handshake. With no tokens, each gives the same bytes as before.
An inlet offers seq/1 in the Labstream-Features header. A labstream
outlet answers it, and liblsl ignores it on both sides.

The outlet numbers each pushed sample with one counter. Each consumer
queue keeps the number after its last sample, so the samples in it
always have consecutive numbers. The writer puts a numbering record
before the first sample of a connection, and before each sample after
a gap. A blocking outlet does the same.

The inlet reads the records and counts the numbers that never arrived.
The count stays across a reconnect to the same outlet, and starts again
for a new outlet. Inlet::samples_lost in labstream-net and Inlet::lost
in labstream give it. The value is None when the connection does not
use seq/1.

The configuration key labstream.Features turns the feature off.

Tests: a model test of the queue numbers, records cut at each byte in
both byte orders, the records of the writer, and three live tests. In
the live tests, the received, lost and dropped samples add up to the
pushed samples, for a full outlet queue and for a lost connection.
docs/extensions.md now gives the status of each item of the conformance
plan. Each item passes on this branch: the header probe on liblsl, main
and seq outlets, the interop matrix with seq on and off, and the unit
and live tests.

oracle/headerprobe.py now accepts one Labstream-Features answer line
with offered tokens only. Without that line, the answer headers must be
the same, and the test pattern must be the same.
A run with seq last in each group showed a slower median at 512 Hz. With
seq first, seq and quick gave the same numbers. At 4000 Hz x 64 channels,
seq/1 does not change the latency or the CPU.
The outlet now writes the opening record at once, after the test
pattern and before any sample, in both the queued and the blocking
mode. A gap from a lost connection thus shows when the inlet connects
again, also on a quiet stream such as a marker stream. Before, it
showed only with the next sample.

New live tests: a_gap_shows_at_once_on_a_quiet_stream and
a_new_outlet_starts_a_new_count. Each fails without its fix (lost
Some(0) instead of Some(20), and Some(1067) instead of Some(0)).

oracle/headerprobe.py now also reads the first data byte. It must be a
sample tag, or the record tag 0xFE when the answer holds seq/1. A test
of the configuration default of labstream.Features is added.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant