Skip to content

feat: disorder, out-of-band, Linux fake and the UDP burst on Linux and macOS - #6

Merged
Nemu-x merged 6 commits into
mainfrom
feat/cross-platform-methods
Oct 29, 2026
Merged

Nemu-x merged 6 commits into
mainfrom
feat/cross-platform-methods

Conversation

@Nemu-x

@Nemu-x Nemu-x commented Oct 29, 2026

Copy link
Copy Markdown
Owner

Stage 2 of docs/vnext.md: the transmission methods on Windows, Linux/Android and macOS, plus disorder and out-of-band from ByeDPI.

Method Windows Linux / Android macOS
Disorder (Options.Disorder) ✓ restore after SIO_TCP_INFO reports the piece sent (counters, not reliable) ✓ restore after SIOCOUTQNSD = 0 (reliable) ✓ Observed: false: written after SO_NWRITE = 0, transmission before the restore inferred
OOB / disoob (Options.OOB, + Disorder) ✓ WSASend + MSG_OOB ✓ ✓
Fake TCP (Options.Fake) ✓ existing ✓ new: vmsplice/splice page overwritten with the original before the hop limit is restored ✗ unsupported: no splice, sendfile copies into mbufs, so a retransmission would carry the fake bytes (xnu uipc_syscalls.c)
Low-TTL UDP burst (WrapPacketConn) ✓ existing ✓ new ✓ new
desync.Options{Split: []desync.Pos{{Offset: 1, FromSNI: true}}, Disorder: true}                         // disorder
desync.Options{Split: split, OOB: &desync.OOBOptions{}}                                                  // oob, byte 'a' by default
desync.Options{Split: split, Disorder: true, OOB: &desync.OOBOptions{}}                                  // disoob
desync.Options{TLSRec: rec, Fake: &desync.FakeOptions{TTL: 4, Prefix: -1}}                              // fake, now also Linux/Android
  • Disorder and OOB need Split or Random.Split and combine with TLSRec, Random, SplitDelay, SplitWaitSent. Validate rejects them with Fake, and Disorder with SplitWaitAcked (the dropped piece is acknowledged only after its retransmission). No TCP socket → ErrSocketMethodRequiresTCP before any byte; an OS without the method → ErrDisorderUnsupported / ErrOOBUnsupported.
  • The hop limit is restored on every path; a failed restore is reported and closes the connection. Waits are bounded by SplitWaitTimeout, the write deadline and Close.
  • Linux fake: MAP_SHARED page (a fork cannot keep a copy with the fake bytes), at most 64 operations at once (ErrFakeBusy), prefixes larger than a pipe in 64 KiB rounds.
  • UDP: on Linux ICMP errors reach an unconnected socket only with IP_RECVERR/IPV6_RECVERR, which the wrapper turns off; macOS never delivers them to unconnected sockets. On macOS dual-stack sockets the default hop limit is restored as its explicit value (restoring -1 would set the IPv4 TTL to 255).
  • PlatformCapabilities() gains Disorder{Supported, Mechanism, Observed, Reliable}, OOB and Unsupported (a reason for each missing method). Because of the map, Capabilities values can no longer be compared with ==.
  • Key() adds ;disorder=1 / ;oob=N only when set. desync-socks: -disorder, -oob, -oob-byte. ByeDPI credit in third_party/NOTICE.

Tests: server-side stream byte-identical for every method and combination (tcp4 and tcp6); with SO_OOBINLINE the server reads the urgent byte exactly at the split point; the hop limit recorded at every write and restored after injected failures, deadlines, timeouts and Close. On Linux a BPF filter on the server drops segments with a hop limit below 8: fake (own payload, template, with TLSRec), disorder and disoob all deliver the original stream after one retransmission; with the page overwrite disabled the server reads the fake bytes (negative control). UDP order/source port/TTL and ICMP tests on Linux.

Checks: gofmt, go mod tidy -diff, vet for windows/linux/darwin on Go 1.26.10 and 1.20.14; tests on Windows (both Go versions, new tests -count=20) and Linux in Docker (1.26.9, -race, -race -count=20 for the native tests, 1.20.14); cross builds incl. android/arm64, riscv64, freebsd, js/wasm. Not verified: macOS at runtime (this PR's CI runs it), Android on a device, low-TTL drops on Windows (loopback does not drop), real networks, middleboxes clearing URG. Fake combined with Split/Disorder/OOB is not implemented.

Options.Disorder sends the first TCP piece of a split ClientHello with hop
limit 1, restores the hop limit and sends the rest, so TCP retransmits the
first piece with the normal limit (ByeDPI --disorder). Options.OOB sends one
urgent (MSG_OOB) byte after the first piece (--oob); both together are
--disoob. Both need Split or Random.Split positions and fail Validate when
combined with Fake or, for Disorder, with SplitWaitAcked.

The hop limit is chosen by peer family and restored on every path. Linux
and Windows restore it only after the kernel reports the low-hop piece as
transmitted (SIOCOUTQNSD; SIO_TCP_INFO counters); macOS writes the piece
once SO_NWRITE reports an empty send buffer, which PlatformCapabilities
reports as not observed. The wait reuses the bounded split-wait poll.

PlatformCapabilities gains Disorder, OOB and per-method Unsupported reasons.
Port ByeDPI's Linux fake: the fake prefix is placed in a shared anonymous
mapping, vmspliced into a pipe and spliced to the socket with the low hop
limit, so the socket's send queue references the page instead of a copy.
Once SIOCOUTQNSD reports the prefix as transmitted, the page is overwritten
with the original bytes and the hop limit restored, so the kernel's
retransmission carries the original. MAP_SHARED keeps a fork from making the
page copy-on-write.

The page is always rewritten before the hop limit is restored, also on
errors and timeouts; waits honour Fake.Timeout, the write deadline and
Close; the pipe and mapping are released and at most 64 operations hold
resources. An error after the transformed send stays terminal.

A loopback test attaches a BPF socket filter to the server that drops
low-hop segments, so the server stream matches only if the retransmission
carried the original bytes (also for disorder and disoob).

macOS reports the fake unsupported: xnu has no splice, and its sendfile
copies the file into socket-buffer mbufs, so no retransmission could carry
bytes other than those sent first.
WrapPacketConn now works on Linux/Android and macOS like on Windows: the
hop-limit option follows the destination family, it is restored before
each original datagram, and a failed restore closes the socket.

ICMP time-exceeded replies stay off the owned socket. Linux reports ICMP
errors on an unconnected UDP socket only with IP_RECVERR/IPV6_RECVERR, so
the wrapper turns them off if the caller enabled them; macOS never delivers
them to an unconnected socket (udp_ctlinput matches the foreign address
exactly). A loopback port-unreachable test shows the error reaching
ReadFrom with IP_RECVERR and not after wrapping.

On a macOS dual-stack socket IPV6_UNICAST_HOPS also sets the IPv4 TTL, so a
default hop limit is restored as its explicit value, and only when the IPv4
and IPv6 defaults are equal.
-disorder, -oob and -oob-byte map to Options.Disorder and Options.OOB; -fake is no longer described as Windows-only.
README methods table and sections, connection I/O details for each method
and OS (including why the macOS fake is not implementable and how ICMP
errors are kept off the UDP socket), roadmap status, validation record,
and the CI capability-report test pattern.
Attaching the classic BPF filter that drops low-hop segments returns EPERM for the unprivileged runner user, so every TestLinuxLowHopDropRetransmitsOriginal case failed on the Linux jobs while it passed as root in Docker. Without the privilege the test now skips with the reason; a Linux CI step runs it again with sudo and DESYNC_REQUIRE_LOWHOP=1, which turns the skip back into a failure.
@Nemu-x
Nemu-x merged commit 9787f03 into main Oct 29, 2026
36 of 38 checks passed
@Nemu-x
Nemu-x deleted the feat/cross-platform-methods branch October 29, 2026 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant