Skip to content

feat: add anti_gro setting to defeat TCP-GRO coalescing#45

Open
tracyhatemice wants to merge 1 commit into
hack3ric:masterfrom
tracyhatemice:anti-gro
Open

feat: add anti_gro setting to defeat TCP-GRO coalescing#45
tracyhatemice wants to merge 1 commit into
hack3ric:masterfrom
tracyhatemice:anti-gro

Conversation

@tracyhatemice

Copy link
Copy Markdown
Contributor

Linux 6.18's PPPoE GRO offload (and any TCP-aware GRO path) coalesces a stream of mimic-produced fake-TCP packets into a single superframe, since mimic emits a constant wire ack_seq across a data burst. The receiver's XDP ingress then demangles the superframe as one packet, producing garbage that WireGuard drops -- collapsing ingress throughput.

New per-filter setting anti_gro (default off) makes egress add a per-packet random jitter to the wire ack_seq, which trips the flush check at net/ipv4/tcp_offload.c:337 (flush |= th->ack_seq ^ th2->ack_seq) and prevents coalescing in both the normal merge and fraglist paths. The padding entropy in conn_padding is adjusted to drop ack_seq when anti_gro is on, keeping receiver-side padding computation symmetric.

Both peers must enable the setting; default off preserves compatibility with prior mimic versions.

Linux 6.18's PPPoE GRO offload (and any TCP-aware GRO path) coalesces a
stream of mimic-produced fake-TCP packets into a single superframe,
since mimic emits a constant wire ack_seq across a data burst. The
receiver's XDP ingress then demangles the superframe as one packet,
producing garbage that WireGuard drops -- collapsing ingress throughput.

New per-filter setting anti_gro (default off) makes egress add a
per-packet random jitter to the wire ack_seq, which trips the flush
check at net/ipv4/tcp_offload.c:337 (flush |= th->ack_seq ^ th2->ack_seq)
and prevents coalescing in both the normal merge and fraglist paths.
The padding entropy in conn_padding is adjusted to drop ack_seq when
anti_gro is on, keeping receiver-side padding computation symmetric.

Both peers must enable the setting; default off preserves bit-for-bit
wire compatibility with prior mimic versions.
@hack3ric

Copy link
Copy Markdown
Owner

Thanks for the PR. But I have some questions:

  • Will middlebox conntrack break because of it? I suspect either bpf_prandom_u32() is either too large as jitter, or ack_seq + some_random_number is not valid at all for a TCP connection. Mimic do have a test for conntrack behaviours; you can modify and test it to see if it works.
  • Can we just disable TCP GRO on affected interfaces using ethtool or others? I think solving the root cause is better than a hack.

@airend

airend commented Jul 10, 2026

Copy link
Copy Markdown
  • Will middlebox conntrack break because of it? I suspect either bpf_prandom_u32() is either too large as jitter, or ack_seq + some_random_number is not valid at all for a TCP connection. Mimic do have a test for conntrack behaviours; you can modify and test it to see if it works.

That's a big concern, that some firewalls (one use case) will be even more confused by these large nonlinear jumps. Also, would doing this increase the overhead a lot?

I can also report that this change breaks the BPF verifier on kernel 7.2-rc2 because it cannot prove that the padding length > 0 at the call site, so the program fails to load.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants