Skip to content

net: UDP recv with MSG_TRUNC must return the datagram length (fixes -u --skip-rx-copy throughput) - #2075

Open
1chunghu wants to merge 2 commits into
esnet:masterfrom
1chunghu:fix/udp-skip-rx-copy-length
Open

1chunghu wants to merge 2 commits into
esnet:masterfrom
1chunghu:fix/udp-skip-rx-copy-length

Conversation

@1chunghu

@1chunghu 1chunghu commented Sep 10, 2026 •

Copy link
Copy Markdown

PLEASE NOTE the following text from the iperf3 license. Submitting a
pull request to the iperf3 repository constitutes "[making]
Enhancements available...publicly":

You are under no obligation whatsoever to provide any bug fixes, patches, or
upgrades to the features, functionality or performance of the source code
("Enhancements") to anyone; however, if you choose to make your Enhancements
available either publicly, or directly to Lawrence Berkeley National
Laboratory, without imposing a separate written license agreement for such
Enhancements, then you hereby grant the following license: a non-exclusive,
royalty-free perpetual license to install, use, modify, prepare derivative
works, incorporate into other computer software, distribute, and sublicense
such enhancements or derivative works thereof, in binary and source code form.

The complete iperf3 license is available in the LICENSE file in the
top directory of the iperf3 source tree.

  • Version of iperf3 (or development branch, such as master or
    3.1-STABLE) to which this pull request applies:

    master (c9b7422). The bug is present since --skip-rx-copy was added in Add SKIP-RX-COPY support - using MSG_TRUNC socket option #1717 (3.16).

  • Issues fixed (if any):

    None filed; found while benchmarking. Related: feature request: support for MSG_ZEROCOPY and MSG_TRUNC flags #1678, Add SKIP-RX-COPY support - using MSG_TRUNC socket option #1717.

  • Brief description of code changes (suitable for use as a commit message):

    net: UDP recv with MSG_TRUNC must return the datagram length

    Nrecv_no_select() / Nread() with MSG_TRUNC return the number of bytes
    copied into the caller's buffer. For a datagram socket recv() reports the
    full datagram length, which is what the caller has to account; returning the
    copied length (12 bytes, the sec/usec/pcount header) makes
    iperf3 -u --skip-rx-copy report roughly 1/100 of the real throughput.
    Return the datagram length for SOCK_DGRAM; TCP behaviour is unchanged.

Reproduction (loopback, Linux 7.2)

$ iperf3 -s -1 &
$ iperf3 -c 127.0.0.1 -u -b 5G -l 1460 -w 4M --skip-rx-copy -J | jq '.end | {sent:.sum_sent.bytes, recv:.sum_received.bytes, gbps:(.sum_received.bits_per_second/1e9)}'
{ "sent": 1875583160, "recv": 20554336, "gbps": 0.055 }      # before (iperf 3.21 release build): receiver accounts 12 bytes per 1460-byte datagram

With this patch, same command:

{ "sent": 1875578780, "recv": 1875578780, "gbps": 5.00 }

TCP with --skip-rx-copy is unaffected (recv with MSG_TRUNC on a stream
socket returns the number of bytes discarded, which is what the loop already
handles): 32692633600 sent / 32692633600 received before and after.

Why

iperf_udp_recv() sizes the receive at sizeof(sec)+sizeof(usec)+sizeof(pcount)
when skip_rx_copy is set, relying on MSG_TRUNC to return the real length
(r = Nrecv_no_select(...) is then added to bytes_received). Both Nread()
and Nrecv_no_select() clamp r to the requested count, so the real length
never reaches the caller. The fix returns r as-is for SOCK_DGRAM in the
MSG_TRUNC branch of both functions (net.c does not include iperf_api.h,
hence SOCK_DGRAM rather than Pudp).

Nrecv_no_select()/Nread() with MSG_TRUNC returned the number of bytes
copied into the caller's buffer.  For a datagram socket recv() reports the
full datagram length, which is what the caller has to account; returning
the copied length (12 bytes) made 'iperf3 -u --skip-rx-copy' report roughly
1/100 of the real throughput (0.07 Gbit/s for a 6.5 Gbit/s stream).

Return the datagram length for SOCK_DGRAM; TCP behaviour is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@davidBar-On

Copy link
Copy Markdown
Contributor

@1chunghu (deleted the previous comment as it was wrong.)
Great that you detected this bug, as it was produced in my PR #1717 (and I have no idea how I missed it 😞)

One suggested enhancement to your change - verify that at least the UDP header was received. Something like that:

            if (prot == SOCK_DGRAM) { /* Pudp; iperf_api.h is not included here */
                if (r < count) {
                    return NET_HARDERROR; // At least the UDP message header should be read
                }
                return r;
            }
            // Non-UDP (e.g. TCP)
            ...

With MSG_TRUNC the returned length is the full datagram size.  If that is
smaller than the requested header length the caller would parse a header
that was never received, so treat it as a hard error instead (suggested by
davidBar-On in review of esnet#2075).  MIN_UDP_BLOCKSIZE equals the header
length, so a well-formed stream can never trip this check.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
1chunghu pushed a commit to 1chunghu/iperf that referenced this pull request Sep 10, 2026
With MSG_TRUNC the returned length is the full datagram size.  If that is
smaller than the requested header length the caller would parse a header
that was never received, so treat it as a hard error instead (suggested by
davidBar-On in review of esnet#2075).  MIN_UDP_BLOCKSIZE equals the header
length, so a well-formed stream can never trip this check.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 9999350)
@1chunghu

Copy link
Copy Markdown
Author

Thanks for the review, and no worries — it's an easy one to miss since the copied length is correct for TCP.

Added the check in 9999350 to both Nrecv() and Nrecv_no_select(). One note: MIN_UDP_BLOCKSIZE is 16, which equals the requested header length in skip-rx-copy mode, so a well-formed stream (32- or 64-bit counters) can never hit the new error path. Re-tested on loopback: -u --skip-rx-copy now reports the full rate with both counter widths, TCP unaffected.

@bmah888

bmah888 commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Thanks for the PR! We'll take a look at this. When I looked at the commits, it appears that you co-authored this with Claude, is that correct? Just checking for clarification.

@1chunghu

Copy link
Copy Markdown
Author

Yes, that's correct — I used Claude Code while working on this, and I add the Co-Authored-By trailer deliberately so that's visible rather than hidden.

To be clear about the split: I found the discrepancy while benchmarking --skip-rx-copy on my own machine (the ~1/100 UDP figure in the description), profiled with gprof to rule out other causes, and decided on the fix. The tool helped draft the patch and the commit message. I reviewed every line, ran the reproduction and the TCP regression check on real hardware, and tested David's suggested header-length guard before pushing 9999350. I understand and accept the license terms for the contribution.

If the project would prefer the trailer removed, or has a policy on AI-assisted contributions I should follow, let me know and I'll amend the commits. Happy to answer any questions about the change itself.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants