net: UDP recv with MSG_TRUNC must return the datagram length (fixes -u --skip-rx-copy throughput) - #2075
net: UDP recv with MSG_TRUNC must return the datagram length (fixes -u --skip-rx-copy throughput)#20751chunghu wants to merge 2 commits into
Conversation
Nrecv_no_select()/Nread() with MSG_TRUNC returned the number of bytes copied into the caller's buffer. For a datagram socket recv() reports the full datagram length, which is what the caller has to account; returning the copied length (12 bytes) made 'iperf3 -u --skip-rx-copy' report roughly 1/100 of the real throughput (0.07 Gbit/s for a 6.5 Gbit/s stream). Return the datagram length for SOCK_DGRAM; TCP behaviour is unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
@1chunghu (deleted the previous comment as it was wrong.) One suggested enhancement to your change - verify that at least the UDP header was received. Something like that: if (prot == SOCK_DGRAM) { /* Pudp; iperf_api.h is not included here */
if (r < count) {
return NET_HARDERROR; // At least the UDP message header should be read
}
return r;
}
// Non-UDP (e.g. TCP)
... |
With MSG_TRUNC the returned length is the full datagram size. If that is smaller than the requested header length the caller would parse a header that was never received, so treat it as a hard error instead (suggested by davidBar-On in review of esnet#2075). MIN_UDP_BLOCKSIZE equals the header length, so a well-formed stream can never trip this check. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
With MSG_TRUNC the returned length is the full datagram size. If that is smaller than the requested header length the caller would parse a header that was never received, so treat it as a hard error instead (suggested by davidBar-On in review of esnet#2075). MIN_UDP_BLOCKSIZE equals the header length, so a well-formed stream can never trip this check. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit 9999350)
|
Thanks for the review, and no worries — it's an easy one to miss since the copied length is correct for TCP. Added the check in 9999350 to both |
|
Thanks for the PR! We'll take a look at this. When I looked at the commits, it appears that you co-authored this with Claude, is that correct? Just checking for clarification. |
|
Yes, that's correct — I used Claude Code while working on this, and I add the To be clear about the split: I found the discrepancy while benchmarking If the project would prefer the trailer removed, or has a policy on AI-assisted contributions I should follow, let me know and I'll amend the commits. Happy to answer any questions about the change itself. |
PLEASE NOTE the following text from the iperf3 license. Submitting a
pull request to the iperf3 repository constitutes "[making]
Enhancements available...publicly":
The complete iperf3 license is available in the
LICENSEfile in thetop directory of the iperf3 source tree.
Version of iperf3 (or development branch, such as
masteror3.1-STABLE) to which this pull request applies:master(c9b7422). The bug is present since--skip-rx-copywas added in Add SKIP-RX-COPY support - using MSG_TRUNC socket option #1717 (3.16).Issues fixed (if any):
None filed; found while benchmarking. Related: feature request: support for MSG_ZEROCOPY and MSG_TRUNC flags #1678, Add SKIP-RX-COPY support - using MSG_TRUNC socket option #1717.
Brief description of code changes (suitable for use as a commit message):
net: UDP recv with MSG_TRUNC must return the datagram length
Nrecv_no_select()/Nread()withMSG_TRUNCreturn the number of bytescopied into the caller's buffer. For a datagram socket
recv()reports thefull datagram length, which is what the caller has to account; returning the
copied length (12 bytes, the
sec/usec/pcountheader) makesiperf3 -u --skip-rx-copyreport roughly 1/100 of the real throughput.Return the datagram length for
SOCK_DGRAM; TCP behaviour is unchanged.Reproduction (loopback, Linux 7.2)
With this patch, same command:
TCP with
--skip-rx-copyis unaffected (recvwithMSG_TRUNCon a streamsocket returns the number of bytes discarded, which is what the loop already
handles): 32692633600 sent / 32692633600 received before and after.
Why
iperf_udp_recv()sizes the receive atsizeof(sec)+sizeof(usec)+sizeof(pcount)when
skip_rx_copyis set, relying onMSG_TRUNCto return the real length(
r = Nrecv_no_select(...)is then added tobytes_received). BothNread()and
Nrecv_no_select()clamprto the requested count, so the real lengthnever reaches the caller. The fix returns
ras-is forSOCK_DGRAMin theMSG_TRUNCbranch of both functions (net.cdoes not includeiperf_api.h,hence
SOCK_DGRAMrather thanPudp).