Repository navigation
perf(relay): larger download writes: 64 KiB reads, TLS records written together - #124
Merged
Merged
Conversation
…n together The download of a proxied connection reads the remote socket and writes an encrypting connection. It read 32 KiB at a time, and every TLS record (16 KiB) went to the client socket with a write of its own, so a loaded server spent most of its download time in write(2). - The copy from a socket switches to 65535-byte buffers (the largest pool buffer) once a read fills a 32 KiB one. Shadowsocks 2022 seals each read as one chunk (chunks go up to 65535 bytes), as sing-box does after its first 512 KB. The socket read waiter still takes a buffer only once data is there, so an idle connection holds none. - getmikan/tls and getmikan/utls write the records of one write together (up to eight full records), so a 64 KiB read leaves as one write of four records, for Trojan, VLESS/REALITY and everything else on TLS. 4-core Docker bench, medians of 3 (MB/s; CPU s/GB): REALITY dl32 2802 -> 4235 (0.67 -> 0.46), Trojan dl32 2737 -> 3936 (0.68 -> 0.49), SS2022 dl1 1711 -> 2029, dl32 3895 -> 4392 (0.49 -> 0.44), XHTTP dl32 1213 -> 1410 (1.55 -> 1.11). Echo RTT under load is lower (REALITY p50 82 -> 57 ms).
This was referenced Oct 18, 2026
Merged
Merged
escapezn
added a commit
to escapezn/mihomo
that referenced
this pull request
Nov 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Download (remote → client) gets the same kind of gain as upload got in #119: fewer, larger writes to the client socket.
Why download was at mihomo's level
Profiles of the bulk download (pprof, Citrus server): ~65–75% of the server CPU is
read(2)/write(2), ~23% AES-GCM. The copy read the remote socket 32 KiB at a time, and:tls.Conn.Writesent every 16 KiB record with a write of its own → 2write(2)per 32 KiB, each carrying a 16 KiB segment through the network stack.IncreaseBuffer), so every write is a 64 KiB chunk. Our copy sealed 32 KiB chunks: twice the writes and seals. The protocol code (shadowaead,shadowaead_2022) is the same in both; the difference was only the buffer size.Changes
common/net: the copy from a socket (copyWait) switches to 65535-byte buffers (the largest pool buffer) once a read fills a 32 KiB one. The socket read waiter takes a buffer only once data is there, so an idle connection still holds none; short transfers never switch.getmikan/tls(crypto/tls: write the records of one write together tls#3) andgetmikan/utls(crypto/tls: write the records of one write together utls#2): the records of one write go out in one write (up to eight full records), encrypted in place. Nothing is held back, so latency is unchanged.go.modpoints at the PR commits (pseudo-versions); after those merge I'll switch to tags.A read-ahead goroutine for download (like the upload one) was tried as well: no gain on top of this (the server is not CPU-bound on one stream, the client is), so it is not included.
Numbers
4-core Docker bench (
test/bench/compare, server cpuset 2 vCPUs), medians of 3 reps, same session; MB/s, CPU s/GB in parentheses. Order: sing-box / mihomo / Citrus before / Citrus after.Single-stream TLS downloads stay client-bound (the Xray client uses ~1.6 cores, the server under 0.5): Trojan dl1 1137 (1125–1202) vs 1183 (1155–1209) before, TUIC dl1 481 (464–557) vs 530 (449–560), within the spread. Upload unchanged.
Latency phase (echo RTT on an idle tunnel while an 8-stream download runs), p50 / p99 ms, before → after: REALITY 82.1 / 148 → 56.9 / 104, Trojan 81.7 / 143 → 63.2 / 116, SS2022 69.2 / 149 → 57.0 / 121, XHTTP 1.43 / 19.6 → 0.77 / 2.89, Hysteria2 1.00 / 2.59 → 0.89 / 2.00. New-connection TTFB under load: REALITY 275 → 189 ms, Trojan 157 → 122 ms.
Tests
TestCopyConnFromSocketGrowsReads: 16 MiB from a socket to a non-socket destination: data intact, writes above 32 KiB appear, none over 65535.TestCopyConnFromSocketShortReads: short messages stay on pool buffers.