Repository navigation
perf: idle tunnels hold less (stacks, TLS raw input, relay buffer) - #129
Merged
Merged
Conversation
An idle tunnel's handler goroutine spends its life blocked in the relay's read, under every frame of the inbound and the tunnel that led there. The runtime shrinks a goroutine stack at GC only when less than a quarter of it is in use, and the setup (TLS handshake, dial) grows it to 16 KiB: with the frames above the read at about 4.7 KiB (Trojan) the stack never came back to 8 KiB, and with the relay goroutine (4 KiB) an idle tunnel kept about 20 KiB of stacks, the largest part of its memory. The large frames above the relay move into functions of their own that return before it starts: tunnel.handleTCPConn's setup (sniffing, rules, limits, the dial: about 1 KiB) into dialTCPConn, the metadata of the sing listener handler's NewConnection and HandleSocket, and the Trojan authentication and request parsing. The Trojan chain above the read drops to about 2.8 KiB, under the quarter of 16 KiB with the runtime's margin, and Hysteria2's likewise, so the GC brings an idle tunnel's handler goroutine back to 8 KiB.
An idle TLS connection now holds a 1 KiB raw input buffer instead of the 2-4 KiB its handshake grew (16-40 KiB after a full record), the frames above its wait for the next record are smaller, and Conn.WaitRead lets the relay wait without a buffer. Pinned to the branch of the fork's pull request until it is released.
copyAdaptive waits for a source that looks idle on a small buffer (2 KiB), taken before the read: an idle Trojan or VLESS/VMess-over-TLS tunnel held it for as long as its client stayed silent. A source that can wait for data without a buffer (getmikan/tls Conn.WaitRead) is now waited on first, and the small buffer is taken once data is there. While a burst runs on full buffers the copy reads as before.
A server session's goroutine waits for the next frame in recvLoop, under Session.Run and the listener's HandleConn, for as long as the session lives. recvLoop's frame (1.1 KiB, the handling of every command) and the authentication in HandleConn kept the frames above that read over a quarter of 16 KiB, so the runtime never shrank the goroutine back to 8 KiB. The frame handling moves to handleFrame, the client part of Run to runClient and the authentication to authenticate; the session goroutine of an idle AnyTLS tunnel now shrinks to 8 KiB at GC.
…ps it The relay unwraps a drained BufferedConn (ReaderReplaceable) before it asks for cached data, so ReadCached, which dropped the bufio.Reader, was never reached: a relayed conn that had been peeked (sniffing, an outbound with a lazy handshake) kept its 4 KiB reader for its whole life. ReaderReplaceable now drops a drained reader, and Read goes straight to the conn without one instead of making it.
This was referenced Oct 18, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Idle-tunnel memory: what an idle tunnel holds on the server, and making it smaller on the common paths (tunnel, sing listener handler, relay, TLS) and in Trojan and AnyTLS.
What an idle tunnel held (heap and goroutine profiles of a server with 5000 idle tunnels,
test/bench/comparewith PPROF, Trojan and Hysteria2)Trojan, per tunnel (cgroup about 58 KiB): goroutine stacks about 20 KiB (two goroutines: the handler at 16 KiB, the relay at 4 KiB), the 4 KiB
bufio.Readerof the inboundBufferedConn(fixed on dev by #125, kept here for peeked conns), 2-4 KiB of TLS raw input grown by the handshake, the relay's 2 KiB small read buffer, 1.4 KiB of AES-GCM state, thetls.Conn(1 KiB), tracker and metadata (about 1 KiB). Hysteria2: the same stacks (20 KiB), the 4 KiB reader, the relay's 2 KiB buffer, the QUIC stream (1.5 KiB).The stacks are the largest part. The runtime shrinks a goroutine stack at GC only when less than a quarter of it is in use. The handler goroutine grows to 16 KiB during the setup (TLS handshake, dial) and then spends its life blocked in the relay's read under every frame of the inbound and the tunnel: about 4.7 KiB for Trojan (with the runtime's 800-byte margin), so it never came back to 8 KiB.
Changes
perf: keep the setup frames off the stack of an idle tunnel):tunnel.handleTCPConn's setup (sniffing, rules, limits, dial; a 1 KiB frame) moves todialTCPConn, which returns before the relay; likewise the metadata of the sing listener handler'sNewConnection/HandleSocket(all sing-based inbounds) and the Trojan authentication and request parsing. The Trojan and Hysteria2 chains above the read drop under a quarter of 16 KiB, so the GC brings the handler goroutine back to 8 KiB.perf(anytls)): a session's goroutine waits inrecvLoop(1.1 KiB frame) underSession.Runand the listener'sHandleConn; frame handling, the client part ofRunand the authentication move to functions of their own.Conn.WaitReadwaits for data without a buffer.perf(relay)):copyAdaptivewaits on a source withWaitRead(a TLS conn) before taking its small buffer, so an idle Trojan or VLESS/VMess-over-TLS tunnel holds no relay buffer. While a burst runs on full buffers the copy reads as before.perf(net)): on top of perf(quic): less work per Hysteria2/TUIC stream; fix a TUIC CONNECT panic #125's lazy reader, a drained reader is dropped when the relay unwraps the conn (the copy asksReaderReplaceablebeforeReadCached, so the old drop inReadCachedwas never reached), andReadwithout a reader goes to the conn.No change to the relay's buffer sizes or batching (#119, #124).
Results
Measured with
test/bench/compare(Docker, server on 2 vCPUs, 2 GiB; Citrus only; REPS=3, median and min-max; cgroup memory per idle tunnel at 1000 and 5000 tunnels; handshake and bulk phases for Trojan, Hysteria2 and AnyTLS, memory only for the others). Measured on the previous dev (cb7eee4, before #124 and #125): before = that dev, after = this branch's commits on it. The branch has since been rebased onto the current dev (#124 download batching, #125 lazy BufferedConn reader, #127 bench) and its tests pass, but a full rerun on the current dev was cut short for the release: the after numbers on top of #125 are not measured. #125 already removes the 4 KiB reader for DIRECT, so part of the BufferedConn gain shown here is now in dev; the stack, TLS, relay and AnyTLS parts are not.Throughput and new connections per second move within the run-to-run spread, except AnyTLS uploads (-4 to -5%, ranges not overlapping at x1); the AnyTLS change only moves frame handling into a function of its own, so this is probably noise but has not been rechecked. Hysteria2 at 5000 tunnels: 30.4 -> 28.2 KiB (sing-box 27.7 in the four-core comparison); Trojan 58.5 -> 37.7 KiB (sing-box 43.6).
Not done here (findings)
sing-shadowsocksshadowaead.NewReader/NewWriterallocate their chunk buffers up front, about 80 + 72 KiB per connection (Windows heap profile: 150 KiB of live heap per idle SS2022 tunnel). The bench does not see it: large fresh allocations are untouched pages, outside RSS. On a long-running server those spans are reused and zeroed, so they become resident. A lazy/pooled buffer needs a fork of sing-shadowsocks.Tests
common/net:TestBufferedConnLazyBuffer,TestBufferedConnDropsDrainedBuffer,TestRelayDropsBufferedConnBuffer,TestCopyConnWaitRead(waited before every small read, not during a burst; EOF and errors of the wait).TestRawInputIdle,TestRawInputCloseNotify,TestWaitRead,BenchmarkRawInputThroughput(unchanged).runtime.MemStatsafter GC, per tunnel): Trojan stacks 24.2 -> 16.1 KiB and live heap 17.1 -> 8.4 KiB; Hysteria2 stacks 24.3 -> 16.2, heap 12.6 -> 8.5; TUIC 16.8 -> 11.2, heap 13.3 -> 6.5; AnyTLS 40.4 -> 24.1, heap 20.0 -> 13.3.