Skip to content

perf: idle tunnels hold less (stacks, TLS raw input, relay buffer) - #129

Merged
Nemu-x merged 5 commits into
devfrom
perf/idle-memory
Oct 18, 2026
Merged

Nemu-x merged 5 commits into
devfrom
perf/idle-memory

Conversation

@Nemu-x

@Nemu-x Nemu-x commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Idle-tunnel memory: what an idle tunnel holds on the server, and making it smaller on the common paths (tunnel, sing listener handler, relay, TLS) and in Trojan and AnyTLS.

What an idle tunnel held (heap and goroutine profiles of a server with 5000 idle tunnels, test/bench/compare with PPROF, Trojan and Hysteria2)

Trojan, per tunnel (cgroup about 58 KiB): goroutine stacks about 20 KiB (two goroutines: the handler at 16 KiB, the relay at 4 KiB), the 4 KiB bufio.Reader of the inbound BufferedConn (fixed on dev by #125, kept here for peeked conns), 2-4 KiB of TLS raw input grown by the handshake, the relay's 2 KiB small read buffer, 1.4 KiB of AES-GCM state, the tls.Conn (1 KiB), tracker and metadata (about 1 KiB). Hysteria2: the same stacks (20 KiB), the 4 KiB reader, the relay's 2 KiB buffer, the QUIC stream (1.5 KiB).

The stacks are the largest part. The runtime shrinks a goroutine stack at GC only when less than a quarter of it is in use. The handler goroutine grows to 16 KiB during the setup (TLS handshake, dial) and then spends its life blocked in the relay's read under every frame of the inbound and the tunnel: about 4.7 KiB for Trojan (with the runtime's 800-byte margin), so it never came back to 8 KiB.

Changes

  • Setup frames off the idle stack (perf: keep the setup frames off the stack of an idle tunnel): tunnel.handleTCPConn's setup (sniffing, rules, limits, dial; a 1 KiB frame) moves to dialTCPConn, which returns before the relay; likewise the metadata of the sing listener handler's NewConnection/HandleSocket (all sing-based inbounds) and the Trojan authentication and request parsing. The Trojan and Hysteria2 chains above the read drop under a quarter of 16 KiB, so the GC brings the handler goroutine back to 8 KiB.
  • AnyTLS (perf(anytls)): a session's goroutine waits in recvLoop (1.1 KiB frame) under Session.Run and the listener's HandleConn; frame handling, the client part of Run and the authentication move to functions of their own.
  • TLS (crypto/tls: idle connections hold less (raw input sized by what is moving, WaitRead) tls#2, pinned to the commit of its branch; that pseudo-version stays valid after the merge): the raw input buffer is sized by what is moving (1 KiB of its own while records are small, a pooled buffer for a large record, returned when drained after a short record; the handshake buffer is dropped), the wait for the next record is in a small frame, and Conn.WaitRead waits for data without a buffer.
  • Relay (perf(relay)): copyAdaptive waits on a source with WaitRead (a TLS conn) before taking its small buffer, so an idle Trojan or VLESS/VMess-over-TLS tunnel holds no relay buffer. While a burst runs on full buffers the copy reads as before.
  • BufferedConn (perf(net)): on top of perf(quic): less work per Hysteria2/TUIC stream; fix a TUIC CONNECT panic #125's lazy reader, a drained reader is dropped when the relay unwraps the conn (the copy asks ReaderReplaceable before ReadCached, so the old drop in ReadCached was never reached), and Read without a reader goes to the conn.

No change to the relay's buffer sizes or batching (#119, #124).

Results

Measured with test/bench/compare (Docker, server on 2 vCPUs, 2 GiB; Citrus only; REPS=3, median and min-max; cgroup memory per idle tunnel at 1000 and 5000 tunnels; handshake and bulk phases for Trojan, Hysteria2 and AnyTLS, memory only for the others). Measured on the previous dev (cb7eee4, before #124 and #125): before = that dev, after = this branch's commits on it. The branch has since been rebased onto the current dev (#124 download batching, #125 lazy BufferedConn reader, #127 bench) and its tests pass, but a full rerun on the current dev was cut short for the release: the after numbers on top of #125 are not measured. #125 already removes the 4 KiB reader for DIRECT, so part of the BufferedConn gain shown here is now in dev; the stack, TLS, relay and AnyTLS parts are not.

protocol metric before after change
trojan KiB/tunnel @1000 51.6 (51.0-51.7) 38.9 (38.0-39.8) -24.6%
trojan KiB/tunnel @5000 58.5 (57.7-58.5) 37.7 (37.6-37.9) -35.6%
trojan new conns/s 5209.3 (5183.2-5504.0) 5441.6 (5283.0-5483.8) +4.5%
trojan MB/s down x1 1190.0 (1152.8-1212.1) 1151.6 (1145.1-1157.1) -3.2%
trojan MB/s down x32 2804.6 (2763.6-2844.2) 2835.9 (2829.6-2859.4) +1.1%
trojan MB/s up x1 1165.1 (1151.5-1172.5) 1188.9 (1164.3-1423.3) +2.0%
trojan MB/s up x32 2920.3 (2904.7-2942.5) 2871.6 (2797.6-2891.0) -1.7%
hy2 KiB/tunnel @1000 22.1 (21.3-23.6) 21.4 (19.2-30.1) -3.1%
hy2 KiB/tunnel @5000 30.4 (30.2-31.1) 28.2 (25.3-30.8) -7.0%
hy2 new conns/s 18273.8 (17482.3-18553.0) 17984.8 (17305.9-18180.9) -1.6%
hy2 MB/s down x1 717.5 (564.9-775.4) 763.6 (753.6-769.5) +6.4%
hy2 MB/s down x32 683.0 (680.1-686.6) 681.9 (648.4-750.6) -0.2%
hy2 MB/s up x1 395.9 (390.9-399.5) 389.2 (377.0-402.3) -1.7%
hy2 MB/s up x32 607.4 (604.5-625.2) 607.5 (592.5-619.9) +0.0%
anytls KiB/tunnel @1000 68.8 (61.6-70.7) 51.3 (50.9-55.2) -25.4%
anytls KiB/tunnel @5000 71.2 (70.5-72.4) 58.5 (57.0-62.5) -17.8%
anytls new conns/s 13442.9 (12919.9-13950.9) 13314.5 (13274.9-13731.1) -1.0%
anytls MB/s down x1 932.5 (927.8-952.7) 946.2 (919.6-954.2) +1.5%
anytls MB/s down x32 2349.9 (2247.5-2367.9) 2331.4 (2319.2-2332.8) -0.8%
anytls MB/s up x1 1696.0 (1682.0-1709.9) 1604.6 (1604.1-1609.4) -5.4%
anytls MB/s up x32 2961.4 (2894.1-2969.4) 2845.9 (2841.9-2926.2) -3.9%
reality KiB/tunnel @1000 53.5 (53.1-54.4) 48.8 (48.0-48.8) -8.9%
reality KiB/tunnel @5000 62.0 (59.2-63.5) 54.3 (52.4-55.4) -12.3%
xhttp KiB/tunnel @1000 48.9 (48.4-49.3) 46.9 (46.6-48.5) -4.1%
xhttp KiB/tunnel @5000 57.6 (57.5-58.9) 56.4 (55.1-57.1) -2.2%
ss2022 KiB/tunnel @1000 56.9 (56.3-58.9) 53.3 (52.8-56.5) -6.2%
ss2022 KiB/tunnel @5000 54.2 (54.1-54.5) 55.6 (52.3-57.1) +2.7%
tuic KiB/tunnel @1000 17.6 (16.5-18.5) 15.4 (15.0-16.3) -12.4%
tuic KiB/tunnel @5000 22.5 (22.4-23.1) 21.0 (20.5-21.6) -6.7%

Throughput and new connections per second move within the run-to-run spread, except AnyTLS uploads (-4 to -5%, ranges not overlapping at x1); the AnyTLS change only moves frame handling into a function of its own, so this is probably noise but has not been rechecked. Hysteria2 at 5000 tunnels: 30.4 -> 28.2 KiB (sing-box 27.7 in the four-core comparison); Trojan 58.5 -> 37.7 KiB (sing-box 43.6).

Not done here (findings)

  • Shadowsocks 2022: sing-shadowsocks shadowaead.NewReader/NewWriter allocate their chunk buffers up front, about 80 + 72 KiB per connection (Windows heap profile: 150 KiB of live heap per idle SS2022 tunnel). The bench does not see it: large fresh allocations are untouched pages, outside RSS. On a long-running server those spans are reused and zeroed, so they become resident. A lazy/pooled buffer needs a fork of sing-shadowsocks.
  • The relay's 2 KiB small buffer of a QUIC stream (Hysteria2, TUIC) stays: quic-go has no wait-without-buffer.
  • REALITY (utls) is untouched.

Tests

  • common/net: TestBufferedConnLazyBuffer, TestBufferedConnDropsDrainedBuffer, TestRelayDropsBufferedConnBuffer, TestCopyConnWaitRead (waited before every small read, not during a burst; EOF and errors of the wait).
  • getmikan/tls: TestRawInputIdle, TestRawInputCloseNotify, TestWaitRead, BenchmarkRawInputThroughput (unchanged).
  • The splits are behaviour-preserving; the Trojan, AnyTLS, tunnel and inbound tests cover them.
  • Local probe (Windows, 2000-3000 idle tunnels, runtime.MemStats after GC, per tunnel): Trojan stacks 24.2 -> 16.1 KiB and live heap 17.1 -> 8.4 KiB; Hysteria2 stacks 24.3 -> 16.2, heap 12.6 -> 8.5; TUIC 16.8 -> 11.2, heap 13.3 -> 6.5; AnyTLS 40.4 -> 24.1, heap 20.0 -> 13.3.

An idle tunnel's handler goroutine spends its life blocked in the relay's
read, under every frame of the inbound and the tunnel that led there. The
runtime shrinks a goroutine stack at GC only when less than a quarter of it
is in use, and the setup (TLS handshake, dial) grows it to 16 KiB: with the
frames above the read at about 4.7 KiB (Trojan) the stack never came back to
8 KiB, and with the relay goroutine (4 KiB) an idle tunnel kept about 20 KiB
of stacks, the largest part of its memory.

The large frames above the relay move into functions of their own that
return before it starts: tunnel.handleTCPConn's setup (sniffing, rules,
limits, the dial: about 1 KiB) into dialTCPConn, the metadata of the sing
listener handler's NewConnection and HandleSocket, and the Trojan
authentication and request parsing. The Trojan chain above the read drops to
about 2.8 KiB, under the quarter of 16 KiB with the runtime's margin, and
Hysteria2's likewise, so the GC brings an idle tunnel's handler goroutine
back to 8 KiB.
An idle TLS connection now holds a 1 KiB raw input buffer instead of the
2-4 KiB its handshake grew (16-40 KiB after a full record), the frames above
its wait for the next record are smaller, and Conn.WaitRead lets the relay
wait without a buffer. Pinned to the branch of the fork's pull request until
it is released.
copyAdaptive waits for a source that looks idle on a small buffer (2 KiB),
taken before the read: an idle Trojan or VLESS/VMess-over-TLS tunnel held it
for as long as its client stayed silent. A source that can wait for data
without a buffer (getmikan/tls Conn.WaitRead) is now waited on first, and the
small buffer is taken once data is there. While a burst runs on full buffers
the copy reads as before.
A server session's goroutine waits for the next frame in recvLoop, under
Session.Run and the listener's HandleConn, for as long as the session lives.
recvLoop's frame (1.1 KiB, the handling of every command) and the
authentication in HandleConn kept the frames above that read over a quarter
of 16 KiB, so the runtime never shrank the goroutine back to 8 KiB. The
frame handling moves to handleFrame, the client part of Run to runClient
and the authentication to authenticate; the session goroutine of an idle
AnyTLS tunnel now shrinks to 8 KiB at GC.
…ps it

The relay unwraps a drained BufferedConn (ReaderReplaceable) before it asks
for cached data, so ReadCached, which dropped the bufio.Reader, was never
reached: a relayed conn that had been peeked (sniffing, an outbound with a
lazy handshake) kept its 4 KiB reader for its whole life. ReaderReplaceable
now drops a drained reader, and Read goes straight to the conn without one
instead of making it.
@Nemu-x
Nemu-x merged commit 84a9969 into dev Oct 18, 2026
14 of 15 checks passed
@Nemu-x
Nemu-x deleted the perf/idle-memory branch October 18, 2026 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant