Skip to content

perf(relay): bulk copy loops yield every millisecond: echo RTT under load 55-63 ms to 20 ms - #131

Merged
Nemu-x merged 1 commit into
devfrom
perf/latency-under-load
Oct 18, 2026
Merged

Nemu-x merged 1 commit into
devfrom
perf/latency-under-load

Conversation

@Nemu-x

@Nemu-x Nemu-x commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator

What the latency-under-load number measures

cmp measure, lat phase: a 64-byte echo every 20 ms on an established tunnel, while 8 unpaced download streams run through the same server (2 vCPUs, saturated: ~1.9 cores busy). The echo goes client → server → sink and back, so it crosses the server's relay twice. Everything is on one Docker bridge (no shaping, no loss), and the client and sink have spare CPU.

Where the time goes

It's the Go scheduler, not socket buffers. A bulk relay loop never blocks: the sink always has data and the client takes it. So it runs for a whole 10 ms time slice before the runtime preempts it. When the echo packet arrives, its goroutine becomes runnable. While every P is busy, that happens only when sysmon polls the network, about every 10 ms. It then waits in the global run queue behind the bulk loops, at 10 ms per turn on 2 Ps: about 40 ms per hop, about 80 ms per round trip. Xray (also Go) shows the same ~77 ms.

Experiments (pre-#124 dev, Trojan and SS2022, 3 reps):

variant echo RTT p50 / p99 ms note
baseline 80 / 139 (Trojan), 70 / 136 (SS2022)
TCP_NOTSENT_LOWAT 16 KiB / 128 KiB on the inbound listener 63 / 127, 62 / 121 (Trojan); 70 / 140, 64 / 136 (SS2022) small effect: the send queues aren't the bottleneck
runtime.Gosched() after every write 20 / 21 throughput −13% on SS2022
Gosched every 256 KiB 20 / 21 same RTT
GOMAXPROCS=16 (idle Ps keep a thread in netpoll) 0.8–1.6 / 17–22 only while there are fewer busy loops than Ps; not a default

With yields, what's left is exactly 2 × the runtime's 10 ms network-poll interval while no P is idle. That's a runtime constant.

The byte-only yield has a cost: Gosched also calls wakep, so on a lone transfer with idle cores a yield every 256 KiB woke an idle P that took the goroutine over. On current dev, SS2022 dl1 dropped from 1986 to 1079 MB/s. So the yield now needs both limits: 256 KiB moved and 1 ms run since the last yield. Interactive connections never reach it.

Change

common/net/relay_yield.go: relayYield.add(n) is called after each write in copyWait, copyAdaptive and both halves of copyBatched. It calls runtime.Gosched() once both limits are reached. There are no socket options and no config: the yield has no throughput trade-off to tune, and TCP_NOTSENT_LOWAT didn't earn a default.

Before / after (current dev, 4-core Docker bench, server 2 vCPUs, REPS=1)

echo RTT p50 p99 new-conn TTFB p50 p99 dl1 MB/s dl32 ul1 ul32 CPU s/GB dl32
SS2022 before 54.7 129.5 69.3 173.7 2005 4233 1426 2750 0.443
SS2022 after 19.9 24.0 29.1 35.3 1986 4189 1680 2749 0.477
Trojan before 63.0 124.2 124.7 192.2 1105 3789 1160 2747 0.510
Trojan after 20.1 24.0 38.8 45.5 1169 3814 1179 2762 0.523

Throughput is within noise. CPU per GB is +3 to +8% at n=1. A 256 KiB-only first cut (n=1) also showed REALITY at 44 → 20 ms p50 and 130 → 39 ms p99, AnyTLS at 37 → 6 / 138 → 17, and XHTTP's p99 at 44 → 20. For reference, sing-box's SS2022 sits at about 47 ms p50.

Tests

  • TestRelayYieldLimits: no yield before either limit; the count starts over after a yield.
  • TestCopyConnYields: with GOMAXPROCS(1), a copy that never blocks runs next to a goroutine that sleeps 100 µs in a loop. Without the yield that goroutine ran about 25 times in 500 ms; with it, 360–400 times (the test needs more than 150).
  • go test ./common/net/..., plus builds for linux/darwin/windows/linux-arm64.

A relay loop of a bulk transfer on a loaded server never blocks, so the Go
scheduler lets it run for its whole 10 ms time slice, and the goroutine of an
interactive connection that the poller made runnable waits behind every such
loop: with 8 downloads on 2 cores, about 40 ms per hop through the server.

The copy loops (copyWait, copyAdaptive, both halves of copyBatched) now call
runtime.Gosched once they have moved 256 KiB and run for 1 ms since their
last yield. Both limits keep the yield rare: a Gosched also wakes an idle
processor, which costs a lone transfer on idle cores when it happens every
few hundred KiB, and a connection that moves little never yields.
@Nemu-x
Nemu-x merged commit 4c6588d into dev Oct 18, 2026
9 checks passed
@Nemu-x
Nemu-x deleted the perf/latency-under-load branch October 18, 2026 11:58
Nemu-x added a commit that referenced this pull request Oct 18, 2026
docs(changelog): #112-#131 and the forks for v1.0.0-beta.2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant