Skip to content

ci(perf): HTTPS/TLS performance gates for 3.9 - #309

Merged
gg582 merged 2 commits into
devfrom
ci/https-perf-gates
Oct 4, 2026
Merged

gg582 merged 2 commits into
devfrom
ci/https-perf-gates

Conversation

@gg582

@gg582 gg582 commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

Summary

CWIST 3.9 had zero TLS coverage in perf gates: benchmarks/webserver.json / scripts/ci/benchmark.py gate plaintext HTTP only. This adds a self-contained HTTPS gate suite, closing the loop on #306 (TLS handshake plateau) and locking in the #307 fix (quickack Nagle/delayed-ACK stall).

New files (4, no changes to existing gates):

File Role
scripts/ci/https_perf_gates.sh Builds a bench server + gate-1 RTT client (embedded C), generates a self-signed ECDSA cert at runtime, runs the 5 gate workloads, emits one JSON metrics row
scripts/ci/https_gates_eval.py Evaluates the row: absolute backstop per metric + same-runner-CPU median comparison (imports runner_baseline.py; history in benchmarks/https_gates.json, seeded)
.github/workflows/perf-https-gates.yml Runs on PRs touching the TLS path, on dev pushes, and on dispatch; updates history only after gates pass on a dev push
benchmarks/https_gates.json Seeded per-CPU history (Ryzen 5600X row)

Gate table

# Metric Workload Backstop Healthy (measured) Pre-#307
1 tls_rtt_p50_ms Sequential TLS client without TCP_NODELAY, 200 fresh connections, first-response RTT p50 — the #307 regression test ≤ 5 ms ~0.09 ms ~43 ms
2 https_churn_rps ab -n 6000 -c 32, new connection per request ≥ 1000/s ~1585/s local, 2013 CI ~683/s
3 https_keepalive_rps wrk -t4 -c100 -d10s ≥ 110k/s 135–179k/s (unchanged by #307)
4 https_big_gbps wrk -t4 -c32 -d10s on a 1 MiB body ≥ 2.5 GB/s ~4.0 GB/s —
5 http_* controls Same three workloads plaintext churn ≥ 6k, keep ≥ 150k, big ≥ 5.5 GB/s 13.5–15.8k / 207–274k / 7.8–13.5 GB/s —

Ratio signal: https_keepalive_ratio (HTTPS/HTTP keep-alive, ~0.65 expected) is recorded but not gated — a large drop hints at TLS-path damage even when absolute gates pass.

Backstop calibration (updated after CI run 37201724744): absolute backstops were initially tuned on a quiet 12-core desktop and proved too tight for shared CI runners (http_big_gbps measured 7.8 vs the 8.0 backstop on a shared EPYC 9V45 while all other gates passed). Commit 94eb670b recalibrated every backstop against the observed CI values with variance headroom; a row rebuilt from that failed run's numbers now passes 7/7, and the pre-#307 row still fails gates 1 and 2.

Runner-relative gating: like the existing runner_baseline.py gates, the absolute backstops are only the any-hardware backstop; once ≥5 history rows exist for a CI CPU model, each metric is also judged against median / allowance (throughput) or median × allowance (RTT), because absolute numbers vary by CI CPU (documented in https_gates_eval.py header).

Local verification (Ryzen 5 5600X, 12 vCPU)

Healthy code (dev + this branch), full end-to-end run of scripts/ci/https_perf_gates.sh + evaluator:

tls_rtt_p50_ms: 0.086   https_churn_rps: 1585   https_keepalive_rps: 179188
http_churn_rps: 13523   http_keepalive_rps: 261719   https_big_gbps: 4.18   http_big_gbps: 13.18
https gates: 7/7 passed  (exit 0)

Negative test (quickack re-arms disabled — simulates pre-#307):

tls_rtt_p50_ms: 43.182  -> FAIL: over the 5.0ms absolute backstop
https_churn_rps: 682.8  -> FAIL: under the 1200.0 absolute backstop
https gates: 5/7 passed  (exit 1)

Gate 1 and gate 2 reproduce the pre-#307 numbers (43ms / 683/s) and fail exactly as designed.

Also verified: make libcwist.a clean, test_https and test_https_park pass on this branch.

Notes

  • CI time: one BoringSSL build (~4–5 min) + ~90 s of measurement; comparable to the existing perf-https-handshake-shards.yml workflow.
  • The workflow's history-commit step runs only on push to dev after gates pass; PR runs (including forks, which get read-only tokens) never modify the history file.
  • Bench server and RTT client are generated into /tmp at runtime; no dependency on example/ certs.

Refs #306, #307

gg582 added 2 commits October 4, 2026 21:15
Extends the perf-gate infrastructure with TLS coverage, following the
runner-baseline convention: an absolute backstop per metric (must hold on
any CI hardware) plus a same-runner-CPU median comparison once >=5 samples
exist for that CPU, since absolute numbers move a lot between CI machines.

Gates (measured CWIST 3.9 baselines in comments):
  1. tls_rtt_p50_ms      sequential TLS client WITHOUT TCP_NODELAY,
     first-response RTT p50 over 200 fresh connections; backstop <= 5ms.
     This is the #307 regression test: pre-#307 measures ~43ms here.
  2. https_churn_rps     ab -n 6000 -c 32 new-conn-per-request; >= 1200
     (~1650 healthy, ~600 pre-#307).
  3. https_keepalive_rps wrk -t4 -c100 -d10s; >= 130k (~167-190k).
  4. https_big_gbps      wrk -t4 -c32 on a 1MiB body; >= 3GB/s (~4GB/s).
  5. http_* plaintext controls for the same workloads (collateral damage).

The https/http keep-alive ratio (~0.65) is recorded as a diagnostic
signal, not gated. Bench server and cert are generated at runtime; the
gate job runs on PRs touching the TLS path and on dev pushes, and appends
to benchmarks/https_gates.json only after gates pass on a dev push.

Verified locally on a Ryzen 5600X: 7/7 gates pass on dev; with the
quickack re-arms disabled the run measures 43.2ms RTT p50 / 683 churn rps
and the evaluator exits 1 (fails gates 1 and 2).
CI run 37201724744 (PR #309, shared AMD EPYC 9V45 runner) failed only
http_big_gbps: 7.8 measured vs the 8.0 backstop, while every other gate
passed with the local-box numbers as reference. The absolute backstops
were calibrated on a quiet 12-core desktop and are too tight for noisy
shared runners. Recalibrate each backstop against the observed CI values
with variance headroom:

  tls_rtt_p50_ms       <=5ms    (unchanged; observed 0.070)
  https_churn_rps      >=1200   -> >=1000   (observed 2013)
  https_keepalive_rps  >=130k   -> >=110k   (observed 135570)
  https_big_gbps       >=3      -> >=2.5    (observed 4.03)
  http_churn_rps       >=8000   -> >=6000   (observed 15793)
  http_keepalive_rps   >=180k   -> >=150k   (observed 206650)
  http_big_gbps        >=8      -> >=5.5    (observed 7.8)

Verified: a row rebuilt from the failed run's observed values now passes
7/7, and the pre-#307 row (43.2ms RTT p50, 683/s churn) still fails gates
1 and 2. The https_keepalive_ratio diagnostic signal is unchanged.
@gg582
gg582 merged commit b52f369 into dev Oct 4, 2026
14 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant