Add http/https vs fetch transport benchmark for the Node.js client#779
Add http/https vs fetch transport benchmark for the Node.js client#779Copilot wants to merge 6 commits into
Conversation
|
|
The benchmark drives `@clickhouse/client` as built from this repository (via the npm workspace symlink), not the published npm release — fix the misleading wording in the README and clients.ts. Replace the broken `tsc --project ... && node ...` recipe (the `src` path alias breaks tsc's rootDir and node globals are unresolved) with `npm run build` + `npx tsx`, and add the build prerequisite so `@clickhouse/client` resolves at runtime. Refresh the sample-results table with real numbers measured locally (macOS/Apple Silicon, Node v24.6.0, ClickHouse head, default config). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rison The fetch() stub routed the response body through the WebStreams (ReadableStream) layer, a known Node-core bottleneck (nodejs/undici#1203) that drains large bodies several times slower than native streams. That made the previous "http/https wins download by ~5x" result an artifact of WebStreams, not a property of the undici transport a real migration would adopt. Switch the stub to undici.request(), which returns a native Node Readable drained the same way as @clickhouse/client — an apples-to-apples transport comparison. With native streams undici is faster across the board locally (latency ~4-5x, download ~1.3-1.4x, upload ~tied/server-bound). Refresh the README sample table, rationale, and caveats accordingly, and add undici as a devDependency. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Benchmark findings: use
|
| Scenario | @clickhouse/client (http/https) |
undici.request() stub |
|---|---|---|
SELECT 1 latency (mean) |
~2.1–2.7 ms | ~0.50 ms (~4–5× faster) |
| Download throughput | ~1310–1440 MiB/s | ~1850–1920 MiB/s (~1.3–1.4× faster) |
| Upload throughput | ~223–225 MiB/s | ~241–245 MiB/s (~tied, server-bound) |
With native streams, undici (low-level API) is faster or tied across the board — supporting the "or better yet with undici directly" framing in #511. Caveats unchanged: this is a bare transport stub vs the full client (omits settings/retries/compression/keep-alive tuning/abort/logging), and loopback numbers will overstate the gaps vs a remote/cloud endpoint. Re-run per workload.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Summary
Issue #511 proposes replacing the Node.js client's legacy
http/httpstransport withfetch/undicifor performance, and asks for trustworthy comparison data before committing to a rewrite. This PR adds a reproducible benchmark that pits the current client against a trivialundici.request()stub over identical requests — no transport rewrite yet, just the data to justify one.benchmarks/transport/— new benchmark:clients.ts— aTransportClientabstraction with two impls driven through the same scenarios:SdkTransportClient(@clickhouse/clientas built from this repo, usesexec()to isolate transport from row parsing) andUndiciTransportClient(minimalundici.request()stub).index.ts— runner for three use cases with warmup + configurable iterations andp50/p90/p99latency: single-request latency (SELECT 1), download throughput (large result set), upload throughput (POSTtonull()so no table setup is needed). Tunable viaCLICKHOUSE_URL,LATENCY_REQUESTS,DOWNLOAD_ROWS,UPLOAD_ROWS,ITERATIONS,WARMUP.stats.ts— dependency-free latency/throughput helpers.README.md— run instructions, config, interpretation, sample local run.undicias a devDependency (Node bundles it internally but does not expose it as an importable module).Why
undici.request()and not the globalfetch()?An earlier revision of this benchmark used
fetch()and showed it losing the download scenario by ~5×. That gap turned out to be entirely the WebStreams (ReadableStream) layer thatfetch()routes the response body through — a known Node-core bottleneck (nodejs/undici#1203) — not a property of undici.undici.request()returns a native NodeReadable(the same stream type@clickhouse/clientdrains), which is the API a real migration would actually adopt and makes this an apples-to-apples transport comparison.Indicative results
Single local run, macOS / Apple Silicon, Node v24.6.0, ClickHouse
head, loopback, default config (LATENCY_REQUESTS=200 DOWNLOAD_ROWS=1000000 UPLOAD_ROWS=1000000 ITERATIONS=10 WARMUP=3), stable across two runs:@clickhouse/client(http/https)undici.request()stubSELECT 1latency (mean)With native streams,
undici.request()is faster or tied across the board — a much clearer signal in favour of the migration than the earlierfetch()-based numbers suggested. Caveats: this is a bare transport stub vs the full client (the stub omits settings/retries/compression/keep-alive tuning/abort/logging), and these are loopback numbers — a remote/cloud endpoint viaCLICKHOUSE_URLwould narrow the relative gaps. Re-run per workload.docker-compose up -d npm run build # so @clickhouse/client resolves at runtime npx tsx benchmarks/transport/index.tsChecklist