Skip to content

fix: benchmarks measured 307 redirects instead of the endpoints - #16

Merged
KissPeter merged 1 commit into
KissPeter:mainfrom
Vladgasik:fix/benchmark-measures-redirects
Sep 24, 2026
Merged

KissPeter merged 1 commit into
KissPeter:mainfrom
Vladgasik:fix/benchmark-measures-redirects

Conversation

@Vladgasik

Copy link
Copy Markdown
Contributor

Situation

The repository publishes throughput comparisons between FastAPI configurations,
measured with ab against containers in docker-compose.yml. The endpoints
under test are declared with a trailing slash (/sync/big_json_response/,
/async/items/)

Complication

33 of those measurement configurations request the endpoints without the
trailing slash. FastAPI's default redirect_slashes=True answers 307 with
empty body and ab doesn't follow redirects. So those runs timed the emission
of a redirect, not the endpoint.

Resolution

Merge this branch (no new dependencies)


Evidence

The mismatch

@app.post("/sync/big_json_response/")          # app_files/app.py:63
container["uri"] = "/sync/big_json_response"   # test_files/test_combo_best_worst.py:20

Against the unmodified combo containers:

$ curl -s -o /dev/null -w '%{http_code} %{size_download}\n' -X POST \
    -H 'Content-Type: application/json' -d @test_files/requestbody \
    http://127.0.0.1:8131/sync/big_json_response
307 0

$ curl -s -o /dev/null -w '%{http_code} %{size_download}\n' -X POST \
    -H 'Content-Type: application/json' -d @test_files/requestbody \
    http://127.0.0.1:8131/sync/big_json_response/
200 916852

What each URL measures

Harness parameters (ab -c 100 -n 500), driven from a container inside the
compose network, warm-up plus 3 runs averaged:

Container URL RPS non-2xx
combo_worst_sync_json /async/big_json_response 4519.4 1500 / 1500
combo_worst_sync_json /async/big_json_response/ 21.6 0
combo_best_async_orjson /async/big_json_response 17182.7 1500 / 1500
combo_best_async_orjson /async/big_json_response/ 46.1 0

The published 3433 RPS on a 1 MB payload would require ~3.4 GB/s of serialisation
and transfer inside a 2-CPU container.

The guard

AssertionError: 200 non-2xx responses from /sync/big_json_response on port 8131.
The measurement is not hitting the endpoint: FastAPI answers 307 when the
trailing slash of the route is missing from the URL.

test_pool_exhaustion, test_robustness and test_concurrency drive the app
through httpx rather than TestContainer, so the assertion does not affect them.
The middlewares, json_classes, sync_async and server_runners tests already
used correct URLs and are unchanged by this PR.

Deliberately out of scope

The measurement tables in index.md, nginx_port_socket.md,
workers_and_threads.md and keepalive.md came from the broken configurations
and need a fresh CI run on your hardware; numbers from a different machine would
not be comparable, so they are left untouched.

Expect a longer suite: the big-JSON tests previously ran at redirect speed
(~4500 RPS) and now run at endpoint speed (tens of RPS).

@KissPeter

Copy link
Copy Markdown
Owner

Thanks @Vladgasik, let me check it

@KissPeter
KissPeter enabled auto-merge September 23, 2026 20:04
@KissPeter
KissPeter merged commit 620e1ea into KissPeter:main Sep 24, 2026
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants