fix(server): cache hits preserve harvester user-agent - #5
Merged
Conversation
`PxSolveDispatcher::solve` and `RoutingDispatcher::solve` were constructing the `PxCookieBundle` with the literal string `"px-harvester"` as its `user_agent` field. The non-cached response path masked this because the body field was sourced from `outcome.user_agent` (the real browser UA), but the cache stored the bundle with the placeholder. On the next request for the same target the cache-hit branch in `solve.rs` reads `bundle.user_agent.clone()` into `SolveOutput.user_agent`, so the JSON body returned `"user_agent": "px-harvester"` instead of the harvester's real UA. Downstream clients that use the returned UA to replay requests (so the TLS ClientHello and the `User-Agent` header look consistent to PX/CF) were getting silently mis-routed: TLS said Firefox, header said `px-harvester`, server flagged it. Concretely this surfaced as a 403 from `pedidosya.com.ar/v4/shoplist/vendors` on the second and later solve calls while the first call worked. Fix: derive `user_agent` from `outcome.user_agent` once, then thread the same value into both the bundle and the `SolveOutput`. Adds a regression test (`bundle_user_agent_matches_harvester`) that exercises the dispatcher and asserts both the response UA and the bundle UA equal the harvester's UA. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
KeyCode17
added a commit
that referenced
this pull request
May 18, 2026
Lifts workspace.package.version 1.3.0 -> 1.4.0 and aligns the 16 internal workspace.dependencies pins in lockstep. Changes since 1.3.0: - fix(server): cache hits preserve harvester user-agent (#5) - feat(server): /v1/fetch endpoint backed by handler-owned browsers (#6) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The cookie bundle stored in the cache had its
user_agentfield hard-coded to the literal string"px-harvester". On a cache hit, the response body'suser_agentfield was sourced frombundle.user_agentand therefore returned"px-harvester"instead of the real browser UA that the harvester actually used.Fix: derive
user_agentfromoutcome.user_agentonce in bothPxSolveDispatcher::solveandRoutingDispatcher::solve, then thread the same value into both the bundle and theSolveOutput.Why
Downstream clients that use the returned UA to replay requests (so the TLS ClientHello and the
User-Agentheader look consistent to PX/CF) were getting silently mis-routed on every cache-hit response: TLS impersonation said Firefox, header saidpx-harvester, server flagged the mismatch. Concretely this surfaced as a 403 frompedidosya.com.ar/v4/shoplist/vendorson the second and subsequent solve calls while the first call worked.The first call worked because that path read
outcome.user_agentdirectly into the response body and only masked the issue at the cache boundary.Test plan
cargo fmt --all -- --checkcargo clippy --workspace --all-targets --all-featuresagainst the lefthook rule setcargo test -p px-server --lib routing— 6/6 passing, including the newbundle_user_agent_matches_harvesterregression testcurl /v1/solvefor the same target — every response now returns the same Firefox UA the Camoufox harvester used, not"px-harvester"Notes for the reviewer
px-cli,px-camoufox, orpx-cloudflaretouched.StaticHandlerfixture.wreq/ curl-impersonate to replay the bundle on the same UA — without this fix, the cached UA always trips the server's UA/JA3 consistency check.🤖 Generated with Claude Code