The cloud gate decided, then handed the verdict to something that overruled it - #364
Merged
Merged
Conversation
…rruled it It charged nobody. `judgeCloudClient` correctly identified a caller in a published cloud range and then called `crawl-gateway`'s `handle()`, whose `isPaidAgent` defaults to the training-crawler list. That gateway re-decided the question we had just answered, saw an ordinary Chrome user agent — which is the entire point of this traffic, it is a real headless browser — returned null, and let the request through. Verified against production before and after: a request carrying an AWS ap-southeast-1 address and a Chrome user agent was answered 200. Every unit test passed throughout, because the verdict was never the broken half; the delegation was, and nothing tested it. So: its own gateway with `isPaidAgent: () => true`, the same shape `explorer-gateway` already uses and for the same reason. By the time `handle` runs, who pays has been decided; asking a second, weaker rule to confirm it can only ever disagree. Its own pass header and `/cloud-pass` sales page too, so a pass bought here is not confused with the crawl or explorer passes, which meter different things — and that path is excluded from the gate, since a sales page you can be refused for reading is a loop rather than an offer. Tests now cover the integration rather than only the verdict: that the gate answers 402, that a non-cloud address passes through untouched, and that the user agent cannot overturn an address-based decision. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ThreatCrush Security Scan331 finding(s) HIGH/CRITICAL: 33 | MEDIUM: 40 | LOW: 258
…and 281 more. Full results in the Security tab. Snippets are redacted; ThreatCrush never prints matched credential material. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#363 charged nobody. Caught by testing it against production rather than trusting the unit tests.
The bug
judgeCloudClientcorrectly identified a caller in a published cloud range, then calledcrawl-gateway'shandle()— whoseisPaidAgentdefaults to the training-crawler list. That gateway re-decided the question we had just answered, saw an ordinary Chrome user agent (which is the entire point of this traffic — it is a real headless browser), returned null, and let the request through.Verified against prod with the gate armed:
Every unit test passed throughout, because the verdict was never the broken half. The delegation was, and nothing tested it.
The fix
Its own gateway with
isPaidAgent: () => true— the same shapeexplorer-gatewayalready uses, for the same reason. By the timehandleruns, who pays has been decided; asking a second, weaker rule to confirm it can only ever disagree.Its own pass header (
x-cloud-pass) and/cloud-passsales page too, so a pass bought here is not confused with the crawl or explorer passes, which meter different things. That path is excluded from the gate — a sales page you can be refused for reading is a loop rather than an offer.Testing
Tests now cover the integration, not only the verdict:
curl,GPTBotand Chrome all charge alike)11 in
cloud-gate, 95 across the proxy + footprint suites,tsc --noEmitclean.Note
CLOUD_CHARGE=pagesis already set in prod, so this takes effect on deploy.🤖 Generated with Claude Code