Enable the x402 gateway's edge controls (0.2.2): OVH denylist, spoofed-browser toll, signed-in exemption - #207
Merged
Merged
Conversation
Bump @profullstack/x402-gateway to 0.2.2 and switch on the two checks for crawlers that do not say who they are: - denyCidrs: the OVH VPS fleet ranges measured 2026-08-28 on rssamplifier (vps-*.vps.ovh.net spoofing Chrome/148) get a tiny 403 before anything else. Last x-forwarded-for hop / x-real-ip only, so a client-seeded first hop cannot get anyone refused. - chargeSpoofedBrowsers: a "Chrome/..." user agent with no Sec-Fetch-Mode header is an HTTP client wearing a copied string and is charged like GPTBot. 0.2.2 never judges anything that declares itself (Googlebot's and Bingbot's evergreen strings pass), nor Firefox/Safari. - exempt: a request carrying a valid-looking Supabase session (sb-auth-token JSON with a JWT-shaped access_token) or a valid-looking v1 API bearer token (btr_ + 64 hex) is never charged. This site is mostly real people, so this is the control that matters most. Tests: OVH last hop and x-real-ip 403 across every listed range, first hop ignored; spoofed Chrome 402, real Chrome passes, Googlebot/Bingbot evergreen and Firefox pass, robots.txt and /crawl still readable; session cookie and API bearer exempt, junk lookalikes are not, and a session does not get a hosting range past the 403. The existing browser helpers now send Sec-Fetch-Mode like every real Chromium. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WJaXiqE9BDoNfoJBhfXroC
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
ThreatCrush Security Scan99 finding(s) HIGH/CRITICAL: 11 | MEDIUM: 29 | LOW: 59
…and 49 more. Full results in the Security tab. Snippets are redacted; ThreatCrush never prints matched credential material. |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
The code-scanning gate flagged src/proxy.test.ts:337, the JWT-shaped session-cookie fixture, under secret-jwt, secret-generic-credential and secret-generic-api-key. The suppression parser takes one rule id per directive and each disable-next-line covers only its own next line, so the fixture is split: the unsigned, 1970-expired JWT and the one-letter refresh token each sit on their own line behind a named directive with the reason written above. The third rule keyed on the combined line and no longer fires. Verified with the CLI the workflow installs (@profullstack/threatcrush@latest, 0.11.9): 2 suppressed, 0 findings. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WJaXiqE9BDoNfoJBhfXroC
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes, for whom
Bumps
@profullstack/x402-gateway0.1.0 -> 0.2.2 and turns on its edge controls insrc/lib/crawl-gateway.ts. The declared-crawler behaviour from #205 is unchanged; this adds handling for crawlers that do not declare themselves.Sec-Fetch-Mode; Firefox and Safari are never judgedAuthorization: Bearer btr_...)exempt)... compatible; Googlebot/2.1 ... Chrome/148 ..., no Sec-Fetch)Chrome/...with noSec-Fetch-Mode(VPS fleets, residential-proxy rotations)402with the x402 offer, same as GPTBot;/robots.txtand/crawlstill readable51.38/16, 54.38/16, 141.94/16, 145.239/16, 149.202/16, 151.80/16, 57.129/16, 213.32/16)403 Not available from this network.before anything else, session or notThe OVH ranges were measured 2026-08-28 on rssamplifier:
vps-*.vps.ovh.nethosts spoofingChrome/148. The list is a named constant with that provenance in a comment.Notes for review
exemptruns before the agent lists in 0.2.2, so a request presenting a valid-looking session is let through even under a declared training UA. That is the intended trade for a site that is mostly people, and it is pinned by a test. "Valid-looking" means thesb-auth-tokenJSON has a JWT-shapedaccess_tokenand arefresh_token; a junk cookie merely namedsb-auth-tokendoes not qualify (tested). No verification happens in the gate; the ordinary session refresh runs afterwards as before.src/lib/api-tokenstokens (btr_+ 64 hex) only; the route still verifies the hash.x-real-ip, else the lastx-forwarded-forhop (the hop our own edge appends). The existing rate limiter insrc/proxy.tsreads the first hop; that is pre-existing and untouched, but worth knowing: the denylist cannot be stepped around by seeding XFF, the rate limiter can.sec-fetch-mode: navigate, as every real Chromium does; the edge-control tests build their own requests with exactly the headers under test.Files
package.json,pnpm-lock.yaml:@profullstack/x402-gateway0.2.2.src/lib/crawl-gateway.ts:denyCidrs,chargeSpoofedBrowsers: true,exempt(+ exportedhasSessionCookie,hasApiBearer).src/proxy.test.ts: +16 edge-control tests; one session test split in two to reflect the exemption.Checks
pnpm typecheck: cleanpnpm lint: 0 errors (248 pre-existing warnings, none in touched files)pnpm test: 2746 passed, 9 skipped; the one failure wassrc/lib/spotify/librespot.test.ts"caps the played list" hitting the 5s cap whilenext buildran concurrently (same flake as Charge AI training crawlers for access with @profullstack/x402-gateway #205/Make the session-refresh test own every Supabase env var it reads #206; passes alone, untouched here)pnpm build: passes,ƒ Proxy (Middleware)compiled,/robots.txtstatic, 134/134 pagesNo new env vars. Not merged.
🤖 Generated with Claude Code
https://claude.ai/code/session_01WJaXiqE9BDoNfoJBhfXroC