From 468e9e4d3be698d3d5780b7d1bc44996e8312fb1 Mon Sep 17 00:00:00 2001 From: Chris Portscheller Date: Tue, 28 Jul 2026 16:20:57 -0500 Subject: [PATCH 1/4] feat(pow): price challenges by accumulated suspicion, on the clock not the hash MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Difficulty was a near-constant: 4, or 5 from a datacenter address, or up to 6 under a high request rate. That is the wrong knob. Browser JS runs 1-3M hash/s and native code 100-500M/s, so difficulty 6 costs a native solver about a millisecond and a budget Android phone about sixteen seconds. Escalating there taxes the slowest legitimate devices and constrains nobody. Wall-clock does not have that property. Nobody can make less time pass, so a minimum challenge age caps how fast one source can mint tokens regardless of the hardware it brings. Each challenge now carries a signed minAgeMs, and that is where the escalation lands. clean or unknown difficulty 4, 1500ms (unchanged for everyone) 1-2 strong bot verdicts difficulty 4, 4000ms 3-5 difficulty 5, 8000ms 6+ difficulty 5, 15000ms Difficulty now caps at 5 rather than 6. A datacenter address no longer raises it at all — a real person on a corporate VPN or iCloud Private Relay was paying several seconds of blocked hashing for a shared IP — and raises the time floor instead, which a hosted scraper feels as reduced throughput and a person filling in a form does not feel at all. At the 1.5s baseline one address mints roughly 40 tokens a minute; at 15s, four. The client is told minAgeMs and waits it out, so an ordinary visitor pays the elevated cost as a short delay rather than as a worse score. That distinction matters most for anyone sharing an egress address with whatever earned the delay. A client that submits early is still scored, but the below-baseline case keeps its existing strength while the between-baseline-and-floor case is contributory: an older cached client does not know to wait and should not be read as automation for it. Suspicion is held per (site key, address) for 15 minutes, records only verdicts at or above 0.8, and stores nothing but timestamps. It is not Workstream F 10.1 cross-session correlation and must not grow into it — same shape and privacy class as the rate limiter beside it. Bench: human FPR 0.00% (0/126), agent TPR 97.33%, both unchanged. Two harness defects surfaced on the way, both invisible until a signal was keyed by source: Sample addresses were a hash of the sample id, putting 180 samples into a 508-address space. They collided constantly — 25 shared addresses, 24 of them putting an agent and a human on the same IP — so the corpus was manufacturing shared egress its labels never claimed. The first source-keyed signal read 12.7% FPR on a panel whose real answer was zero. Addresses are now assigned by position and the pool throws rather than wrapping. Runs also inherited server-side state from each other: one signal fired on 2/75 agents against a fresh server and 74/75 against one already benchmarked. Each run now namespaces its own site key. bench/package.json pointed npm test at lib/lib.test.js, which has never existed; CI runs npm test in server-node, so nothing invoked it. Replaced with real tests for the address pool and wired into the bench workflow. Also completes the HMAC hardening pass in server-node/index.js, which it had missed: token signatures were truncated to 64 bits on both generation and verification while Go, Python and the standalone server used the full digest. Tokens issued by an older library build will not verify after this upgrade. --- .github/workflows/bench.yml | 7 + README.md | 50 ++++++- bench/README.md | 39 ++++-- bench/lib/replay.js | 72 ++++++++++- bench/lib/replay.test.js | 66 ++++++++++ bench/run-bench.js | 14 ++ client/fcaptcha.js | 43 +++++- server-go/main.go | 6 + server-go/scoring.go | 77 +++++++---- server-go/suspicion.go | 207 +++++++++++++++++++++++++++++ server-go/suspicion_test.go | 223 ++++++++++++++++++++++++++++++++ server-node/index.js | 69 +++++++--- server-node/package.json | 5 +- server-node/server.js | 70 +++++++--- server-node/suspicion.js | 172 ++++++++++++++++++++++++ server-node/suspicion.test.js | 175 +++++++++++++++++++++++++ server-python/server.py | 77 ++++++++--- server-python/suspicion.py | 164 +++++++++++++++++++++++ server-python/test_suspicion.py | 118 +++++++++++++++++ test/test-detection.js | 126 ++++++++++++++++++ 20 files changed, 1682 insertions(+), 98 deletions(-) create mode 100644 bench/lib/replay.test.js create mode 100644 server-go/suspicion.go create mode 100644 server-go/suspicion_test.go create mode 100644 server-node/suspicion.js create mode 100644 server-node/suspicion.test.js create mode 100644 server-python/suspicion.py create mode 100644 server-python/test_suspicion.py diff --git a/.github/workflows/bench.yml b/.github/workflows/bench.yml index dded049..7e9f3da 100644 --- a/.github/workflows/bench.yml +++ b/.github/workflows/bench.yml @@ -45,6 +45,13 @@ jobs: - name: End-to-end detection suite run: node test/test-detection.js + # The harness itself, before it is trusted to measure anything. Sample + # addresses have to stay unique or every per-source measurement is taken + # against manufactured IP sharing. + - name: Harness unit tests + working-directory: bench + run: npm test + # Replays the committed corpus. No browser needed: capture is a separate, # manual step whose output lives in bench/corpus/. - name: Benchmark gate diff --git a/README.md b/README.md index a4aa180..1e7e96a 100644 --- a/README.md +++ b/README.md @@ -293,8 +293,34 @@ to honest mobile users for no gain in the constraint that actually binds. Number in [bench/POW-PRIMITIVE.md](bench/POW-PRIMITIVE.md). If you want proof of work to genuinely raise an attacker's cost, the lever is the -server-measured elapsed time, not the hash. That is what adaptive difficulty -should key on. +server-measured elapsed time, not the hash. That is what the adaptive cost keys +on. + +Each challenge carries a `minAgeMs`: how long it must be held before a solution +is accepted without penalty. A visitor who has done nothing wrong gets the same +1500ms floor the server has always applied. A source that has recently produced +strong bot verdicts gets more, up to 15 seconds — which takes it from roughly 40 +tokens a minute to four, on hardware no amount of money can speed up. + +Difficulty barely moves, and now caps at 5 rather than 6. Difficulty 6 costs a +native solver about a millisecond and a budget Android phone about sixteen +seconds; escalating there taxes the slowest legitimate devices and constrains +nobody. Being on a datacenter address no longer raises difficulty at all — a +real person on a corporate VPN or iCloud Private Relay was paying several seconds +of blocked hashing for a shared IP — and raises the time floor instead, which a +hosted scraper feels as reduced throughput and a person filling in a form does +not feel at all. + +The client is told `minAgeMs` and waits it out, so for an ordinary visitor the +cost is a short delay rather than a worse score. That distinction matters most +for anyone sharing an egress address with whatever earned the delay. A client +that submits early anyway is scored, but only as contributory evidence — an +older cached client does not know to wait, and should not be treated as +automation for it. + +Suspicion is held per (site key, address) for 15 minutes, records only verdicts +at or above 0.8, and stores nothing but timestamps. It is not cross-session +correlation and should not grow into it. ### Behavioral Biometrics - Mouse trajectory, velocity, and acceleration curves @@ -381,6 +407,7 @@ Get a Proof of Work challenge. Called automatically by the client on page load. "challengeId": "abc123...", "prefix": "abc123:1703356800000:4", "difficulty": 4, + "minAgeMs": 1500, "expiresAt": 1703357100000, "nonce": "f1e2d3...", "sig": "def456..." @@ -389,9 +416,22 @@ Get a Proof of Work challenge. Called automatically by the client on page load. The `nonce` is generated per-challenge by the server; the client echoes it back in `signals.meta.challengeNonce` and the server verifies it, preventing challenge replay. -Difficulty scales based on: -- Datacenter IPs: +1 difficulty -- High request rate: +1 difficulty (max 6) +`minAgeMs` is how long the client should hold the solved challenge before +submitting. The bundled client does this for you. Both fields are covered by +`sig`, so neither can be talked down on the way back. + +Cost scales with what the requesting source has recently been caught doing: + +| source | difficulty | minAgeMs | +|---|---|---| +| clean, or unknown | 4 | 1500 | +| 1–2 recent strong bot verdicts | 4 | 4000 | +| 3–5 | 5 | 8000 | +| 6+ | 5 | 15000 | + +Datacenter addresses, high request rates and exceeded rate limits raise the time +floor only — never the difficulty. See [what proof of work does and does not buy +you](#what-the-proof-of-work-does-and-does-not-buy-you) for why. ### POST /api/verify Verify a checkbox CAPTCHA submission. diff --git a/bench/README.md b/bench/README.md index 7f12b15..bb61a12 100644 --- a/bench/README.md +++ b/bench/README.md @@ -116,14 +116,37 @@ sample and measures nothing but itself. `lib/pow.js` paces the solve and waits out the challenge age, so `duration` stays true wall-clock — a fabricated duration would be measuring the fabrication. -**Every sample gets its own client IP.** The server escalates PoW difficulty per -`pow:{siteKey}:{ip}`, so a few hundred samples from one address would spend the -run solving 16.7M-hash challenges *and* pick up rate-limit detections that the -earlier samples never saw — the measurement would drift with position in the -corpus. Samples present a distinct RFC 5737 documentation address via -`X-Forwarded-For`, which works because loopback is in the default trusted-proxy -set. Agent classes that really do arrive from datacenter ranges can say so with -`clientIp`. +**Every sample gets its own client IP.** The server keys rate limits and +accumulated suspicion per `pow:{siteKey}:{ip}`, so a few hundred samples from one +address would pick up escalations the earlier samples never saw — the measurement +would drift with position in the corpus. Samples present a distinct RFC 5737 +documentation address via `X-Forwarded-For`, which works because loopback is in +the default trusted-proxy set. Agent classes that really do arrive from +datacenter ranges can say so with `clientIp`. + +Those addresses are assigned **by position**, not by hashing the sample id, and +that distinction was learned the hard way. Hashing put 180 samples into a +508-address space; by the birthday bound they collided constantly — 25 shared +addresses, 24 of them putting an agent and a human on the same IP. The corpus was +manufacturing shared egress its labels never claimed, and nothing noticed for as +long as no signal was keyed by source. The first one that was, read 12.7% FPR on +a panel whose real answer was zero. `lib/replay.test.js` now holds the pool +collision-free and throws rather than wrapping if the corpus outgrows it. + +**Each run namespaces its own server-side state.** The same per-source keying +means a second run against a long-lived server measures the residue of the first. +Measured: one signal fired on 2/75 agents against a fresh server and 74/75 +against one already benchmarked. Each run generates a unique site key, so runs +are independent without the harness having to own the server's lifecycle. + +**The pacer does not honour the server's `minAgeMs`.** The server tells clients +how long to hold a solved challenge and the shipped client obeys; the harness +deliberately waits a fixed interval just past the universal baseline instead. A +replayer that honoured the delay would satisfy the timing gate by construction, +and the panel could never observe it firing on a human — the same trap as pinning +a property during normalization and then reporting that nothing disagrees with +it. Leaving the pacer fixed means any escalation applied to a human persona shows +up as a signal over budget, which is what it should do. --- diff --git a/bench/lib/replay.js b/bench/lib/replay.js index c8fcb7c..c57b5ae 100644 --- a/bench/lib/replay.js +++ b/bench/lib/replay.js @@ -22,6 +22,22 @@ * connection really does not. Samples may set `clientIp` to say so; anything * that does not gets a documentation-range address (RFC 5737), which no * detection treats as anything in particular. + * + * Those addresses are assigned by position and must stay unique — see + * neutralIpAt for what happened when they were hashed instead. + * + * ## Why the pacer does not honour the server's minAgeMs + * + * The server tells a client how long to hold a solved challenge, and the + * shipped client waits that long. This harness deliberately does not: it always + * waits a fixed interval just past the universal baseline. + * + * That is the conservative choice for measuring false positives. A replayer + * that honoured the delay would satisfy the timing gate by construction and the + * panel could never observe it firing on a human — the same trap as pinning a + * property during normalization and then reporting that nothing disagrees with + * it. Leaving the pacer fixed means any escalation applied to a human persona + * shows up as a signal over budget, which is exactly what it should do. */ const crypto = require('crypto'); @@ -63,13 +79,46 @@ function distinguishDevice(signals, id) { }; } -/** RFC 5737 TEST-NET-2 and TEST-NET-3: reserved, routable nowhere, in no CIDR list. */ -const NEUTRAL_PREFIXES = ['198.51.100', '203.0.113']; +/** RFC 5737 TEST-NET-1, -2 and -3: reserved, routable nowhere, in no CIDR list. */ +const NEUTRAL_PREFIXES = ['192.0.2', '198.51.100', '203.0.113']; + +/** .0 and .255 are the network and broadcast addresses; keep to 1-254. */ +const NEUTRAL_POOL_SIZE = NEUTRAL_PREFIXES.length * 254; + +/** + * The address for the Nth sample in a corpus. + * + * Assigned by position rather than by hashing the sample id, and that is the + * whole point. Hashing put 180 samples into a 508-address space, which by the + * birthday bound collided constantly: 25 shared addresses, 24 of them putting + * an agent and a human on the same IP. The corpus was manufacturing shared + * egress that its labels never claimed, so any per-source signal — rate limits, + * reputation, accumulated suspicion — was being measured against a fiction. + * + * Nothing caught it until a per-source signal existed to notice, which is the + * same shape of mistake as normalization pinning a property the panel then + * could not disagree about. Assign by index and the collisions cannot come back. + */ +function neutralIpAt(index) { + if (index >= NEUTRAL_POOL_SIZE) { + throw new Error( + `corpus has outgrown the neutral address pool (${NEUTRAL_POOL_SIZE} addresses). ` + + 'Add another RFC 5737 prefix rather than letting samples share an address — ' + + 'shared addresses silently break every per-source measurement.' + ); + } + return `${NEUTRAL_PREFIXES[Math.floor(index / 254)]}.${(index % 254) + 1}`; +} +/** + * Address for a single sample replayed outside a corpus run. + * + * Collision-prone by construction — see neutralIpAt. Fine for a one-off, wrong + * for a panel. + */ function neutralIpFor(id) { const digest = crypto.createHash('sha256').update(id).digest(); const prefix = NEUTRAL_PREFIXES[digest[0] % NEUTRAL_PREFIXES.length]; - // .0 and .255 are the network and broadcast addresses; keep to 1-254. return `${prefix}.${(digest[1] % 254) + 1}`; } @@ -134,6 +183,13 @@ async function replaySample(serverUrl, sample, opts = {}) { */ async function replayCorpus(serverUrl, samples, opts = {}) { const concurrency = opts.concurrency || 4; + + // Give every sample its own address before anything is replayed, so no two + // samples share server-side per-source state. + samples.forEach((sample, i) => { + if (!sample.clientIp) sample.clientIp = neutralIpAt(i); + }); + const results = new Array(samples.length); let next = 0; let done = 0; @@ -152,4 +208,12 @@ async function replayCorpus(serverUrl, samples, opts = {}) { return results; } -module.exports = { DEFAULT_UA, distinguishDevice, neutralIpFor, replayCorpus, replaySample }; +module.exports = { + DEFAULT_UA, + distinguishDevice, + neutralIpAt, + neutralIpFor, + NEUTRAL_POOL_SIZE, + replayCorpus, + replaySample, +}; diff --git a/bench/lib/replay.test.js b/bench/lib/replay.test.js new file mode 100644 index 0000000..9d2c9ac --- /dev/null +++ b/bench/lib/replay.test.js @@ -0,0 +1,66 @@ +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert'); + +const { neutralIpAt, neutralIpFor, NEUTRAL_POOL_SIZE } = require('./replay'); + +// These exist because the harness had no tests, and the defect they guard +// against was invisible for exactly that reason. +// +// Sample addresses used to be a hash of the sample id, which put 180 samples +// into a 508-address space. By the birthday bound they collided constantly: 25 +// shared addresses, 24 of them putting an agent and a human on the same IP. The +// corpus was manufacturing shared egress its labels never claimed, so every +// per-source measurement was being taken against a fiction — and nothing +// noticed until a per-source signal existed to notice. + +test('every position gets a distinct address', () => { + const seen = new Set(); + for (let i = 0; i < NEUTRAL_POOL_SIZE; i++) { + const ip = neutralIpAt(i); + assert.ok(!seen.has(ip), `position ${i} reused ${ip}`); + seen.add(ip); + } + assert.strictEqual(seen.size, NEUTRAL_POOL_SIZE); +}); + +test('addresses stay inside the RFC 5737 documentation ranges', () => { + // Reserved, routable nowhere, and in no datacenter CIDR list — so an address + // never becomes a signal by accident. + const allowed = /^(192\.0\.2|198\.51\.100|203\.0\.113)\.(\d{1,3})$/; + for (let i = 0; i < NEUTRAL_POOL_SIZE; i++) { + const m = allowed.exec(neutralIpAt(i)); + assert.ok(m, `${neutralIpAt(i)} is outside the documentation ranges`); + const host = Number(m[2]); + assert.ok(host >= 1 && host <= 254, `${neutralIpAt(i)} is a network or broadcast address`); + } +}); + +// Silently wrapping around would reintroduce the collisions this replaced. +test('outgrowing the pool throws rather than wrapping', () => { + assert.throws(() => neutralIpAt(NEUTRAL_POOL_SIZE), /outgrown the neutral address pool/); +}); + +test('assignment is stable across calls', () => { + assert.strictEqual(neutralIpAt(0), neutralIpAt(0)); + assert.strictEqual(neutralIpAt(700), neutralIpAt(700)); +}); + +// Kept for one-off replays, and collision-prone by construction. The test +// records that it is not safe for a panel, so nobody reaches for it there. +test('the hashed variant is deterministic but may collide', () => { + assert.strictEqual(neutralIpFor('a/b/0000'), neutralIpFor('a/b/0000')); + + const seen = new Map(); + let collisions = 0; + for (let i = 0; i < 200; i++) { + const ip = neutralIpFor(`sample/${i}`); + if (seen.has(ip)) collisions++; + seen.set(ip, i); + } + assert.ok( + collisions > 0, + 'if this stops colliding the pool grew; prefer neutralIpAt for corpora regardless' + ); +}); diff --git a/bench/run-bench.js b/bench/run-bench.js index 9b44e7f..a49e822 100644 --- a/bench/run-bench.js +++ b/bench/run-bench.js @@ -24,6 +24,7 @@ const path = require('path'); const { deriveVariant, loadCorpus } = require('./lib/corpus'); const { computeMetrics, evaluateGate } = require('./lib/metrics'); const { renderReport } = require('./lib/report'); +const crypto = require('crypto'); const { replayCorpus } = require('./lib/replay'); const { makeRng } = require('./lib/rng'); @@ -109,8 +110,21 @@ async function main() { ); } + // Namespace this run's server-side per-source state. + // + // The server keys rate limits and accumulated suspicion by (siteKey, address). + // Sample addresses are assigned by position, so two runs against the same + // long-lived server reuse the same addresses and the second run measures the + // residue of the first. Measured: one signal fired on 2/75 agents against a + // fresh server and 74/75 against one that had already been benchmarked. + // + // A per-run site key makes runs independent without the harness having to own + // the server's lifecycle. + const siteKey = `bench-${crypto.randomBytes(6).toString('hex')}`; + const started = Date.now(); const results = await replayCorpus(args.server, corpus, { + siteKey, concurrency: args.concurrency, onProgress: args.quiet ? null diff --git a/client/fcaptcha.js b/client/fcaptcha.js index 90a8ebd..985ece2 100644 --- a/client/fcaptcha.js +++ b/client/fcaptcha.js @@ -2229,6 +2229,7 @@ constructor() { this.workers = []; this.challenge = null; + this.challengeReceivedAt = null; this.solution = null; this.solving = false; this.solvePromise = null; @@ -2319,6 +2320,11 @@ return this._generateLocalChallenge(); } this.challenge = await response.json(); + // Time the wait from when the challenge arrived, not from the + // timestamp inside it. Receipt is necessarily later than issue, so a + // wait measured from here always clears the server's floor — and it + // costs nothing to a client whose clock disagrees with the server's. + this.challengeReceivedAt = Date.now(); return this.challenge; } catch (e) { console.warn('PoW challenge fetch failed, using local challenge:', e); @@ -2335,9 +2341,34 @@ expiresAt: Date.now() + 300000, local: true // Flag that this is a local challenge }; + this.challengeReceivedAt = Date.now(); return this.challenge; } + // Hold a solved challenge until it is old enough for the server to accept + // without penalty. + // + // The server prices a challenge by how suspicious the requesting source + // has recently been, and the part of that price it can actually enforce is + // wall-clock: an attacker can buy faster hardware but cannot make less time + // pass. Waiting here is what turns that price into a short delay for an + // ordinary visitor instead of a worse score — which matters most for + // someone who shares an address with whatever earned the delay. + // + // Usually free: by the time a person has finished with the form, the wait + // has already elapsed. + async _awaitMinAge() { + const minAge = Number(this.challenge && this.challenge.minAgeMs) || 0; + if (!minAge || !this.challengeReceivedAt) return; + + // Clamp. A server that asks for an implausible delay gets a bounded one + // rather than a page that never finishes. + const remaining = Math.min(minAge, 30000) - (Date.now() - this.challengeReceivedAt); + if (remaining > 0) { + await new Promise((r) => setTimeout(r, remaining)); + } + } + // Start solving in background (legacy - solve without signals binding) async startSolving(siteKey) { return this._solve(siteKey, null); @@ -2366,8 +2397,17 @@ const finish = (fn, value) => { if (settled) return; settled = true; - this.solving = false; this._terminateWorkers(); + // Resolvers wait out the server's floor first; rejections do not, + // since there is no solution to hold back. + if (fn === resolve) { + this._awaitMinAge().then(() => { + this.solving = false; + fn(value); + }); + return; + } + this.solving = false; fn(value); }; @@ -2427,6 +2467,7 @@ reset() { this._terminateWorkers(); this.challenge = null; + this.challengeReceivedAt = null; this.solution = null; this.solving = false; this.solvePromise = null; diff --git a/server-go/main.go b/server-go/main.go index b1583e8..c6aa059 100644 --- a/server-go/main.go +++ b/server-go/main.go @@ -620,6 +620,11 @@ type PoWChallengeResponse struct { ExpiresAt int64 `json:"expiresAt"` Nonce string `json:"nonce"` Sig string `json:"sig"` + // MinAgeMs tells the client how long to hold a solved challenge before + // submitting it. Honouring it is how an ordinary visitor pays an elevated + // cost as a short wait instead of as a worse score — which matters most for + // people sharing an egress address with whatever earned the elevation. + MinAgeMs int64 `json:"minAgeMs"` } func powChallengeHandler(engine *ScoringEngine, trust *ProxyTrust, siteKeys *SiteKeyGuard) http.HandlerFunc { @@ -637,6 +642,7 @@ func powChallengeHandler(engine *ScoringEngine, trust *ProxyTrust, siteKeys *Sit ExpiresAt: challenge.ExpiresAt, Nonce: challenge.Nonce, Sig: challenge.Sig, + MinAgeMs: challenge.MinAgeMs, } w.Header().Set("Content-Type", "application/json") diff --git a/server-go/scoring.go b/server-go/scoring.go index 3689886..7107c74 100644 --- a/server-go/scoring.go +++ b/server-go/scoring.go @@ -73,7 +73,11 @@ type PoWChallenge struct { ExpiresAt int64 `json:"expiresAt"` Nonce string `json:"nonce"` Sig string `json:"sig"` - IP string `json:"-"` // Not sent to client + // MinAgeMs is how old this challenge must be before its solution is + // accepted without penalty. Sent to the client, which waits it out, and + // covered by Sig so it cannot be lowered on the way back. + MinAgeMs int64 `json:"minAgeMs"` + IP string `json:"-"` // Not sent to client } // PoWSolution from client @@ -91,6 +95,9 @@ type PoWVerifyResult struct { Difficulty int ServerElapsed int64 Nonce string + // MinAgeMs is the floor this particular challenge was issued with, which + // varies with how suspicious the source was at the time. + MinAgeMs int64 } // PoWChallengeStore manages challenges and replay-protects spent solutions. @@ -178,6 +185,7 @@ type ScoringEngine struct { fingerprintStore *FingerprintStore powStore *PoWChallengeStore tokenStore *TokenStore + suspicion *SuspicionLedger weights map[ThreatCategory]float64 uaPatterns []*regexp.Regexp webBotAuth *webbotauth.Verifier @@ -250,6 +258,7 @@ func NewScoringEngine(secretKey string) *ScoringEngine { fingerprintStore: newFingerprintStore(), powStore: newPoWChallengeStore(), tokenStore: newTokenStore(), + suspicion: NewSuspicionLedger(), weights: map[ThreatCategory]float64{ CategoryVisionAI: 0.15, CategoryHeadless: 0.15, @@ -364,13 +373,31 @@ func (e *ScoringEngine) VerifyWithHeaders(signals map[string]interface{}, ip, si } } - if powResult.Valid && powResult.ServerElapsed < 1500 { - detections = append(detections, DetectionResult{ - Category: CategoryBot, - Score: 0.8, - Confidence: 0.85, - Reason: fmt.Sprintf("Challenge solved too fast (%dms server-side)", powResult.ServerElapsed), - }) + // Two thresholds, because they mean different things. Under the + // universal baseline nothing legitimate can have happened: no human + // completes an interaction that fast, so it scores as it always has. + // + // Between the baseline and this source's own elevated floor is weaker + // evidence. A client that predates adaptive cost does not know to wait, + // and neither does one served from a stale cache, so a full-strength + // penalty there would punish the wrong people. It contributes instead. + if powResult.Valid { + switch { + case powResult.ServerElapsed < baseMinAgeMs: + detections = append(detections, DetectionResult{ + Category: CategoryBot, + Score: 0.8, + Confidence: 0.85, + Reason: fmt.Sprintf("Challenge solved too fast (%dms server-side)", powResult.ServerElapsed), + }) + case powResult.ServerElapsed < powResult.MinAgeMs: + detections = append(detections, DetectionResult{ + Category: CategoryBot, + Score: 0.5, + Confidence: 0.5, + Reason: fmt.Sprintf("Challenge submitted before the required delay for this source (%dms of %dms)", powResult.ServerElapsed, powResult.MinAgeMs), + }) + } } } else { // No PoW solution provided - hard fail @@ -459,6 +486,12 @@ func (e *ScoringEngine) VerifyWithHeaders(signals map[string]interface{}, ip, si token = e.generateToken(ip, siteKey, finalScore) } + // Feed the ledger so the next challenge this source asks for is priced on + // what it just did. Recorded here rather than in the handlers so every + // caller — both endpoints and the library API — contributes without having + // to remember to. + e.suspicion.Record(siteKey, ip, finalScore) + return &VerificationResult{ Success: success, Score: finalScore, @@ -500,29 +533,23 @@ func (e *ScoringEngine) GeneratePoWChallenge(siteKey, ip string, isDatacenter bo now := time.Now().UnixMilli() expiresAt := now + (5 * 60 * 1000) // 5 minutes - // Difficulty scaling - difficulty := 4 // Default: ~100-500ms on average hardware - if isDatacenter { - difficulty = 5 // Harder for datacenter IPs - } - - // Check rate for this IP + // Cost scaling. See suspicion.go for why the escalation lands almost + // entirely on MinAgeMs rather than on Difficulty. rateKey := "pow:" + siteKey + ":" + ip - _, count := e.rateLimiter.Check(rateKey, 60, 20) - if count > 10 { - difficulty = min(6, difficulty+1) - } + exceeded, count := e.rateLimiter.Check(rateKey, 60, 20) + cost := ComputeChallengeCost(e.suspicion.Count(siteKey, ip), isDatacenter, count, exceeded) - prefix := challengeID + ":" + formatInt64(now) + ":" + formatInt(difficulty) + prefix := challengeID + ":" + formatInt64(now) + ":" + formatInt(cost.Difficulty) challenge := &PoWChallenge{ ID: challengeID, SiteKey: siteKey, Prefix: prefix, - Difficulty: difficulty, + Difficulty: cost.Difficulty, Timestamp: now, ExpiresAt: expiresAt, Nonce: nonce, + MinAgeMs: cost.MinAgeMs, IP: ip, } @@ -534,6 +561,7 @@ func (e *ScoringEngine) GeneratePoWChallenge(siteKey, ip string, isDatacenter bo "expiresAt": challenge.ExpiresAt, "difficulty": challenge.Difficulty, "prefix": challenge.Prefix, + "minAgeMs": challenge.MinAgeMs, }) h := hmac.New(sha256.New, []byte(e.secretKey)) h.Write(sigData) @@ -611,7 +639,12 @@ func (e *ScoringEngine) VerifyPoWSolution(solution *PoWSolution, siteKey string, // Delete challenge (one-time use) — inline, lock already held delete(e.powStore.challenges, solution.ChallengeID) - return PoWVerifyResult{Valid: true, Difficulty: challenge.Difficulty, ServerElapsed: serverElapsed, Nonce: challenge.Nonce} + minAge := challenge.MinAgeMs + if minAge <= 0 { + minAge = baseMinAgeMs // challenge predates adaptive cost + } + + return PoWVerifyResult{Valid: true, Difficulty: challenge.Difficulty, ServerElapsed: serverElapsed, Nonce: challenge.Nonce, MinAgeMs: minAge} } func formatInt64(n int64) string { diff --git a/server-go/suspicion.go b/server-go/suspicion.go new file mode 100644 index 0000000..7771d51 --- /dev/null +++ b/server-go/suspicion.go @@ -0,0 +1,207 @@ +package main + +import ( + "sync" + "time" + + "github.com/hashicorp/golang-lru/v2/expirable" +) + +// Adaptive challenge cost: what a source pays is a function of what that source +// has recently been caught doing, rather than a constant. +// +// # Why the cost is mostly time, not hashing +// +// A constant difficulty is strictly dominated: it either fails to inconvenience +// an attacker or it does hurt real users. The measurements behind that claim: +// browser JS runs 1-3M hash/s, native code 100-500M/s, so difficulty 6 costs a +// native solver about a millisecond and a budget Android phone about sixteen +// seconds. Raising difficulty is close to a pure tax on the slowest legitimate +// devices. +// +// Wall-clock is the knob that does not have that property. Nobody can make less +// time pass, so a minimum challenge age caps how fast one source can mint +// tokens no matter what hardware it brings. So suspicion moves the time floor +// first and difficulty barely at all. +// +// # What this deliberately is not +// +// This is not Workstream F 10.1 (cross-session correlation). It stores strong- +// verdict timestamps per source, nothing else: no behavioral vectors, no +// per-fingerprint history, no traces that survive the window. It is the same +// shape and the same privacy class as the rate limiter sitting next to it, and +// it should stay that way — 10.1 has a privacy load this does not, and the two +// should not be conflated because they happen to both be "server-side memory". + +const ( + // suspicionStrongScore is the verdict score at or above which a + // verification counts as evidence. Deliberately high: a marginal verdict is + // exactly the case where the scoring might be wrong about a real person, and + // making the next person from that address wait is not worth the guess. + suspicionStrongScore = 0.8 + + // suspicionWindow is how long a strong verdict keeps counting. Short enough + // that a shared egress address recovers on its own within a coffee break. + suspicionWindow = 15 * time.Minute + + // suspicionMaxHits bounds the per-source slice. Only the count matters and + // every tier saturates well below this, so there is nothing to gain from + // remembering more. + suspicionMaxHits = 16 + + // suspicionMaxSources bounds the table. Sources with no strong verdicts + // never get an entry at all, so this only has to cover addresses actively + // failing verification. + suspicionMaxSources = 50_000 +) + +// SuspicionLedger records recent strong verdicts per source. +// +// Entries are created only when a source produces a strong verdict, so the +// common case — a legitimate visitor — allocates nothing and looks up nothing +// but a miss. +type SuspicionLedger struct { + mu sync.Mutex + hits *expirable.LRU[string, []int64] +} + +func NewSuspicionLedger() *SuspicionLedger { + return &SuspicionLedger{ + hits: expirable.NewLRU[string, []int64](suspicionMaxSources, nil, suspicionWindow), + } +} + +func suspicionKey(siteKey, ip string) string { + return siteKey + "|" + ip +} + +// Record notes a verdict. Scores below the strong threshold are ignored +// entirely rather than recorded and weighted, so a source that merely looks +// unusual never accumulates anything. +func (s *SuspicionLedger) Record(siteKey, ip string, score float64) { + if s == nil || score < suspicionStrongScore || ip == "" { + return + } + + s.mu.Lock() + defer s.mu.Unlock() + + key := suspicionKey(siteKey, ip) + now := time.Now().UnixMilli() + cutoff := now - suspicionWindow.Milliseconds() + + existing, _ := s.hits.Get(key) + kept := make([]int64, 0, len(existing)+1) + for _, t := range existing { + if t > cutoff { + kept = append(kept, t) + } + } + kept = append(kept, now) + if len(kept) > suspicionMaxHits { + kept = kept[len(kept)-suspicionMaxHits:] + } + + s.hits.Add(key, kept) +} + +// Count returns how many strong verdicts this source produced inside the +// window. The LRU's TTL runs from the last write, so counting from the +// timestamps rather than trusting the entry's existence is what makes an old +// hit actually decay while newer ones keep the entry alive. +func (s *SuspicionLedger) Count(siteKey, ip string) int { + if s == nil || ip == "" { + return 0 + } + + s.mu.Lock() + defer s.mu.Unlock() + + hits, ok := s.hits.Get(suspicionKey(siteKey, ip)) + if !ok { + return 0 + } + + cutoff := time.Now().UnixMilli() - suspicionWindow.Milliseconds() + n := 0 + for _, t := range hits { + if t > cutoff { + n++ + } + } + return n +} + +// ChallengeCost is what a source has to pay for a challenge. +type ChallengeCost struct { + // Difficulty is the leading-zero count the PoW hash must reach. + Difficulty int + // MinAgeMs is how old the challenge must be before a solution for it is + // accepted without penalty. The client is told this value and waits it out, + // so for an honest client the cost is latency rather than a worse score. + MinAgeMs int64 +} + +// Baseline cost. A clean visitor pays exactly this, which is what the server +// has always charged everyone. +const ( + baseDifficulty = 4 + baseMinAgeMs = 1500 +) + +// maxDifficulty caps the compute knob at 5, below the 6 this server used to +// reach. Difficulty 6 buys about a millisecond of attacker time and spends +// about sixteen seconds of a budget phone's; the escalation belongs in MinAgeMs +// where an attacker cannot buy their way out of it. +const maxDifficulty = 5 + +// maxMinAgeMs caps the time knob at 15s. At the 1.5s baseline one address can +// mint roughly 40 tokens a minute; at 15s, four. Pushing further buys little +// and is felt by anyone sharing a poisoned egress address. +const maxMinAgeMs = 15_000 + +// ComputeChallengeCost maps accumulated suspicion onto a cost. +// +// Note what does NOT raise difficulty here: being on a datacenter address. That +// used to jump straight to difficulty 5, which charges a real person on a +// corporate VPN or iCloud Private Relay several seconds of blocked hashing on a +// slow phone for the offence of having a shared IP. It now moves the time floor +// instead, which a datacenter-hosted scraper feels as reduced throughput and a +// person filling in a form does not feel at all. +func ComputeChallengeCost(strongHits int, isDatacenter bool, requestCount int, rateExceeded bool) ChallengeCost { + cost := ChallengeCost{Difficulty: baseDifficulty, MinAgeMs: baseMinAgeMs} + + switch { + case strongHits >= 6: + cost.Difficulty, cost.MinAgeMs = 5, 15_000 + case strongHits >= 3: + cost.Difficulty, cost.MinAgeMs = 5, 8_000 + case strongHits >= 1: + cost.MinAgeMs = 4_000 + } + + // Floors from signals that are suggestive rather than damning. They raise + // the time floor and never the difficulty. + raiseTo := func(ms int64) { + if cost.MinAgeMs < ms { + cost.MinAgeMs = ms + } + } + if isDatacenter { + raiseTo(3_000) + } + if requestCount > 10 { + raiseTo(6_000) + } + if rateExceeded { + raiseTo(10_000) + } + + if cost.Difficulty > maxDifficulty { + cost.Difficulty = maxDifficulty + } + if cost.MinAgeMs > maxMinAgeMs { + cost.MinAgeMs = maxMinAgeMs + } + return cost +} diff --git a/server-go/suspicion_test.go b/server-go/suspicion_test.go new file mode 100644 index 0000000..567c9c5 --- /dev/null +++ b/server-go/suspicion_test.go @@ -0,0 +1,223 @@ +package main + +import ( + "crypto/hmac" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "testing" +) + +// The property that matters most: a visitor who has done nothing wrong pays +// exactly what everyone paid before adaptive cost existed. If this drifts, the +// feature has started taxing the people it was designed not to touch. +func TestCleanSourcePaysTheBaseline(t *testing.T) { + cost := ComputeChallengeCost(0, false, 0, false) + if cost.Difficulty != baseDifficulty || cost.MinAgeMs != baseMinAgeMs { + t.Errorf("a clean source must pay the baseline, got difficulty %d / minAge %dms (want %d / %dms)", + cost.Difficulty, cost.MinAgeMs, baseDifficulty, baseMinAgeMs) + } +} + +func TestCostEscalatesWithStrongVerdicts(t *testing.T) { + cases := []struct { + hits int + wantDifficulty int + wantMinAge int64 + }{ + {0, 4, 1500}, + {1, 4, 4000}, + {2, 4, 4000}, + {3, 5, 8000}, + {5, 5, 8000}, + {6, 5, 15000}, + {50, 5, 15000}, + } + for _, c := range cases { + got := ComputeChallengeCost(c.hits, false, 0, false) + if got.Difficulty != c.wantDifficulty || got.MinAgeMs != c.wantMinAge { + t.Errorf("%d strong verdicts: got difficulty %d / minAge %dms, want %d / %dms", + c.hits, got.Difficulty, got.MinAgeMs, c.wantDifficulty, c.wantMinAge) + } + } +} + +// The escalation must stay on the knob an attacker cannot buy their way out of. +// Difficulty 6 costs a native solver about a millisecond and a budget phone +// about sixteen seconds, so reaching it would be a tax on slow devices and +// nothing else. +func TestDifficultyNeverExceedsFive(t *testing.T) { + for hits := 0; hits < 100; hits++ { + for _, dc := range []bool{false, true} { + for _, ex := range []bool{false, true} { + got := ComputeChallengeCost(hits, dc, 1000, ex) + if got.Difficulty > 5 { + t.Fatalf("difficulty reached %d (hits=%d datacenter=%v exceeded=%v); the escalation belongs in MinAgeMs", + got.Difficulty, hits, dc, ex) + } + if got.MinAgeMs > maxMinAgeMs { + t.Fatalf("minAge reached %dms, above the %dms cap", got.MinAgeMs, maxMinAgeMs) + } + } + } + } +} + +// A datacenter address used to jump straight to difficulty 5, which charges a +// real person on a corporate VPN or iCloud Private Relay several seconds of +// blocked hashing for having a shared IP. It must now move only the time floor. +func TestDatacenterMovesTimeNotDifficulty(t *testing.T) { + cost := ComputeChallengeCost(0, true, 0, false) + if cost.Difficulty != baseDifficulty { + t.Errorf("a datacenter address must not raise difficulty, got %d", cost.Difficulty) + } + if cost.MinAgeMs <= baseMinAgeMs { + t.Errorf("a datacenter address should raise the time floor, got %dms", cost.MinAgeMs) + } +} + +func TestRateSignalsRaiseOnlyTheTimeFloor(t *testing.T) { + busy := ComputeChallengeCost(0, false, 50, false) + if busy.Difficulty != baseDifficulty { + t.Errorf("a high request count must not raise difficulty, got %d", busy.Difficulty) + } + if busy.MinAgeMs <= baseMinAgeMs { + t.Errorf("a high request count should raise the time floor, got %dms", busy.MinAgeMs) + } + + limited := ComputeChallengeCost(0, false, 50, true) + if limited.MinAgeMs <= busy.MinAgeMs { + t.Errorf("an exceeded rate limit should cost more than a merely busy one: %dms vs %dms", + limited.MinAgeMs, busy.MinAgeMs) + } + if limited.Difficulty != baseDifficulty { + t.Errorf("an exceeded rate limit must not raise difficulty, got %d", limited.Difficulty) + } +} + +// Marginal verdicts are exactly the case where the scoring might be wrong about +// a real person. Recording them would make the next visitor from that address +// wait for a guess. +func TestOnlyStrongVerdictsAreRecorded(t *testing.T) { + l := NewSuspicionLedger() + for _, score := range []float64{0.0, 0.3, 0.5, 0.7, 0.79} { + l.Record("site", "203.0.113.7", score) + } + if n := l.Count("site", "203.0.113.7"); n != 0 { + t.Errorf("scores below %.2f must not accumulate, counted %d", suspicionStrongScore, n) + } + + l.Record("site", "203.0.113.7", 0.8) + l.Record("site", "203.0.113.7", 0.95) + if n := l.Count("site", "203.0.113.7"); n != 2 { + t.Errorf("expected 2 strong verdicts, counted %d", n) + } +} + +// Suspicion is per site key as well as per address, so one site's abusers do +// not price another site's visitors. +func TestLedgerIsScopedPerSiteAndAddress(t *testing.T) { + l := NewSuspicionLedger() + for i := 0; i < 6; i++ { + l.Record("site-a", "203.0.113.7", 0.95) + } + + if n := l.Count("site-b", "203.0.113.7"); n != 0 { + t.Errorf("a different site key must not inherit suspicion, counted %d", n) + } + if n := l.Count("site-a", "203.0.113.8"); n != 0 { + t.Errorf("a different address must not inherit suspicion, counted %d", n) + } + if n := l.Count("site-a", "203.0.113.7"); n != 6 { + t.Errorf("expected 6 for the recorded source, counted %d", n) + } +} + +func TestLedgerIgnoresAnEmptyAddress(t *testing.T) { + l := NewSuspicionLedger() + l.Record("site", "", 0.99) + if n := l.Count("site", ""); n != 0 { + t.Errorf("an empty address must not accumulate, counted %d", n) + } +} + +// The hit slice is bounded, and the bound must not corrupt the count for the +// tiers that actually exist. +func TestLedgerBoundsRetainedHits(t *testing.T) { + l := NewSuspicionLedger() + for i := 0; i < suspicionMaxHits*3; i++ { + l.Record("site", "203.0.113.7", 0.99) + } + n := l.Count("site", "203.0.113.7") + if n != suspicionMaxHits { + t.Errorf("expected the count to saturate at %d, got %d", suspicionMaxHits, n) + } + if got := ComputeChallengeCost(n, false, 0, false); got.MinAgeMs != maxMinAgeMs { + t.Errorf("a saturated source should reach the top tier, got %dms", got.MinAgeMs) + } +} + +// A challenge whose minAgeMs could be lowered on the way back to the server +// would let a client price its own delay. +func TestChallengeSignatureCoversMinAge(t *testing.T) { + e := NewScoringEngine("test-secret") + + clean := e.GeneratePoWChallenge("site", "203.0.113.20", false) + if clean.MinAgeMs != baseMinAgeMs { + t.Fatalf("clean source got minAge %dms, want %d", clean.MinAgeMs, baseMinAgeMs) + } + + for i := 0; i < 6; i++ { + e.suspicion.Record("site", "203.0.113.21", 0.95) + } + suspicious := e.GeneratePoWChallenge("site", "203.0.113.21", false) + if suspicious.MinAgeMs <= clean.MinAgeMs { + t.Errorf("a suspicious source should be charged more time: %dms vs %dms", + suspicious.MinAgeMs, clean.MinAgeMs) + } + + // Signing the same challenge with the delay talked down must not reproduce + // the signature it was issued with. + sign := func(minAge int64) string { + payload, _ := json.Marshal(map[string]interface{}{ + "id": suspicious.ID, + "siteKey": suspicious.SiteKey, + "timestamp": suspicious.Timestamp, + "expiresAt": suspicious.ExpiresAt, + "difficulty": suspicious.Difficulty, + "prefix": suspicious.Prefix, + "minAgeMs": minAge, + }) + h := hmac.New(sha256.New, []byte("test-secret")) + h.Write(payload) + return hex.EncodeToString(h.Sum(nil)) + } + + if sign(suspicious.MinAgeMs) != suspicious.Sig { + t.Fatal("the test is not reproducing the server's signing input; fix the test before trusting the assertion below") + } + if sign(baseMinAgeMs) == suspicious.Sig { + t.Error("minAgeMs is not covered by the challenge signature — a client could talk its own delay down") + } +} + +// A challenge issued before adaptive cost existed carries no minAgeMs. It must +// fall back to the baseline rather than to zero, which would disable the timing +// gate entirely for anything still holding an old challenge. +func TestChallengeWithoutMinAgeFallsBackToBaseline(t *testing.T) { + e := NewScoringEngine("test-secret") + challenge := e.GeneratePoWChallenge("site", "203.0.113.30", false) + + e.powStore.mu.Lock() + e.powStore.challenges[challenge.ID].MinAgeMs = 0 + e.powStore.mu.Unlock() + + solution := solvePoW(t, challenge) + res := e.VerifyPoWSolution(solution, "site") + if !res.Valid { + t.Fatalf("solution should verify: %s", res.Reason) + } + if res.MinAgeMs != baseMinAgeMs { + t.Errorf("a challenge without minAge should fall back to %dms, got %dms", baseMinAgeMs, res.MinAgeMs) + } +} diff --git a/server-node/index.js b/server-node/index.js index 7f36003..7f5533d 100644 --- a/server-node/index.js +++ b/server-node/index.js @@ -12,6 +12,7 @@ const crypto = require('crypto'); const detection = require('./detection'); const { ProxyTrust } = require('./clientip'); +const { SuspicionLedger, computeChallengeCost, BASE_MIN_AGE_MS } = require('./suspicion'); // ============================================================================= // PoW Challenge Store (can be extended with Redis) @@ -25,7 +26,7 @@ class PoWChallengeStore { this.expirationMs = options.expirationMs || 5 * 60 * 1000; // 5 minutes } - generate(siteKey, ip, difficulty = 4) { + generate(siteKey, ip, difficulty = 4, minAgeMs = BASE_MIN_AGE_MS) { const challengeId = crypto.randomBytes(16).toString('hex'); const timestamp = Date.now(); const expiresAt = timestamp + this.expirationMs; @@ -36,14 +37,16 @@ class PoWChallengeStore { timestamp, expiresAt, difficulty, + // How long the client must hold this challenge before submitting a + // solution. Inside the signed payload so it cannot be talked down. + minAgeMs, prefix: `${challengeId}:${timestamp}:${difficulty}` }; // Sign the challenge const sig = crypto.createHmac('sha256', this.secret) .update(JSON.stringify(challengeData)) - .digest('hex') - .slice(0, 16); + .digest('hex'); challengeData.sig = sig; @@ -100,7 +103,13 @@ class PoWChallengeStore { this.usedSolutions.add(solutionKey); this.challenges.delete(challengeId); - return { valid: true, difficulty: challenge.difficulty }; + return { + valid: true, + difficulty: challenge.difficulty, + serverElapsed: Date.now() - challenge.timestamp, + // Fall back for challenges issued before adaptive cost existed. + minAgeMs: challenge.minAgeMs || BASE_MIN_AGE_MS + }; } _cleanup() { @@ -207,29 +216,31 @@ class ScoringEngine { this.powStore = options.powStore || new PoWChallengeStore({ secret: this.secret }); this.rateLimiter = options.rateLimiter || new RateLimiter(); this.fingerprintStore = options.fingerprintStore || new FingerprintStore(); + this.suspicion = options.suspicion || new SuspicionLedger(); this.weights = options.weights || WEIGHTS; } // Generate a PoW challenge generateChallenge(siteKey, ip, options = {}) { + // See suspicion.js for why the escalation lands almost entirely on + // minAgeMs rather than on difficulty. let difficulty = options.difficulty || 4; + let minAgeMs = BASE_MIN_AGE_MS; if (options.scaleByReputation !== false) { - if (detection.isDatacenterIP(ip)) { - difficulty = Math.max(difficulty, 5); - } - const rateKey = `pow:${siteKey}:${ip}`; const [exceeded, count] = this.rateLimiter.check(rateKey, 60, 20); - if (count > 10) { - difficulty = Math.min(6, difficulty + 1); - } - if (exceeded) { - difficulty = 6; - } + const cost = computeChallengeCost( + this.suspicion.count(siteKey, ip), + detection.isDatacenterIP(ip), + count, + exceeded + ); + difficulty = Math.max(difficulty, cost.difficulty); + minAgeMs = cost.minAgeMs; } - return this.powStore.generate(siteKey, ip, difficulty); + return this.powStore.generate(siteKey, ip, difficulty, minAgeMs); } // Verify signals and return score @@ -260,6 +271,26 @@ class ScoringEngine { confidence: 0.8, reason: `PoW verification failed: ${powResult.reason}` }); + } else if (powResult.serverElapsed < BASE_MIN_AGE_MS) { + // Under the universal baseline nothing legitimate can have happened — + // no human completes an interaction that fast. + detections.push({ + category: 'bot', + score: 0.8, + confidence: 0.85, + reason: `Challenge solved too fast (${powResult.serverElapsed}ms server-side)` + }); + } else if (powResult.serverElapsed < powResult.minAgeMs) { + // Between the baseline and this source's own elevated floor is weaker + // evidence: a client predating adaptive cost, or one served from a + // stale cache, does not know to wait. It contributes rather than + // deciding. + detections.push({ + category: 'bot', + score: 0.5, + confidence: 0.5, + reason: `Challenge submitted before the required delay for this source (${powResult.serverElapsed}ms of ${powResult.minAgeMs}ms)` + }); } } else { detections.push({ @@ -308,6 +339,10 @@ class ScoringEngine { const success = finalScore < 0.5; const token = success ? this._generateToken(ip, siteKey, finalScore) : null; + // Feed the ledger so the next challenge this source asks for is priced on + // what it just did. + this.suspicion.record(siteKey, ip, finalScore); + return { success, score: finalScore, @@ -332,7 +367,7 @@ class ScoringEngine { delete decoded.sig; const payload = JSON.stringify(decoded, Object.keys(decoded).sort()); - const expectedSig = crypto.createHmac('sha256', this.secret).update(payload).digest('hex').slice(0, 16); + const expectedSig = crypto.createHmac('sha256', this.secret).update(payload).digest('hex'); if (!crypto.timingSafeEqual(Buffer.from(sig), Buffer.from(expectedSig))) { return { valid: false, reason: 'invalid_signature' }; @@ -707,7 +742,7 @@ class ScoringEngine { }; const payload = JSON.stringify(data, Object.keys(data).sort()); - const sig = crypto.createHmac('sha256', this.secret).update(payload).digest('hex').slice(0, 16); + const sig = crypto.createHmac('sha256', this.secret).update(payload).digest('hex'); data.sig = sig; return Buffer.from(JSON.stringify(data)).toString('base64url'); diff --git a/server-node/package.json b/server-node/package.json index f98d5c2..def7a43 100644 --- a/server-node/package.json +++ b/server-node/package.json @@ -15,8 +15,9 @@ "test:clientip": "node clientip.test.js", "test:limits": "node limits.test.js", "test:detection": "node detection.test.js", - "test": "node clientip.test.js && node limits.test.js && node detection.test.js && node inputforensics.test.js && node webbotauth.test.js", - "test:forensics": "node inputforensics.test.js" + "test": "node clientip.test.js && node limits.test.js && node detection.test.js && node inputforensics.test.js && node webbotauth.test.js && node --test suspicion.test.js", + "test:forensics": "node inputforensics.test.js", + "test:suspicion": "node --test suspicion.test.js" }, "dependencies": { "cors": "^2.8.5", diff --git a/server-node/server.js b/server-node/server.js index d1c109c..9eac36b 100644 --- a/server-node/server.js +++ b/server-node/server.js @@ -12,6 +12,7 @@ const detection = require('./detection'); const webbotauth = require('./webbotauth'); const { ProxyTrust } = require('./clientip'); const { BoundedMap, BoundedSet, SiteKeyGuard } = require('./limits'); +const { SuspicionLedger, computeChallengeCost, BASE_MIN_AGE_MS } = require('./suspicion'); const { detectInputForensics } = require('./inputforensics'); const app = express(); @@ -32,6 +33,10 @@ const PROXY_TRUST = ProxyTrust.fromEnv(); // limits.js — the cap is unconditional; FCAPTCHA_SITE_KEYS adds an allowlist. const SITE_KEYS = SiteKeyGuard.fromEnv(); +// Recent strong verdicts per source, used to price the next challenge that +// source asks for. Bounded and short-lived; see suspicion.js. +const suspicionLedger = new SuspicionLedger(); + // Express's own `trust proxy` would re-derive req.ip from the same headers on // its own terms; IP resolution goes through PROXY_TRUST.clientIP exclusively. app.set('trust proxy', false); @@ -97,7 +102,7 @@ const powChallengeStore = { usedSolutions: new BoundedSet(), // Generate a new challenge - generate(siteKey, ip, difficulty = 4) { + generate(siteKey, ip, difficulty = 4, minAgeMs = BASE_MIN_AGE_MS) { const challengeId = crypto.randomBytes(16).toString('hex'); const nonce = crypto.randomBytes(16).toString('hex'); const timestamp = Date.now(); @@ -110,6 +115,9 @@ const powChallengeStore = { timestamp, expiresAt, difficulty, + // How long the client must hold this challenge before submitting a + // solution. Inside the signed payload so it cannot be talked down. + minAgeMs, nonce, prefix: `${challengeId}:${timestamp}:${difficulty}` }; @@ -180,7 +188,14 @@ const powChallengeStore = { // Calculate server-side elapsed time (un-spoofable) const serverElapsed = Date.now() - challenge.createdAt; - return { valid: true, difficulty: challenge.difficulty, serverElapsed, nonce: challenge.nonce }; + return { + valid: true, + difficulty: challenge.difficulty, + serverElapsed, + nonce: challenge.nonce, + // Fall back for challenges issued before adaptive cost existed. + minAgeMs: challenge.minAgeMs || BASE_MIN_AGE_MS + }; }, _cleanup() { @@ -1311,14 +1326,26 @@ function runVerification(signals, ip, siteKey, userAgent, headers = {}, ja3Hash } } - if (powValid && powVerification.serverElapsed < 1500) { - // Server-side timing: challenge was solved too fast (un-spoofable) + // Server-side timing, the one cost an attacker cannot buy their way out + // of. Two thresholds, because they mean different things. + if (powValid && powVerification.serverElapsed < BASE_MIN_AGE_MS) { + // Under the universal baseline nothing legitimate can have happened. detections.push({ category: 'bot', score: 0.8, confidence: 0.85, reason: `Challenge solved too fast (${powVerification.serverElapsed}ms server-side)` }); + } else if (powValid && powVerification.serverElapsed < (powVerification.minAgeMs || BASE_MIN_AGE_MS)) { + // Between the baseline and this source's own elevated floor is weaker + // evidence: a client predating adaptive cost, or one served from a stale + // cache, does not know to wait. It contributes rather than deciding. + detections.push({ + category: 'bot', + score: 0.5, + confidence: 0.5, + reason: `Challenge submitted before the required delay for this source (${powVerification.serverElapsed}ms of ${powVerification.minAgeMs}ms)` + }); } } else { // No PoW solution provided - hard fail @@ -1409,6 +1436,10 @@ function runVerification(signals, ip, siteKey, userAgent, headers = {}, ja3Hash const success = finalScore < 0.5; const token = success ? generateToken(ip, siteKey, finalScore) : null; + // Feed the ledger so the next challenge this source asks for is priced on + // what it just did. + suspicionLedger.record(siteKey, ip, finalScore); + return { success, score: finalScore, @@ -1506,25 +1537,18 @@ app.get('/api/pow/challenge', (req, res) => { const ip = PROXY_TRUST.clientIP(req); const siteKey = SITE_KEYS.normalize(req.query.siteKey, ip); - // Difficulty scaling based on IP reputation - let difficulty = 4; // Default: ~100-500ms on average hardware - - // Increase difficulty for suspicious IPs - if (detection.isDatacenterIP(ip)) { - difficulty = 5; // ~1-3 seconds - } - - // Check rate - high request rate gets harder challenges + // Cost scaling. See suspicion.js for why the escalation lands almost + // entirely on minAgeMs rather than on difficulty. const rateKey = `pow:${siteKey}:${ip}`; const [exceeded, count] = rateLimiter.check(rateKey, 60, 20); - if (count > 10) { - difficulty = Math.min(6, difficulty + 1); // Up to difficulty 6 - } - if (exceeded) { - difficulty = 6; // Maximum difficulty for rate-limited IPs - } + const cost = computeChallengeCost( + suspicionLedger.count(siteKey, ip), + detection.isDatacenterIP(ip), + count, + exceeded + ); - const challenge = powChallengeStore.generate(siteKey, ip, difficulty); + const challenge = powChallengeStore.generate(siteKey, ip, cost.difficulty, cost.minAgeMs); res.json({ challengeId: challenge.id, @@ -1532,7 +1556,11 @@ app.get('/api/pow/challenge', (req, res) => { difficulty: challenge.difficulty, expiresAt: challenge.expiresAt, nonce: challenge.nonce, - sig: challenge.sig + sig: challenge.sig, + // Tells the client how long to hold a solved challenge before submitting. + // Honouring it is how an ordinary visitor pays an elevated cost as a short + // wait instead of as a worse score. + minAgeMs: challenge.minAgeMs }); }); diff --git a/server-node/suspicion.js b/server-node/suspicion.js new file mode 100644 index 0000000..53874f5 --- /dev/null +++ b/server-node/suspicion.js @@ -0,0 +1,172 @@ +'use strict'; + +/** + * Adaptive challenge cost: what a source pays is a function of what that source + * has recently been caught doing, rather than a constant. + * + * Why the cost is mostly time, not hashing + * ---------------------------------------- + * A constant difficulty is strictly dominated: it either fails to inconvenience + * an attacker or it does hurt real users. The measurements behind that claim: + * browser JS runs 1-3M hash/s, native code 100-500M/s, so difficulty 6 costs a + * native solver about a millisecond and a budget Android phone about sixteen + * seconds. Raising difficulty is close to a pure tax on the slowest legitimate + * devices. + * + * Wall-clock is the knob that does not have that property. Nobody can make less + * time pass, so a minimum challenge age caps how fast one source can mint + * tokens no matter what hardware it brings. So suspicion moves the time floor + * first and difficulty barely at all. + * + * What this deliberately is not + * ----------------------------- + * This is not Workstream F 10.1 (cross-session correlation). It stores strong- + * verdict timestamps per source, nothing else: no behavioral vectors, no + * per-fingerprint history, no traces that survive the window. It is the same + * shape and the same privacy class as the rate limiter sitting next to it, and + * it should stay that way — 10.1 has a privacy load this does not, and the two + * should not be conflated because they happen to both be "server-side memory". + */ + +// The verdict score at or above which a verification counts as evidence. +// Deliberately high: a marginal verdict is exactly the case where the scoring +// might be wrong about a real person, and making the next person from that +// address wait is not worth the guess. +const STRONG_SCORE = 0.8; + +// How long a strong verdict keeps counting. Short enough that a shared egress +// address recovers on its own within a coffee break. +const WINDOW_MS = 15 * 60 * 1000; + +// Only the count matters and every tier saturates well below this. +const MAX_HITS = 16; + +// Bounds the table. Sources with no strong verdicts never get an entry at all, +// so this only has to cover addresses actively failing verification. +const MAX_SOURCES = 50000; + +/** + * Records recent strong verdicts per source. + * + * Entries are created only when a source produces a strong verdict, so the + * common case — a legitimate visitor — allocates nothing and looks up nothing + * but a miss. + */ +class SuspicionLedger { + constructor() { + this.hits = new Map(); + } + + static _key(siteKey, ip) { + return `${siteKey}|${ip}`; + } + + /** + * Note a verdict. Scores below the strong threshold are ignored entirely + * rather than recorded and weighted, so a source that merely looks unusual + * never accumulates anything. + */ + record(siteKey, ip, score) { + if (!ip || typeof score !== 'number' || score < STRONG_SCORE) return; + + const key = SuspicionLedger._key(siteKey, ip); + const now = Date.now(); + const cutoff = now - WINDOW_MS; + + const kept = (this.hits.get(key) || []).filter((t) => t > cutoff); + kept.push(now); + + // Map preserves insertion order, so re-inserting makes this the most + // recently touched key and the eviction below drops the stalest source. + this.hits.delete(key); + this.hits.set(key, kept.length > MAX_HITS ? kept.slice(-MAX_HITS) : kept); + + if (this.hits.size > MAX_SOURCES) { + this.hits.delete(this.hits.keys().next().value); + } + } + + /** + * How many strong verdicts this source produced inside the window. Counted + * from the timestamps rather than from the entry's existence, so an old hit + * actually decays while newer ones keep the entry alive. + */ + count(siteKey, ip) { + if (!ip) return 0; + const hits = this.hits.get(SuspicionLedger._key(siteKey, ip)); + if (!hits) return 0; + + const cutoff = Date.now() - WINDOW_MS; + const n = hits.filter((t) => t > cutoff).length; + if (n === 0) this.hits.delete(SuspicionLedger._key(siteKey, ip)); + return n; + } +} + +// Baseline cost. A clean visitor pays exactly this, which is what the server +// has always charged everyone. +const BASE_DIFFICULTY = 4; +const BASE_MIN_AGE_MS = 1500; + +// Caps the compute knob at 5, below the 6 this server used to reach. Difficulty +// 6 buys about a millisecond of attacker time and spends about sixteen seconds +// of a budget phone's; the escalation belongs in minAgeMs where an attacker +// cannot buy their way out of it. +const MAX_DIFFICULTY = 5; + +// Caps the time knob at 15s. At the 1.5s baseline one address can mint roughly +// 40 tokens a minute; at 15s, four. Pushing further buys little and is felt by +// anyone sharing a poisoned egress address. +const MAX_MIN_AGE_MS = 15000; + +/** + * Maps accumulated suspicion onto a cost. + * + * Note what does NOT raise difficulty here: being on a datacenter address. That + * used to jump straight to difficulty 5, which charges a real person on a + * corporate VPN or iCloud Private Relay several seconds of blocked hashing on a + * slow phone for the offence of having a shared IP. It now moves the time floor + * instead, which a datacenter-hosted scraper feels as reduced throughput and a + * person filling in a form does not feel at all. + * + * @returns {{difficulty: number, minAgeMs: number}} + */ +function computeChallengeCost(strongHits, isDatacenter, requestCount, rateExceeded) { + let difficulty = BASE_DIFFICULTY; + let minAgeMs = BASE_MIN_AGE_MS; + + if (strongHits >= 6) { + difficulty = 5; + minAgeMs = 15000; + } else if (strongHits >= 3) { + difficulty = 5; + minAgeMs = 8000; + } else if (strongHits >= 1) { + minAgeMs = 4000; + } + + // Floors from signals that are suggestive rather than damning. They raise the + // time floor and never the difficulty. + const raiseTo = (ms) => { + if (minAgeMs < ms) minAgeMs = ms; + }; + if (isDatacenter) raiseTo(3000); + if (requestCount > 10) raiseTo(6000); + if (rateExceeded) raiseTo(10000); + + return { + difficulty: Math.min(difficulty, MAX_DIFFICULTY), + minAgeMs: Math.min(minAgeMs, MAX_MIN_AGE_MS) + }; +} + +module.exports = { + SuspicionLedger, + computeChallengeCost, + STRONG_SCORE, + WINDOW_MS, + BASE_DIFFICULTY, + BASE_MIN_AGE_MS, + MAX_DIFFICULTY, + MAX_MIN_AGE_MS +}; diff --git a/server-node/suspicion.test.js b/server-node/suspicion.test.js new file mode 100644 index 0000000..31a34b1 --- /dev/null +++ b/server-node/suspicion.test.js @@ -0,0 +1,175 @@ +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert'); + +const { + SuspicionLedger, + computeChallengeCost, + BASE_DIFFICULTY, + BASE_MIN_AGE_MS, + MAX_MIN_AGE_MS +} = require('./suspicion'); + +// --- what a clean visitor pays ---------------------------------------------- + +// The property that matters most: a visitor who has done nothing wrong pays +// exactly what everyone paid before adaptive cost existed. If this drifts, the +// feature has started taxing the people it was designed not to touch. +test('a clean source pays the baseline', () => { + const cost = computeChallengeCost(0, false, 0, false); + assert.strictEqual(cost.difficulty, BASE_DIFFICULTY); + assert.strictEqual(cost.minAgeMs, BASE_MIN_AGE_MS); +}); + +test('cost escalates with strong verdicts', () => { + const cases = [ + [0, 4, 1500], + [1, 4, 4000], + [2, 4, 4000], + [3, 5, 8000], + [5, 5, 8000], + [6, 5, 15000], + [50, 5, 15000] + ]; + for (const [hits, difficulty, minAgeMs] of cases) { + assert.deepStrictEqual( + computeChallengeCost(hits, false, 0, false), + { difficulty, minAgeMs }, + `${hits} strong verdicts` + ); + } +}); + +// The escalation must stay on the knob an attacker cannot buy their way out of. +// Difficulty 6 costs a native solver about a millisecond and a budget phone +// about sixteen seconds, so reaching it would be a tax on slow devices and +// nothing else. +test('difficulty never exceeds 5 and minAge never exceeds its cap', () => { + for (let hits = 0; hits < 100; hits++) { + for (const dc of [false, true]) { + for (const ex of [false, true]) { + const cost = computeChallengeCost(hits, dc, 1000, ex); + assert.ok(cost.difficulty <= 5, `difficulty reached ${cost.difficulty}`); + assert.ok(cost.minAgeMs <= MAX_MIN_AGE_MS, `minAge reached ${cost.minAgeMs}`); + } + } + } +}); + +// A datacenter address used to jump straight to difficulty 5, which charges a +// real person on a corporate VPN or iCloud Private Relay several seconds of +// blocked hashing for having a shared IP. +test('a datacenter address moves time, not difficulty', () => { + const cost = computeChallengeCost(0, true, 0, false); + assert.strictEqual(cost.difficulty, BASE_DIFFICULTY); + assert.ok(cost.minAgeMs > BASE_MIN_AGE_MS); +}); + +test('rate signals raise only the time floor', () => { + const busy = computeChallengeCost(0, false, 50, false); + assert.strictEqual(busy.difficulty, BASE_DIFFICULTY); + assert.ok(busy.minAgeMs > BASE_MIN_AGE_MS); + + const limited = computeChallengeCost(0, false, 50, true); + assert.strictEqual(limited.difficulty, BASE_DIFFICULTY); + assert.ok(limited.minAgeMs > busy.minAgeMs); +}); + +// --- the ledger -------------------------------------------------------------- + +// Marginal verdicts are exactly the case where the scoring might be wrong about +// a real person. Recording them would make the next visitor from that address +// wait for a guess. +test('only strong verdicts are recorded', () => { + const l = new SuspicionLedger(); + for (const score of [0, 0.3, 0.5, 0.7, 0.79]) l.record('site', '203.0.113.7', score); + assert.strictEqual(l.count('site', '203.0.113.7'), 0); + + l.record('site', '203.0.113.7', 0.8); + l.record('site', '203.0.113.7', 0.95); + assert.strictEqual(l.count('site', '203.0.113.7'), 2); +}); + +// Suspicion is per site key as well as per address, so one site's abusers do +// not price another site's visitors. +test('the ledger is scoped per site and address', () => { + const l = new SuspicionLedger(); + for (let i = 0; i < 6; i++) l.record('site-a', '203.0.113.7', 0.95); + + assert.strictEqual(l.count('site-b', '203.0.113.7'), 0); + assert.strictEqual(l.count('site-a', '203.0.113.8'), 0); + assert.strictEqual(l.count('site-a', '203.0.113.7'), 6); +}); + +test('an empty address never accumulates', () => { + const l = new SuspicionLedger(); + l.record('site', '', 0.99); + assert.strictEqual(l.count('site', ''), 0); +}); + +test('a non-numeric score is ignored rather than coerced', () => { + const l = new SuspicionLedger(); + l.record('site', '203.0.113.7', undefined); + l.record('site', '203.0.113.7', null); + l.record('site', '203.0.113.7', '0.99'); + assert.strictEqual(l.count('site', '203.0.113.7'), 0); +}); + +test('retained hits are bounded and still reach the top tier', () => { + const l = new SuspicionLedger(); + for (let i = 0; i < 48; i++) l.record('site', '203.0.113.7', 0.99); + const n = l.count('site', '203.0.113.7'); + assert.strictEqual(n, 16); + assert.strictEqual(computeChallengeCost(n, false, 0, false).minAgeMs, MAX_MIN_AGE_MS); +}); + +// --- integration with the scoring engine ------------------------------------ + +test('the engine prices a challenge from the ledger, and the sig covers minAgeMs', () => { + const fcaptcha = require('./index'); + const engine = fcaptcha.createScoringEngine({ secret: 'test-secret' }); + + const clean = engine.generateChallenge('site', '203.0.113.20'); + assert.strictEqual(clean.difficulty, BASE_DIFFICULTY); + assert.strictEqual(clean.minAgeMs, BASE_MIN_AGE_MS); + + for (let i = 0; i < 6; i++) engine.suspicion.record('site', '203.0.113.21', 0.95); + const suspicious = engine.generateChallenge('site', '203.0.113.21'); + assert.ok(suspicious.minAgeMs > clean.minAgeMs); + assert.ok(suspicious.difficulty <= 5); + + // Signing the same challenge with the delay talked down must not reproduce + // the signature it was issued with. + const crypto = require('crypto'); + const sign = (minAgeMs) => { + const { sig, ...rest } = suspicious; + return crypto + .createHmac('sha256', 'test-secret') + .update(JSON.stringify({ ...rest, minAgeMs })) + .digest('hex'); + }; + assert.strictEqual( + sign(suspicious.minAgeMs), + suspicious.sig, + 'the test is not reproducing the server signing input; fix it before trusting the assertion below' + ); + assert.notStrictEqual( + sign(BASE_MIN_AGE_MS), + suspicious.sig, + 'minAgeMs is not covered by the challenge signature — a client could talk its own delay down' + ); +}); + +// The library was missed by the pass that removed HMAC truncation elsewhere, so +// its tokens carried a 64-bit signature while Go, Python and the standalone +// server all used the full digest. +test('token signatures are full-length', () => { + const fcaptcha = require('./index'); + const engine = fcaptcha.createScoringEngine({ secret: 'test-secret' }); + + const token = engine._generateToken('203.0.113.40', 'site', 0.1); + const decoded = JSON.parse(Buffer.from(token, 'base64url').toString()); + assert.strictEqual(decoded.sig.length, 64, 'expected a full SHA-256 HMAC, not a truncation'); + assert.strictEqual(engine.verifyToken(token).valid, true); +}); diff --git a/server-python/server.py b/server-python/server.py index 015e929..af38b2d 100644 --- a/server-python/server.py +++ b/server-python/server.py @@ -24,6 +24,11 @@ from clientip import ProxyTrust from sitekeys import SiteKeyGuard from inputforensics import detect_input_forensics +from suspicion import ( + SuspicionLedger, + compute_challenge_cost, + BASE_MIN_AGE_MS, +) # Keep in sync with server-node/package.json and client/fcaptcha.js on release. app = FastAPI(title="FCaptcha", version="1.20.0") @@ -239,16 +244,13 @@ def generate(self, site_key: str, ip: str, is_datacenter: bool = False) -> Dict: now = int(time.time() * 1000) expires_at = now + (5 * 60 * 1000) # 5 minutes - # Difficulty scaling - difficulty = 4 # Default: ~100-500ms on average hardware - if is_datacenter: - difficulty = 5 # Harder for datacenter IPs - - # Check rate for this IP + # Cost scaling. See suspicion.py for why the escalation lands almost + # entirely on min_age_ms rather than on difficulty. rate_key = f"pow:{site_key}:{ip}" - _, count = rate_limiter.check(rate_key, 60, 20) - if count > 10: - difficulty = min(6, difficulty + 1) + exceeded, count = rate_limiter.check(rate_key, 60, 20) + difficulty, min_age_ms = compute_challenge_cost( + suspicion_ledger.count(site_key, ip), is_datacenter, count, exceeded + ) prefix = f"{challenge_id}:{now}:{difficulty}" @@ -260,6 +262,9 @@ def generate(self, site_key: str, ip: str, is_datacenter: bool = False) -> Dict: "timestamp": now, "expiresAt": expires_at, "nonce": nonce, + # How long the client must hold this challenge before submitting a + # solution. Inside the signed payload so it cannot be talked down. + "minAgeMs": min_age_ms, "ip": ip } @@ -270,7 +275,8 @@ def generate(self, site_key: str, ip: str, is_datacenter: bool = False) -> Dict: "timestamp": now, "expiresAt": expires_at, "difficulty": difficulty, - "prefix": prefix + "prefix": prefix, + "minAgeMs": min_age_ms }, sort_keys=True) sig = hmac.new(SECRET_KEY.encode(), sig_data.encode(), hashlib.sha256).hexdigest() challenge["sig"] = sig @@ -288,7 +294,11 @@ def generate(self, site_key: str, ip: str, is_datacenter: bool = False) -> Dict: "difficulty": difficulty, "expiresAt": expires_at, "nonce": nonce, - "sig": sig + "sig": sig, + # Tells the client how long to hold a solved challenge before + # submitting. Honouring it is how an ordinary visitor pays an + # elevated cost as a short wait instead of as a worse score. + "minAgeMs": min_age_ms } def verify(self, solution: PoWSolution, site_key: str, signals_hash: str = None) -> Dict: @@ -336,7 +346,14 @@ def verify(self, solution: PoWSolution, site_key: str, signals_hash: str = None) # Delete challenge (one-time use) del self.challenges[solution.challengeId] - return {"valid": True, "difficulty": challenge["difficulty"], "serverElapsed": server_elapsed, "nonce": challenge.get("nonce")} + return { + "valid": True, + "difficulty": challenge["difficulty"], + "serverElapsed": server_elapsed, + "nonce": challenge.get("nonce"), + # Fall back for challenges issued before adaptive cost existed. + "minAgeMs": challenge.get("minAgeMs") or BASE_MIN_AGE_MS, + } def _cleanup(self): now = int(time.time() * 1000) @@ -371,6 +388,10 @@ def mark_used(self, sig: str) -> bool: rate_limiter = RateLimiter() + +# Recent strong verdicts per source, used to price the next challenge that +# source asks for. Bounded and short-lived; see suspicion.py. +suspicion_ledger = SuspicionLedger() fingerprint_store = FingerprintStore() pow_store = PoWChallengeStore() token_store = TokenStore() @@ -1357,12 +1378,28 @@ def run_verification( "Challenge nonce mismatch (signals not bound to challenge)" )) - if pow_result["valid"] and pow_result.get("serverElapsed", 99999) < 1500: - # Server-side timing: challenge was solved too fast (un-spoofable) - detections.append(Detection( - ThreatCategory.BOT, 0.8, 0.85, - f"Challenge solved too fast ({pow_result['serverElapsed']}ms server-side)" - )) + # Server-side timing, the one cost an attacker cannot buy their way out + # of. Two thresholds, because they mean different things. + if pow_result["valid"]: + elapsed = pow_result.get("serverElapsed", 99999) + min_age = pow_result.get("minAgeMs") or BASE_MIN_AGE_MS + if elapsed < BASE_MIN_AGE_MS: + # Under the universal baseline nothing legitimate can have + # happened - no human completes an interaction that fast. + detections.append(Detection( + ThreatCategory.BOT, 0.8, 0.85, + f"Challenge solved too fast ({elapsed}ms server-side)" + )) + elif elapsed < min_age: + # Between the baseline and this source's own elevated floor is + # weaker evidence: a client predating adaptive cost, or one + # served from a stale cache, does not know to wait. It + # contributes rather than deciding. + detections.append(Detection( + ThreatCategory.BOT, 0.5, 0.5, + f"Challenge submitted before the required delay for this " + f"source ({elapsed}ms of {min_age}ms)" + )) else: # No PoW solution provided - hard fail detections.append(Detection( @@ -1458,6 +1495,10 @@ def run_verification( success = final_score < 0.5 token = generate_token(ip, site_key, final_score) if success else None + # Feed the ledger so the next challenge this source asks for is priced on + # what it just did. + suspicion_ledger.record(site_key, ip, final_score) + return { "success": success, "score": final_score, diff --git a/server-python/suspicion.py b/server-python/suspicion.py new file mode 100644 index 0000000..92b714d --- /dev/null +++ b/server-python/suspicion.py @@ -0,0 +1,164 @@ +"""Adaptive challenge cost: what a source pays is a function of what that source +has recently been caught doing, rather than a constant. + +Why the cost is mostly time, not hashing +---------------------------------------- +A constant difficulty is strictly dominated: it either fails to inconvenience an +attacker or it does hurt real users. The measurements behind that claim: browser +JS runs 1-3M hash/s, native code 100-500M/s, so difficulty 6 costs a native +solver about a millisecond and a budget Android phone about sixteen seconds. +Raising difficulty is close to a pure tax on the slowest legitimate devices. + +Wall-clock is the knob that does not have that property. Nobody can make less +time pass, so a minimum challenge age caps how fast one source can mint tokens +no matter what hardware it brings. So suspicion moves the time floor first and +difficulty barely at all. + +What this deliberately is not +----------------------------- +This is not Workstream F 10.1 (cross-session correlation). It stores strong- +verdict timestamps per source, nothing else: no behavioral vectors, no +per-fingerprint history, no traces that survive the window. It is the same shape +and the same privacy class as the rate limiter sitting next to it, and it should +stay that way - 10.1 has a privacy load this does not, and the two should not be +conflated because they happen to both be "server-side memory". +""" + +import threading +import time +from collections import OrderedDict +from typing import Dict, List, Tuple + +# The verdict score at or above which a verification counts as evidence. +# Deliberately high: a marginal verdict is exactly the case where the scoring +# might be wrong about a real person, and making the next person from that +# address wait is not worth the guess. +STRONG_SCORE = 0.8 + +# How long a strong verdict keeps counting. Short enough that a shared egress +# address recovers on its own within a coffee break. +WINDOW_MS = 15 * 60 * 1000 + +# Only the count matters and every tier saturates well below this. +MAX_HITS = 16 + +# Bounds the table. Sources with no strong verdicts never get an entry at all, +# so this only has to cover addresses actively failing verification. +MAX_SOURCES = 50_000 + + +class SuspicionLedger: + """Records recent strong verdicts per source. + + Entries are created only when a source produces a strong verdict, so the + common case - a legitimate visitor - allocates nothing and looks up nothing + but a miss. + """ + + def __init__(self) -> None: + self._hits: "OrderedDict[str, List[int]]" = OrderedDict() + self._lock = threading.Lock() + + @staticmethod + def _key(site_key: str, ip: str) -> str: + return f"{site_key}|{ip}" + + def record(self, site_key: str, ip: str, score: float) -> None: + """Note a verdict. + + Scores below the strong threshold are ignored entirely rather than + recorded and weighted, so a source that merely looks unusual never + accumulates anything. + """ + if not ip or score is None or score < STRONG_SCORE: + return + + key = self._key(site_key, ip) + now = int(time.time() * 1000) + cutoff = now - WINDOW_MS + + with self._lock: + kept = [t for t in self._hits.get(key, []) if t > cutoff] + kept.append(now) + self._hits[key] = kept[-MAX_HITS:] + # move_to_end makes this the most recently touched key, so the + # eviction below drops the stalest source rather than an active one. + self._hits.move_to_end(key) + + while len(self._hits) > MAX_SOURCES: + self._hits.popitem(last=False) + + def count(self, site_key: str, ip: str) -> int: + """How many strong verdicts this source produced inside the window. + + Counted from the timestamps rather than from the entry's existence, so + an old hit actually decays while newer ones keep the entry alive. + """ + if not ip: + return 0 + + key = self._key(site_key, ip) + cutoff = int(time.time() * 1000) - WINDOW_MS + + with self._lock: + hits = self._hits.get(key) + if not hits: + return 0 + n = sum(1 for t in hits if t > cutoff) + if n == 0: + self._hits.pop(key, None) + return n + + +# Baseline cost. A clean visitor pays exactly this, which is what the server has +# always charged everyone. +BASE_DIFFICULTY = 4 +BASE_MIN_AGE_MS = 1500 + +# Caps the compute knob at 5, below the 6 this server used to reach. Difficulty +# 6 buys about a millisecond of attacker time and spends about sixteen seconds +# of a budget phone's; the escalation belongs in min_age_ms where an attacker +# cannot buy their way out of it. +MAX_DIFFICULTY = 5 + +# Caps the time knob at 15s. At the 1.5s baseline one address can mint roughly +# 40 tokens a minute; at 15s, four. Pushing further buys little and is felt by +# anyone sharing a poisoned egress address. +MAX_MIN_AGE_MS = 15_000 + + +def compute_challenge_cost( + strong_hits: int, + is_datacenter: bool = False, + request_count: int = 0, + rate_exceeded: bool = False, +) -> Tuple[int, int]: + """Map accumulated suspicion onto a cost, as ``(difficulty, min_age_ms)``. + + Note what does NOT raise difficulty here: being on a datacenter address. + That used to jump straight to difficulty 5, which charges a real person on a + corporate VPN or iCloud Private Relay several seconds of blocked hashing on + a slow phone for the offence of having a shared IP. It now moves the time + floor instead, which a datacenter-hosted scraper feels as reduced throughput + and a person filling in a form does not feel at all. + """ + difficulty = BASE_DIFFICULTY + min_age_ms = BASE_MIN_AGE_MS + + if strong_hits >= 6: + difficulty, min_age_ms = 5, 15_000 + elif strong_hits >= 3: + difficulty, min_age_ms = 5, 8_000 + elif strong_hits >= 1: + min_age_ms = 4_000 + + # Floors from signals that are suggestive rather than damning. They raise + # the time floor and never the difficulty. + if is_datacenter: + min_age_ms = max(min_age_ms, 3_000) + if request_count > 10: + min_age_ms = max(min_age_ms, 6_000) + if rate_exceeded: + min_age_ms = max(min_age_ms, 10_000) + + return min(difficulty, MAX_DIFFICULTY), min(min_age_ms, MAX_MIN_AGE_MS) diff --git a/server-python/test_suspicion.py b/server-python/test_suspicion.py new file mode 100644 index 0000000..9fa9991 --- /dev/null +++ b/server-python/test_suspicion.py @@ -0,0 +1,118 @@ +"""Tests for adaptive challenge cost.""" + +import unittest + +from suspicion import ( + SuspicionLedger, + compute_challenge_cost, + BASE_DIFFICULTY, + BASE_MIN_AGE_MS, + MAX_MIN_AGE_MS, + MAX_HITS, +) + + +class TestChallengeCost(unittest.TestCase): + def test_clean_source_pays_the_baseline(self): + """The property that matters most: a visitor who has done nothing wrong + pays exactly what everyone paid before adaptive cost existed. If this + drifts, the feature has started taxing the people it was designed not + to touch.""" + self.assertEqual( + compute_challenge_cost(0), (BASE_DIFFICULTY, BASE_MIN_AGE_MS) + ) + + def test_cost_escalates_with_strong_verdicts(self): + for hits, difficulty, min_age in [ + (0, 4, 1500), + (1, 4, 4000), + (2, 4, 4000), + (3, 5, 8000), + (5, 5, 8000), + (6, 5, 15000), + (50, 5, 15000), + ]: + with self.subTest(hits=hits): + self.assertEqual( + compute_challenge_cost(hits), (difficulty, min_age) + ) + + def test_difficulty_never_exceeds_five(self): + """The escalation must stay on the knob an attacker cannot buy their + way out of. Difficulty 6 costs a native solver about a millisecond and + a budget phone about sixteen seconds, so reaching it would be a tax on + slow devices and nothing else.""" + for hits in range(100): + for dc in (False, True): + for ex in (False, True): + difficulty, min_age = compute_challenge_cost(hits, dc, 1000, ex) + self.assertLessEqual(difficulty, 5) + self.assertLessEqual(min_age, MAX_MIN_AGE_MS) + + def test_datacenter_moves_time_not_difficulty(self): + """A datacenter address used to jump straight to difficulty 5, which + charges a real person on a corporate VPN or iCloud Private Relay + several seconds of blocked hashing for having a shared IP.""" + difficulty, min_age = compute_challenge_cost(0, is_datacenter=True) + self.assertEqual(difficulty, BASE_DIFFICULTY) + self.assertGreater(min_age, BASE_MIN_AGE_MS) + + def test_rate_signals_raise_only_the_time_floor(self): + busy_difficulty, busy_age = compute_challenge_cost(0, request_count=50) + self.assertEqual(busy_difficulty, BASE_DIFFICULTY) + self.assertGreater(busy_age, BASE_MIN_AGE_MS) + + limited_difficulty, limited_age = compute_challenge_cost( + 0, request_count=50, rate_exceeded=True + ) + self.assertEqual(limited_difficulty, BASE_DIFFICULTY) + self.assertGreater(limited_age, busy_age) + + +class TestSuspicionLedger(unittest.TestCase): + def test_only_strong_verdicts_are_recorded(self): + """Marginal verdicts are exactly the case where the scoring might be + wrong about a real person. Recording them would make the next visitor + from that address wait for a guess.""" + ledger = SuspicionLedger() + for score in (0.0, 0.3, 0.5, 0.7, 0.79): + ledger.record("site", "203.0.113.7", score) + self.assertEqual(ledger.count("site", "203.0.113.7"), 0) + + ledger.record("site", "203.0.113.7", 0.8) + ledger.record("site", "203.0.113.7", 0.95) + self.assertEqual(ledger.count("site", "203.0.113.7"), 2) + + def test_scoped_per_site_and_address(self): + """Suspicion is per site key as well as per address, so one site's + abusers do not price another site's visitors.""" + ledger = SuspicionLedger() + for _ in range(6): + ledger.record("site-a", "203.0.113.7", 0.95) + + self.assertEqual(ledger.count("site-b", "203.0.113.7"), 0) + self.assertEqual(ledger.count("site-a", "203.0.113.8"), 0) + self.assertEqual(ledger.count("site-a", "203.0.113.7"), 6) + + def test_empty_address_never_accumulates(self): + ledger = SuspicionLedger() + ledger.record("site", "", 0.99) + self.assertEqual(ledger.count("site", ""), 0) + + def test_none_score_is_ignored(self): + ledger = SuspicionLedger() + ledger.record("site", "203.0.113.7", None) + self.assertEqual(ledger.count("site", "203.0.113.7"), 0) + + def test_retained_hits_are_bounded(self): + ledger = SuspicionLedger() + for _ in range(MAX_HITS * 3): + ledger.record("site", "203.0.113.7", 0.99) + + n = ledger.count("site", "203.0.113.7") + self.assertEqual(n, MAX_HITS) + self.assertEqual(compute_challenge_cost(n)[1], MAX_MIN_AGE_MS) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test-detection.js b/test/test-detection.js index c60dd63..d837baa 100755 --- a/test/test-detection.js +++ b/test/test-detection.js @@ -1991,6 +1991,131 @@ async function testMobileSensorDetection() { } } +// Adaptive challenge cost. The escalation lives on the wall-clock floor rather +// than on difficulty, because a native solver clears difficulty 6 in about a +// millisecond while a budget phone spends about sixteen seconds on it — so +// raising difficulty taxes slow devices and constrains nobody. +async function testAdaptiveChallengeCost() { + log('\n[Adaptive Challenge Cost]', colors.cyan); + + const { createHash } = await import('crypto'); + const solve = (prefix, difficulty, signalsHash) => { + const target = '0'.repeat(difficulty); + for (let nonce = 0; nonce < 10000000; nonce++) { + const hash = createHash('sha256').update(`${prefix}:${signalsHash}:${nonce}`).digest('hex'); + if (hash.startsWith(target)) return { nonce, hash }; + } + return null; + }; + + const challengeFor = async (ip) => { + const res = await fetch(`${SERVER_URL}/api/pow/challenge?siteKey=adaptive`, { + headers: { 'X-Forwarded-For': ip }, + }); + return res.json(); + }; + + // A visitor who has done nothing wrong pays exactly what everyone paid before + // adaptive cost existed. + const clean = await challengeFor('192.0.2.150'); + if (clean.difficulty === 4 && clean.minAgeMs === 1500) { + passed++; + log(' ✓ A clean source pays the baseline (difficulty 4, 1500ms)', colors.green); + } else { + failed++; + log(` ✗ A clean source should pay the baseline, got difficulty ${clean.difficulty} / ${clean.minAgeMs}ms`, colors.red); + } + + // Drive one address to a strong verdict repeatedly, then confirm its next + // challenge costs more time — and no more hashing than the cap allows. + const abuser = '192.0.2.151'; + // navigator.webdriver === true is dispositive, so this lands at or above the + // 0.8 the ledger requires. Anything weaker is deliberately not recorded. + const botSignals = { + behavioral: { totalPoints: 0, trajectoryLength: 0, keyEvents: 0, touchEvents: 0 }, + environmental: { webdriver: true, automationFlags: { plugins: 0 } }, + }; + for (let i = 0; i < 6; i++) { + await fetch(`${SERVER_URL}/api/verify`, { + method: 'POST', + headers: { 'Content-Type': 'application/json', 'X-Forwarded-For': abuser }, + body: JSON.stringify({ siteKey: 'adaptive', signals: botSignals }), + }); + } + + const escalated = await challengeFor(abuser); + if (escalated.minAgeMs > clean.minAgeMs) { + passed++; + log(` ✓ A source with strong verdicts is charged more time (${escalated.minAgeMs}ms vs ${clean.minAgeMs}ms)`, colors.green); + } else { + failed++; + log(` ✗ A source with strong verdicts should be charged more time, got ${escalated.minAgeMs}ms`, colors.red); + } + + if (escalated.difficulty <= 5) { + passed++; + log(` ✓ Difficulty stays at or below 5 under escalation (got ${escalated.difficulty})`, colors.green); + } else { + failed++; + log(` ✗ Difficulty reached ${escalated.difficulty}; the escalation belongs on the time floor`, colors.red); + } + + // Submitting before that source's floor is scored — but as contributory + // evidence, not as a verdict, because an older cached client does not know to + // wait and would otherwise be punished for the delay it was never told about. + const signals = { + behavioral: { + totalPoints: 80, trajectoryLength: 350, interactionDuration: 1500, + velocityVariance: 0.8, microTremorScore: 0.6, directionChanges: 15, + mouseEventRate: 60, approachPoints: 12, + }, + environmental: { automationFlags: { chrome: true, platform: 'MacIntel', plugins: 5 } }, + meta: { challengeNonce: escalated.nonce }, + }; + const signalsJson = JSON.stringify(signals); + const signalsHash = createHash('sha256').update(signalsJson).digest('hex'); + const solution = solve(escalated.prefix, escalated.difficulty, signalsHash); + + // Past the universal baseline, short of this source's own floor. + await new Promise((r) => setTimeout(r, 1700)); + + const res = await fetch(`${SERVER_URL}/api/verify`, { + method: 'POST', + headers: { 'Content-Type': 'application/json', 'X-Forwarded-For': abuser }, + body: JSON.stringify({ + siteKey: 'adaptive', + signals, + signalsJson, + powSolution: { + challengeId: escalated.challengeId, + nonce: solution.nonce, + hash: solution.hash, + signalsHash, + }, + }), + }); + const verdict = await res.json(); + const reasons = (verdict.detections || []).map((d) => d.reason || ''); + const early = reasons.find((r) => r.includes('before the required delay')); + const tooFast = reasons.find((r) => r.includes('solved too fast')); + + if (early) { + passed++; + log(' ✓ Submitting before the source-specific floor is detected', colors.green); + } else { + failed++; + log(` ✗ Submitting before the source-specific floor was not detected. Reasons: ${reasons.join(' | ')}`, colors.red); + } + + if (!tooFast) { + passed++; + log(' ✓ Past the universal baseline, the strong "too fast" detection stays quiet', colors.green); + } else { + failed++; + log(' ✗ The strong "too fast" detection fired past the universal baseline', colors.red); + } +} + async function testProofOfWork() { log('\n[Proof of Work]', colors.cyan); @@ -2379,6 +2504,7 @@ async function runTests() { await testTouchSubmitIsNotProgrammatic(); await testMobileSensorDetection(); await testProofOfWork(); + await testAdaptiveChallengeCost(); await testSignalCommitment(); await testChallengeNonce(); await testKeystrokeCadence(); From c547f780e2b5aa34eb62da32aaba6ace847fd996 Mon Sep 17 00:00:00 2001 From: Chris Portscheller Date: Tue, 28 Jul 2026 16:25:50 -0500 Subject: [PATCH 2/4] fix(ci): track the bench manifest, and stop enumerating image sources MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three failures, all the same shape: a file the build needs is not where the build looks. `.gitignore` had `bench/*.json` to drop replay output, which also dropped bench/package.json. The harness could not be installed from a fresh clone, its playwright dependency was unrecorded, and CI could not run anything through npm in that directory — which is also why bench's `npm test` pointed at a file that had never existed without anyone noticing. Narrowed to `bench/*results*.json`. Both the Node and Python Dockerfiles enumerated the modules their entrypoint needs, under a comment instructing the reader to keep the list in step. The list went out of step immediately: neither image had the new module and both crashed on startup. That is the same failure as upstream #23, and the instruction did not prevent it because nothing about a Dockerfile review makes a missing require visible. They now copy the source and drop test files, so the list cannot drift again. Verified by building all three images and starting them: each serves /health and a challenge carrying difficulty and minAgeMs. --- .gitignore | 6 +++- bench/package-lock.json | 62 ++++++++++++++++++++++++++++++++++++++++ bench/package.json | 15 ++++++++++ server-node/Dockerfile | 16 ++++------- server-python/Dockerfile | 15 ++++------ 5 files changed, 94 insertions(+), 20 deletions(-) create mode 100644 bench/package-lock.json create mode 100644 bench/package.json diff --git a/.gitignore b/.gitignore index 575d22e..80fe1a8 100644 --- a/.gitignore +++ b/.gitignore @@ -46,5 +46,9 @@ docs/ # Bench harness: replay output and captured-corpus scratch. The corpus itself # (bench/corpus/) IS committed — it is the measurement baseline, and a benchmark # whose inputs are not in the repo cannot be reproduced or argued with. -bench/*.json +# +# Narrow, because `bench/*.json` also swallowed package.json: the harness could +# not be installed from a fresh clone, and CI could not run anything through npm +# in that directory. +bench/*results*.json bench/test-results/ diff --git a/bench/package-lock.json b/bench/package-lock.json new file mode 100644 index 0000000..d602477 --- /dev/null +++ b/bench/package-lock.json @@ -0,0 +1,62 @@ +{ + "name": "@webdecoy/fcaptcha-bench", + "version": "0.0.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "@webdecoy/fcaptcha-bench", + "version": "0.0.0", + "devDependencies": { + "playwright": "^1.49.0" + } + }, + "node_modules/fsevents": { + "version": "2.3.2", + "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz", + "integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + } + }, + "node_modules/playwright": { + "version": "1.62.0", + "resolved": "https://registry.npmjs.org/playwright/-/playwright-1.62.0.tgz", + "integrity": "sha512-Z14dG305dgaLu6foB1TXQagFiW8JfSUIUaUuPaKQ6NtBPKF1P/qXcqfh6c6K/icPqdy37JmjbiBXf6JNg6Sylw==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "playwright-core": "1.62.0" + }, + "bin": { + "playwright": "cli.js" + }, + "engines": { + "node": ">=20" + }, + "optionalDependencies": { + "fsevents": "2.3.2" + } + }, + "node_modules/playwright-core": { + "version": "1.62.0", + "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.62.0.tgz", + "integrity": "sha512-nsNRyq0r2zsG8AcRHWknc9QRA5XCueC7gWMrs+Gx2tlZn9hcl8zudfh00lhJPY1DE7NmZ6bDsT9g2yey8mXljA==", + "dev": true, + "license": "Apache-2.0", + "bin": { + "playwright-core": "cli.js" + }, + "engines": { + "node": ">=20" + } + } + } +} diff --git a/bench/package.json b/bench/package.json new file mode 100644 index 0000000..ca28672 --- /dev/null +++ b/bench/package.json @@ -0,0 +1,15 @@ +{ + "name": "@webdecoy/fcaptcha-bench", + "private": true, + "version": "0.0.0", + "description": "Measurement harness: labeled corpus, replay, and per-signal false-positive reporting for FCaptcha", + "scripts": { + "bench": "node run-bench.js", + "capture": "node capture/record.js", + "install-browsers": "playwright install chromium", + "test": "node --test \"lib/*.test.js\"" + }, + "devDependencies": { + "playwright": "^1.49.0" + } +} diff --git a/server-node/Dockerfile b/server-node/Dockerfile index b7618f2..291d535 100644 --- a/server-node/Dockerfile +++ b/server-node/Dockerfile @@ -16,16 +16,12 @@ WORKDIR /app/server-node COPY server-node/package.json server-node/package-lock.json* ./ RUN npm install --omit=dev -# Every module server.js requires. Copying the entrypoint alone leaves the image -# unable to start, and nothing in a Dockerfile review makes that visible — keep -# this list in step with server.js's requires. -COPY server-node/server.js \ - server-node/detection.js \ - server-node/clientip.js \ - server-node/limits.js \ - server-node/webbotauth.js \ - server-node/inputforensics.js \ - ./ +# All of it, rather than a list of modules to keep in step with server.js's +# requires. That list was the defect: a module added to the server and not to the +# Dockerfile produces an image that crashes on startup, which no Dockerfile +# review makes visible and which the previous instruction here did not prevent. +COPY server-node/*.js ./ +RUN rm -f *.test.js # Sibling of server-node/, so ../client/fcaptcha.js resolves. COPY client/fcaptcha.js /app/client/fcaptcha.js diff --git a/server-python/Dockerfile b/server-python/Dockerfile index 8c88578..98b6393 100644 --- a/server-python/Dockerfile +++ b/server-python/Dockerfile @@ -16,15 +16,12 @@ WORKDIR /app/server-python COPY server-python/requirements.txt . RUN pip install --no-cache-dir -r requirements.txt -# Every module server.py imports. Copying the entrypoint alone leaves the image -# unable to start, and nothing in a Dockerfile review makes that visible — keep -# this list in step with server.py's imports. -COPY server-python/server.py \ - server-python/detection.py \ - server-python/clientip.py \ - server-python/sitekeys.py \ - server-python/inputforensics.py \ - ./ +# All of it, rather than a list of modules to keep in step with server.py's +# imports. That list was the defect: a module added to the server and not to the +# Dockerfile produces an image that crashes on startup, which no Dockerfile +# review makes visible and which the previous instruction here did not prevent. +COPY server-python/*.py ./ +RUN rm -f test_*.py # Sibling of server-python/, so ../client/fcaptcha.js resolves. COPY client/fcaptcha.js /app/client/fcaptcha.js From f2f3f8fcbb9bd9ca7587b054466d9408556c1a86 Mon Sep 17 00:00:00 2001 From: Chris Portscheller Date: Tue, 28 Jul 2026 16:30:18 -0500 Subject: [PATCH 3/4] fix(bench): let the shell expand the test glob node --test only handles glob patterns itself from Node 22; CI runs Node 20 and received the literal string. Unquoted, so the shell expands it on both. --- bench/package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bench/package.json b/bench/package.json index ca28672..4d2544a 100644 --- a/bench/package.json +++ b/bench/package.json @@ -7,7 +7,7 @@ "bench": "node run-bench.js", "capture": "node capture/record.js", "install-browsers": "playwright install chromium", - "test": "node --test \"lib/*.test.js\"" + "test": "node --test lib/*.test.js" }, "devDependencies": { "playwright": "^1.49.0" From 96519ebaf91cea5bc39e67970404a01f8147156f Mon Sep 17 00:00:00 2001 From: Chris Portscheller Date: Tue, 28 Jul 2026 16:39:54 -0500 Subject: [PATCH 4/4] ci: run the Go and Python unit tests, and let discovery see all of them MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Neither had ever run in CI. Node's are covered by the benchmark workflow, so every check on a pull request was green while two of the three implementations were unverified. Among the tests that had never once executed there: - TestOnlyJA4TLSIsImplemented, the only thing standing between this MIT project and a FoxIO License 1.1 module that cannot legally ship in it - TestWeightsSumToOne, guarding the scoring weights invariant - every Python test, on an implementation whose only CI coverage was a container smoke test The sync rule says a change lands in all three servers, which is worth little if only one is checked. Go now runs vet and `go test -race`; Python installs requirements.txt and runs discovery. Discovery had to start working first. The Python tests are plain functions collected by a decorator rather than TestCase methods, and `unittest discover` could not see them: test_clientip and test_sitekeys guarded their runner under __main__ and reported "Ran 0 tests" without complaint, while test_detection and test_inputforensics ran at import and called sys.exit mid-discovery, surfacing as errors against suites that passed. Either way the count was wrong — 12 tests and two errors, where there are 43. testkit.py now holds the registry, so the same functions run both ways: readable per-test output from a single file, and visible to unittest for discovery and CI. It also replaces four copies of the same runner block. Verified that an injected failure fails through both paths and is attributed to the file that owns it. --- .github/workflows/unit-tests.yml | 67 ++++++++++++++++++++++ README.md | 20 ++++++- server-python/test_clientip.py | 23 ++------ server-python/test_detection.py | 23 ++------ server-python/test_inputforensics.py | 23 ++------ server-python/test_sitekeys.py | 23 ++------ server-python/testkit.py | 84 ++++++++++++++++++++++++++++ 7 files changed, 192 insertions(+), 71 deletions(-) create mode 100644 .github/workflows/unit-tests.yml create mode 100644 server-python/testkit.py diff --git a/.github/workflows/unit-tests.yml b/.github/workflows/unit-tests.yml new file mode 100644 index 0000000..bc03441 --- /dev/null +++ b/.github/workflows/unit-tests.yml @@ -0,0 +1,67 @@ +name: Unit tests + +# Go and Python unit tests, which nothing ran until now. +# +# Node's have been covered by the benchmark workflow since it existed, so the +# gap was invisible from the outside: every check on a pull request was green +# while two of the three implementations were unverified. Among the tests that +# had never run in CI: +# +# - TestOnlyJA4TLSIsImplemented, which is the only thing standing between this +# MIT project and a FoxIO License 1.1 module that cannot legally ship in it +# - TestWeightsSumToOne, which guards the scoring weights invariant +# - every Python test, on an implementation with no coverage but a container +# smoke test +# +# The sync rule says a change lands in all three servers. That is worth very +# little if only one of them is checked. + +on: + push: + branches: [main] + pull_request: + +jobs: + go: + name: Go + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-go@v5 + with: + # crypto/tls only exposes ClientHelloInfo.Extensions from 1.24, which + # native JA4 needs. go.mod says so; keep this in step with it. + go-version: '1.24' + cache-dependency-path: server-go/go.sum + + - name: Vet + working-directory: server-go + run: go vet ./... + + - name: Test + working-directory: server-go + run: go test -race ./... + + python: + name: Python + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v5 + with: + python-version: '3.11' + cache: pip + cache-dependency-path: server-python/requirements.txt + + - name: Install dependencies + working-directory: server-python + run: pip install -r requirements.txt + + # Discovery, not a loop over files: a file that stops being discoverable + # should show up as a drop in the count rather than as silence. It used to + # report 12 tests and two import errors where there are in fact 43. + - name: Test + working-directory: server-python + run: python -m unittest discover -p "test_*.py" -v diff --git a/README.md b/README.md index 1e7e96a..289d320 100644 --- a/README.md +++ b/README.md @@ -651,7 +651,7 @@ fcaptcha/ └── README.md ``` -> All three servers implement the same detection engine and must stay in sync. The Go scoring is unit-tested (`go test ./server-go/...`); `test/test-detection.js` exercises the full pipeline against a running server. +> All three servers implement the same detection engine and must stay in sync. Each has unit tests that run in CI, and `test/test-detection.js` exercises the full pipeline against a running server. ## Development @@ -671,10 +671,24 @@ open demo/index.html ### Running Tests -Go unit tests (no server required): +Unit tests, no server required. All three run in CI on every pull request. ```bash -cd server-go && go test ./... +cd server-go && go test -race ./... # Go +cd server-node && npm test # Node +cd server-python && python -m unittest discover -p "test_*.py" # Python +``` + +The Python tests are plain functions collected by a decorator rather than +`TestCase` methods, so `testkit.py` bridges them into `unittest` — run a single +file directly (`python test_sitekeys.py`) for readable per-test output. Discovery +and direct execution both report the same set; if the discovered count drops, a +file has stopped being discoverable. + +The measurement harness has its own tests: + +```bash +cd bench && npm test ``` End-to-end detection suite (runs against a live server): diff --git a/server-python/test_clientip.py b/server-python/test_clientip.py index 121c201..12d1bef 100644 --- a/server-python/test_clientip.py +++ b/server-python/test_clientip.py @@ -4,7 +4,7 @@ Run: python3 test_clientip.py """ -import sys +from testkit import TestRegistry from clientip import MAX_PROXY_MISCONFIG_WARNINGS, ProxyTrust @@ -31,12 +31,7 @@ def __init__(self, host, headers=None): self.headers = _Headers({k.lower(): v for k, v in (headers or {}).items()}) -_tests = [] - - -def test(fn): - _tests.append(fn) - return fn +test = TestRegistry() @test @@ -171,14 +166,8 @@ def untrusted_forwarding_is_warned_about_boundedly(): assert "further warnings suppressed" in buf.getvalue() +ClientIpTests = test.testcase("ClientIpTests") + + if __name__ == "__main__": - failures = 0 - for fn in _tests: - try: - fn() - print(f" ok {fn.__name__}") - except AssertionError as exc: - failures += 1 - print(f" FAIL {fn.__name__}\n {exc}") - print(f"\n{len(_tests) - failures}/{len(_tests)} passed") - sys.exit(1 if failures else 0) + test.main() diff --git a/server-python/test_detection.py b/server-python/test_detection.py index d17492d..b71936d 100644 --- a/server-python/test_detection.py +++ b/server-python/test_detection.py @@ -8,7 +8,7 @@ Run: python test_detection.py """ -import sys +from testkit import TestRegistry from detection import analyze_headers from server import ( @@ -19,12 +19,7 @@ DISPOSITIVE_FLOOR, ) -tests = [] - - -def test(fn): - tests.append(fn) - return fn +test = TestRegistry() def browser_headers(): @@ -115,14 +110,8 @@ def dispositive_floor(): assert apply_dispositive_floor(0.97, declared) == 0.97 -failed = 0 -for fn in tests: - try: - fn() - print(f" ok {fn.__name__}") - except AssertionError as e: - failed += 1 - print(f" FAIL {fn.__name__}\n {e}") +DetectionTests = test.testcase("DetectionTests") + -print(f"\n{len(tests) - failed}/{len(tests)} passed") -sys.exit(0 if failed == 0 else 1) +if __name__ == "__main__": + test.main() diff --git a/server-python/test_inputforensics.py b/server-python/test_inputforensics.py index 929c743..a9d735e 100644 --- a/server-python/test_inputforensics.py +++ b/server-python/test_inputforensics.py @@ -6,7 +6,7 @@ that passes on invented inputs proves nothing about real ones. """ -import sys +from testkit import TestRegistry from inputforensics import ( check_font_platform_coherence, @@ -16,12 +16,7 @@ check_typing_cadence, ) -tests = [] - - -def test(fn): - tests.append(fn) - return fn +test = TestRegistry() def field(interval, variance, dwell, key_count=43, paste_count=0): @@ -133,14 +128,8 @@ def blocked_font_list_never_flagged(): assert check_font_platform_coherence(fonts, "MacIntel") == [], fonts -failed = 0 -for fn in tests: - try: - fn() - print(f" ok {fn.__name__}") - except AssertionError as e: - failed += 1 - print(f" FAIL {fn.__name__}\n {e}") +InputForensicsTests = test.testcase("InputForensicsTests") + -print(f"\n{len(tests) - failed}/{len(tests)} passed") -sys.exit(0 if failed == 0 else 1) +if __name__ == "__main__": + test.main() diff --git a/server-python/test_sitekeys.py b/server-python/test_sitekeys.py index 5ba026d..e021556 100644 --- a/server-python/test_sitekeys.py +++ b/server-python/test_sitekeys.py @@ -2,7 +2,7 @@ Tests for site_key state bounds. Run: python3 test_sitekeys.py """ -import sys +from testkit import TestRegistry from sitekeys import ( MAX_TRACKED_IPS, @@ -11,12 +11,7 @@ SiteKeyGuard, ) -_tests = [] - - -def test(fn): - _tests.append(fn) - return fn +test = TestRegistry() @test @@ -106,14 +101,8 @@ def edge_cases_do_not_raise(): assert g.normalize("k", None) == "k" +SiteKeyTests = test.testcase("SiteKeyTests") + + if __name__ == "__main__": - failures = 0 - for fn in _tests: - try: - fn() - print(f" ok {fn.__name__}") - except AssertionError as exc: - failures += 1 - print(f" FAIL {fn.__name__}\n {exc}") - print(f"\n{len(_tests) - failures}/{len(_tests)} passed") - sys.exit(1 if failures else 0) + test.main() diff --git a/server-python/testkit.py b/server-python/testkit.py new file mode 100644 index 0000000..a839eda --- /dev/null +++ b/server-python/testkit.py @@ -0,0 +1,84 @@ +"""Shared plumbing for the Python test files. + +These tests are plain functions collected by a decorator rather than +``unittest.TestCase`` methods, which keeps them readable and dependency-free. +The cost was that ``python -m unittest discover`` could not see them: two files +guarded their runner under ``__main__`` and reported *Ran 0 tests* without +complaint, and two ran at import time and called ``sys.exit`` mid-discovery, +which surfaced as an error against a suite that actually passed. Either way the +count was wrong and nothing in CI ran them at all. + +So a registry does both jobs. ``main`` keeps the readable output for a human +running one file; ``testcase`` exposes the same functions to ``unittest`` so +discovery counts them instead of skipping them silently. +""" + +import sys +import unittest + + +class TestRegistry: + """Collects test functions and makes them runnable two ways. + + Used as a decorator:: + + test = TestRegistry() + + @test + def a_thing_holds(): + assert ... + + ThingTests = test.testcase("ThingTests") # for unittest discovery + + if __name__ == "__main__": + test.main() + """ + + def __init__(self): + self._tests = [] + + def __call__(self, fn): + self._tests.append(fn) + return fn + + def __iter__(self): + return iter(self._tests) + + def __len__(self): + return len(self._tests) + + def testcase(self, name): + """Build a TestCase whose methods are the registered functions. + + Call it at the bottom of the module, after everything is registered — + a registry read at import time would otherwise be empty and discovery + would go back to reporting zero. + """ + methods = {} + for fn in self._tests: + def method(self, _fn=fn): + _fn() + + method.__doc__ = fn.__doc__ + methods[f"test_{fn.__name__}"] = method + + # Without this the class reports its module as `testkit`, because that + # is where type() was called — so a failure names this file instead of + # the one holding the test that broke. + if self._tests: + methods["__module__"] = self._tests[0].__module__ + + return type(name, (unittest.TestCase,), methods) + + def main(self): + """Run everything, print a line per test, exit non-zero on failure.""" + failures = 0 + for fn in self._tests: + try: + fn() + print(f" ok {fn.__name__}") + except AssertionError as exc: + failures += 1 + print(f" FAIL {fn.__name__}\n {exc}") + print(f"\n{len(self._tests) - failures}/{len(self._tests)} passed") + sys.exit(1 if failures else 0)