feat(detector): pipeline-security fixes, gpu-substance perf tell, and trust-boundary hardening#226
Merged
Merged
Conversation
…flowconn + fleet landings Stale-doc sweep after v1.16.0. frontier.md: add the KS_GL_RENDERER Mesa-patch capability + reconcile the composition note and buy-list (the Chromium renderer wall fell in software, not just the GPU caps); bump the reconcile date. coherent-stack.md: overturn the Firefox-only claim (patched Mesa gives Chromium/WebKit a coherent renderer natively; coherent-except-webgpu), and fix the windows-firefox residual (the flowConn fix landed the collector POST, no gVisor). evaders/README: add the mesa-patch GPU-coherence capability. fleet/README: add the morph-diffuse strategy row (sheds ja4_prefix). README stats already fresh.
…zed; perf is the durable tell Probed the Mesa-patched Chromium for the substance the renderer-string spoof left behind. GL_VERSION + extensions are ANGLE-normalized (GPU-agnostic); the caps vector is llvmpipe's real values but telling them from a real GPU needs a reference DB (external, no real GPU in-sandbox). The existing webgpu_webgl_vs already catches it data-free. The one durable in-sandbox tell left is performance: software renders ~40-100x slower than any real GPU (a physics gap), so a hardware renderer claim with software throughput convicts, FP-safe by the gap. Next: a fixed-budget WebGL benchmark tell.
… loop earned-closed Grounded the physics tell: a forced-readback WebGL benchmark renders in 1072ms on llvmpipe vs ~10ms on any real GPU (a ~100x gap, >=10x even for throttled weak hardware) — a hardware-renderer claim at software speed is spoofed. Fully specced (collector benchmark + browser.webgl_render_ms + an experimental br.webgl_perf_vs_renderer). Deferred the collector integration: it's experimental + forward-looking (the current spoof is already caught by webgpu_webgl_vs) and its FP-promotion needs a real-GPU corpus. Loop earned-closed: the blue side already holds the spoof in-sandbox; deeper substance is external-data-bound.
…an override edge fp The replay axis surfaced a foundational gap. The collector POSTs to /ingest via the edge proxy, the edge forwards the body verbatim, and merge_sessions keeps latest-per-(layer,kind) by client observed_at with NO source-trust check. Grounded: a client-forged layer=network ja4 with a later timestamp overrides the edge's real server-observed one (/session shows the forged value). So the 'unforgeable' network layer is client-forgeable, and a full captured payload replays coherently. Fix (next): a source-trust boundary so layer=network signals are accepted only from the authenticated edge, not client-proxied POSTs.
The edge is the sole authority for network signals (it observes the raw ClientHello/TCP/H2). A client POSTing to /ingest via the edge proxy could inject a forged layer=network signal that overrode the edge's server-observed fingerprint in the detector's latest-per-kind merge — collapsing the network layer's unforgeability. sanitizeClientIngest strips layer=network from client-proxied /ingest bodies before forwarding (the edge's own signals go via a separate direct POST, untouched); fail-open on parse error. The collector never emits network signals (138 browser + 18 behavioral, zero network), so it's FP-safe. Grounded red->blue: a forged ja4 through the edge is now stripped (session network=[]) vs the prior override; browser/behavioral pass through. Unit tests + full edge vet/test green.
…ium-only Grounded stealth-WebKit + the patched Mesa: WebKit reports a fixed WebGL identity (Apple GPU / WebKit WebGL / WebKit, maxTex 16384) regardless of the driver — Safari's privacy normalisation — so the patch has zero effect on WebKit and WebKit never leaked llvmpipe anyway. Corrects the ADR-0008 engine-agnostic claim: the patch fixes Chromium only. Full stealth-WebKit verdict: bot 0.6 with GPU tells NONE (GPU coherent); the blockers are cross-OS (navplatform/font/tcp/tls_grease/dpr under a Linux container + macOS UA) + automation. A coherent Safari slice is cross-OS-bound, not Mesa-unlocked.
…(chromium-only) Follow-through on the WebKit grounding: the Mesa patch is Chromium-only, not engine-agnostic. WebKit self-normalises its WebGL renderer (Apple GPU / WebKit), ignoring the driver, so the patch is moot for it — but WebKit never leaked the software renderer either. Fixes the claims added last session.
…perimental) A string patch (the Mesa red evader) can name a hardware GPU while rendering on software, clearing webgl_software + the caps/tamper family — but it cannot fake SPEED. The collector runs a forced-readback WebGL benchmark (a bare draw is deferred, times ~0), pushes browser.webgl_render_ms, and emits webgl_perf_vs_renderer when the renderer names hardware yet renders slower than 50ms. Corroborating + experimental until a real-GPU corpus calibrates the threshold (sandbox has only software GL). Grounded red<->blue: the Mesa-patched Chromium fires it (render_ms=102ms) with webgl_software silent; a plain llvmpipe Chromium does not (it self-names software). The durable substance tell — survives a webgpu closure since speed can't be string-spoofed. Suite green (507 passed, 95.71%); catalog regenerated.
…hromium blocked) Grounded that a coherent linux-chrome profile can't be added: the registry's invariant is zero firing coherence tells, but a Chromium morph via the Mesa patch trips webgpu_webgl_vs (env-bound) AND our own new webgl_perf_vs_renderer (the perf tell catches the software speed behind the hardware string). The blue tell we just shipped reinforces that Chromium isn't registry-coherent in-sandbox. Documented in morph_profiles.yaml so it isn't retried; 1a (bake the driver) deferred as low-ROI productization of a now-caught morph. Priority 1's value was the perf tell; the three frontiers are substantially exhausted.
…al client signals The rung-2 fix stripped only layer=network, but reputation is server-authoritative too (the detector's IP classifier produces asn_is_datacenter/is_abuse_listed) and the collector never emits it (140 browser + 18 behavioral, zero network/reputation). A client could inject a clean reputation signal to clear a datacenter/abuse tell via the same merge. sanitizeClientIngest now keeps ONLY browser+behavioral, dropping every other layer. Grounded: a forged network ja4 + a forged reputation through the edge are both stripped, browser kept. Full edge vet/test green.
…op terminus This cycle found and closed a new detection axis — signal integrity / replay — so add it to the frontier board (earned-closed): the edge is now the sole authority for server-observed layers after the trust-boundary fix. Note the durable perf tell in the same axis. The three-frontiers loop's high-value in-sandbox rungs are exhausted; the remaining candidates (detector-side defense-in-depth, Dockerfile bake, Linux-WebKitGTK) are marginal or external-bound and deliberately not pursued.
…xact-build reference Traced the session flow (edge ks_sid cookie -> collector reads -> edge posts network signals). The base collector has no server-issued liveness binding today. Freshness-only nonce echo (1a) is marginal (a scripted replayer echoes the fresh cookie; whole-body replay is already caught by the network layer post trust-boundary-fix). The valuable form is a nonce-seeded fingerprint (1b), but verifying the seeded response needs the claimed build's expected behaviour = the P2 reference. So #1 is coupled to #2: capture the reference first, then nonce-seed it. Kicked off the 3-engine reference capture.
…y level; version is external Captured the real Chromium reference and checked it against the registry: the premise that JA4/H2/CH-UA are checked independently is wrong. Four shipped rules (tls_vs_ua_browser, h2_vs_ua_browser, ja4_tool_vs_ua, h2_unknown_vs_ua) already bind the per-layer engine hints to the UA family, grounded + FP-safe with a Chromium-family collapse. The real Chromium ref confirms coherence (all hints=chrome, silent). So the family-level joint constraint is banked; the remaining version-level leverage needs a browser-version reference matrix (JA4/H2 are engine-stable) = external-data-bound.
Checked engine-level JS-quirk coherence against the registry: comprehensively shipped. Engine-vs-UA is cross-checked via apple_ua_nonwebkit / firefox_ua_nongecko / ch_ua_on_non_chromium_ua / vendor_engine, and the forged-JA4 gap (uTLS Chrome-identical JA4) is closed by tls_ext_order_static (per-connection extension permutation). A deeper JS-engine tell would be redundant with these. Twice-in-a-row grounded no-new-rung: #2 family + #3 engine are done, version-level is external (engine-stable JA4/H2), #1 liveness is hard/arena-converging. The suite is deeper than the loop assumed — a confidence result, not a gap.
…disclosure)
The edge proxied every path to the internal detector, so /scoreboard (every visitor's verdict) and
/session/{id} (any session's raw JA4/IP) were reachable unauthenticated through edge:8443 —
require_admin
gates them but is open when no admin token is set (the shipped compose sets none). Grounded the
exploit,
then added a path deny-list (isAdminPath) that 404s /scoreboard, /session, /verdict and the FastAPI
docs
before proxying (path.Clean+lowercased so /Session, //, /../ and %2e can't bypass). Grounded after:
those
paths 404, while /, /healthz, /ingest, and cookie-scoped /inspect still work. Unit + full edge tests
green.
The /ingest POST had no body-size limit, and the edge's sanitizeClientIngest does io.ReadAll — so the edge buffered the whole body before the detector even parsed it. Grounded: a ~20MB body was accepted (200); a 1GB body would OOM the edge + detector. Fixed: read via io.LimitReader(1MiB+1) and 413 anything larger before buffering or proxying. Grounded after: 1.1MB -> 413, 20MB -> reset after 1MiB, normal -> 200. Unit + full edge tests green. Session_id entropy cleared safe (128-bit crypto/rand).
…ounded clears) Audited /ingest type-confusion and X-KS-Session spoofing. Type-confusion is safe: 7 malformed values (dict/number/null/list/bool/deep-nested) all scored to 200 with no crash, and ingest.py type-guards throughout (isinstance(value, str)) so malformed values are skipped, not crashed on. X-KS-Session spoof is safe: the edge Set-overwrites the header and correlation is by the 128-bit cookie-derived session_id. Two grounded safe-clears, no fix. Next targets: arena solve-verification, edge request-smuggling.
…/replay/bypass) Audited the arena challenge/solve verification. Safe: the secret is 256-bit crypto/rand per-start (unforgeable HMAC/ed25519 tokens, no default key); the captcha answer is server-side and never serialised; verify is single-use take() + answer-checked before signing, body capped at 64KiB. Grounded live through the edge: wrong answer, unknown id, and replayed consumed id all return ok:false with no token. PoW is server-verified, PACT tracks redemption + expiry. Second consecutive safe-clear; edge request-smuggling is the last fresh target.
…e audit terminus Audited the edge proxy. Smuggling grounded safe via raw TLS: CL.TE conflict -> 422 (Go rejects framing, smuggled request never processed), dup Content-Length -> 400, normal -> 200; no desync. IP trust safe: IP-rep uses RemoteAddr (unspoofable), and the detector runs plain uvicorn (no proxy_headers) so request.client.host is the real peer, not client-controllable. Noted low-severity: the arena rate gate buckets per-edge not per-client (recorded, not a client exploit). Third safe-clear -> audit terminus: 2 fixes + 3 safe-clears this loop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A grounded batch (19 commits). Every fix below was demonstrated as a live exploit, fixed, and re-grounded; every detection change fires on the evader and stays silent on a real browser.
Security fixes (pipeline audit — attack the detection system, not the fingerprint)
/ingest(latest-per-kind merge, no source-trust). The edge now strips all server-authoritative layers (network, reputation) from client-proxied bodies. Restores the "you can't fake your JA4" foundation./scoreboard(every visitor's verdict) and/session/{id}(raw JA4/IP) were reachable through the public edge (admin token unset by default). The edge now 404s admin paths before proxying (path.Clean+lowercased against bypass)./ingesthad no body-size limit (20MB accepted; the edge buffered it all). Now a 1 MiBLimitReadercap → 413.Cleared safe (grounded): session integrity (128-bit
crypto/rand), input type-confusion, X-KS-Session spoof, arena gate verification (unforgeable secret, single-use answers), edge request-smuggling (CL/TE → 422) + IP trust.Detection + evasion
webgl_perf_vs_renderer— the durable GPU-substance tell: a renderer that names hardware but renders at software speed (catches the Mesa-patch spoof via physics, survives a WebGPU closure). Grounded red↔blue, experimental/corroborating.frontier.md(new signal-integrity axis) +coherent-stack.md; documented why the morph registry stays camoufox-only.All commits authored + committed by datascry; conventional-commit; detector 95% coverage + edge vet/test green locally.