Repository navigation
[pull] main from firecrawl:main - #393
Merged
Merged
Conversation
…4392) * feat(api): add opt-in prompt-injection check to JSON-mode extraction Scraped page content can contain instructions that hijack the JSON-mode extraction model into ignoring the caller's schema or leaking data. Add a checkPromptInjection option that runs a dedicated classifier before extraction and blocks the request (SCRAPE_PROMPT_INJECTION_DETECTED, 403) when it detects an attempt, billing for the check regardless of outcome since it incurs real cost either way. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * fix(api): gate prompt-injection credit-delta test to production only Self-hosted idmux has no real per-team isolation -- it falls back to one shared static identity, so a team-wide credit delta gets polluted by whatever else is running concurrently in the suite. Split the block-path assertions (which need no isolation) from the credit-delta assertion (gated to TEST_PRODUCTION, matching billing.test.ts's existing convention). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * fix(api): address cubic review findings on prompt-injection guard - Randomize the untrusted-content delimiter tag per call so page content can't spoof a matching closing tag and escape the boundary. - Resolve the actual model id from getModel() for cost tracking instead of assuming the requested model name, so a MODEL_NAME override doesn't get mispriced. - Record a guard-call attempt even when it fails, and key both the success and failure billing paths off that same recorded-call signal instead of just whether the check was requested. - Add a real credit-delta assertion to the v2 block-path test (previously claimed by name only) and match billing.test.ts's concurrent-with-isolated- identity pattern in both v1 and v2. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * fix(api): only bill the guard call when a provider request actually dispatched The fail-open catch was recording (and thus billing for) every guard failure, including errors that never reach the provider (schema/argument validation before the request is sent). Only record the call for NoObjectGeneratedError and APICallError, which specifically indicate a request was actually dispatched to OpenAI. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * fix(api): invert dispatch check to a closed denylist of pre-flight errors The previous allowlist (NoObjectGeneratedError, APICallError) missed other AI SDK error types that also only occur after a real request was sent (JSONParseError, InvalidResponseDataError, EmptyResponseBodyError, RetryError wrapping dispatched attempts). Rather than keep growing that allowlist, invert it: bill by default, excluding only the small, closed set of errors that never reach the network (bad model/prompt/config). New SDK error types now default to "billed" instead of silently missing the charge until specifically added. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * feat(sdks): expose checkPromptInjection across all language SDKs Adds the JSON-mode prompt-injection guard option to every SDK's JSON format type: dot-net, elixir, go, java, js, php, python, ruby, rust. Serialization was verified per-language (build/lint/compile plus a functional smoke test for python, ruby, and elixir where camelCase conversion isn't automatic). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * fix(api): run prompt-injection guard on SmartScrape-derived content too Cubic caught that the guard only inspected the initial page markdown -- when SmartScrape triggers, extraction actually runs against smartScrape-derived markdown that never passed through the check, letting injections on interactive pages bypass the opt-in guard entirely. Also documents the deliberate fail-open tradeoff on guard outages per a second, lower-severity finding. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * test(ruby-sdk): add JsonFormat#to_h coverage Cubic noted JsonFormat was the only sibling format class without a to_h test. Locks down schema/prompt serialization plus the check_prompt_injection:false vs. unset distinction, since .compact only drops nil and must not drop an explicit false. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * test(ruby-sdk): trim JsonFormat test to match sibling style The added test was more verbose than the sibling QueryFormat/ QuestionFormat tests (multiple assertions, an untested-for concern). Reduced to one assertion per test, matching the existing pattern. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * perf(api): run prompt-injection guard concurrently with extraction Guard and the first-pass extraction call now fire together instead of sequentially, cutting added latency from guard+extraction down to max(guard, extraction). The guard's settlement is still inspected before the extraction result is ever read, so a flagged page still never reaches the caller -- only the ordering of the two dispatches changed, not the blocking guarantee. Scoped to scrape's JSON-mode path only (extractData's sole caller); SmartScrape's own guard call is untouched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * fix(api): chunk guard input instead of trimming, closing a bypass trimToTokenLimit discarded everything past 120k tokens, but extraction has no length cap of its own -- an attacker could pad a page past that boundary and place the injection after it, invisible to the guard. Now chunks the full page into 120k-token windows and classifies each; blocks if any chunk flags. Verified live: an injection placed past the old boundary now gets caught (previously bypassed it entirely). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * fix(api): drop tiktoken from guard chunking, use plain char chunks tiktoken's encode() has pathological worst-case behavior on low-entropy input: a single call took 126s at 500k chars and crashed the WASM module at 2M chars in testing. The guard doesn't need exact token counts, just a safe upper bound per classifier call, so chunking is now plain character slicing -- no tokenizer, no unbounded-time risk, verified at 1ms for 5M pathological chars vs. tiktoken's multi-minute hang/crash on the same class of input. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * fix(api): tighten guard chunking per cubic review - Lower GUARD_MAX_CHUNK_CHARS to 32000 -- 100k UTF-16 chars of dense/ multi-byte content could exceed gpt-4o-mini's 128k-token window, and a resulting failure fails open (bypassing the check). - Overlap consecutive chunks by 2000 chars so an injection split exactly across a chunk boundary can't evade both halves. - Record NoObjectGeneratedError's usage instead of zeroing it -- the provider still returned a billable response. - Batch classifier calls (5 at a time) instead of firing all chunks concurrently, bounding rate-limit exposure and stopping later batches from being scheduled once a batch detects an injection. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * style(api): trim narrative comments in the guard to one line each Comments were explaining debugging history (tiktoken hanging/crashing) rather than current code behavior -- that belongs in the commit message, not the source. Cut every multi-line comment down to one line, dropping the ones that just restated the diff. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * feat(api): cap JSON extraction content size for all json-mode requests Move the oversized-content guard from the prompt-injection check specifically to extractData itself, so any JSON-mode extraction -- not just checkPromptInjection requests -- is protected from being sent multi-megabyte content with no cap of its own. New JsonExtractionContentTooLargeError (SCRAPE_JSON_CONTENT_TOO_LARGE, 403) fires before any extraction or guard call is dispatched. Verified live: fires instantly with zero API calls, on both checkPromptInjection on and off, while normal-sized pages are unaffected. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 * fix(api): return 400 for oversized JSON extraction content, not 403 403 is this codebase's convention for policy/security blocks (media access, injection detected, unsafe domains); a size limitation is an unprocessable-request condition, matching the 400 precedent set by SCRAPE_ACTIONS_NOT_SUPPORTED. 413 was considered and rejected: it's defined for oversized request entities, not oversized fetched resources, and gets special treatment from proxies. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVzcYnVDegGLruH2ZQR714 --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com>
) Partner-provisioned accounts are created at runtime by the partner API, so `FIREBILL_ORG_IDS` cannot keep up: every new one needed a config change and a fleet restart before its usage billed correctly. Until then the monolith billed Autumn directly, unsplit — the account's own small allowance paid for usage the partner had agreed to fund, silently and with nothing erroring. That is not hypothetical. It happened on 2026-08-25 while testing: an account provisioned minutes earlier scraped, and the charge went straight to Autumn because its org was not yet on the list. `shouldRouteToFirebill` gains a third way in, checked before the sticky sample: if (firebillOrgIds().has(orgId)) return true; // test orgs if (opts?.gatewayProvisioned) return true; // new return sampled(orgId, config.FIREBILL_ROLLOUT_PERCENT); **This branch is meant to be deleted.** It exists only because the ramp is below 100. A partner's usage cannot depend on a sampling bucket, and a sticky percentage is a probability. At 100 every org routes anyway, the branch stops changing any outcome, and it should go along with the lookup feeding it. ## Existence of the row, not `gateway_enabled` Routing asks only whether `partner_provisioned_accounts` has a row for the team. Provisioning is the durable fact; whether to split *right now* stays firebill's decision via `gateway_enabled`, so flipping that kill switch stops splitting without also changing which service bills. ## The caching is the whole risk A positive is cached forever: provisioning is one-way, so it cannot go stale, and re-reading would put a query on every billing event. A negative is cached for `FIREBILL_GATEWAY_NEGATIVE_TTL_SECONDS` (60 by default) because it *does* go stale — the moment a partner provisions a new account, every pod holding a negative bills its usage outside firebill. The existing `customerOrgCache` has no expiry, which is right for an immutable org_id and would have been wrong here. A failed lookup is not cached at all. Caching it as a negative would turn one replica blip into a TTL's worth of gateway usage billed to the wrong account. The 60s exposure is bounded in a second way: a newly provisioned account has a full balance, so early charges land on its own credits either way. The divergence only appears once it is drained, by which point it is long cached positive. ## Options object, not a positional boolean The existing suite caught this: `orgs.filter(shouldRouteToFirebill)` passes the array *index* as the second argument, so a positional boolean routed every org except element zero. An options object cannot be fooled that way — destructuring a field off a number yields undefined — and TypeScript now rejects the callback form outright. A test pins the runtime behaviour too. ## Verification 112 tests pass across the autumn and rollout suites, `tsc --noEmit` clean, prettier clean. New tests: a partner-provisioned team routes at 0 percent and off the allowlist; a positive survives the row disappearing; a failed lookup routes direct and is re-asked on the next event. The shared Drizzle test stub returned one row for every query, so the new lookup saw the teams row and read every team as partner-provisioned. It now distinguishes the two lookups by selected columns. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )