Skip to content

docs(agent): document source-data retrieval, in-flight following, and agent errors - #1324

Draft
hmishra2250 wants to merge 4 commits into
mainfrom
docs/agent-transparency-gaps
Draft

docs(agent): document source-data retrieval, in-flight following, and agent errors#1324
hmishra2250 wants to merge 4 commits into
mainfrom
docs/agent-transparency-gaps

Conversation

@hmishra2250

@hmishra2250 hmishra2250 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes the three biggest transparency gaps on the agent docs: how to get the raw content a run produced, how to follow a run while it's in flight, and what the agent endpoints return when they fail.

  • features/agent.mdx: adds "Following a run in progress" (states plainly that there's no SSE/websocket, with a 3-row table for trace polling, webhooks, and live view, and a worked poll example that groups by agent.id before sorting on producerSequence) and "Getting the agent's source data" (artifact.updated to GET /agent/{jobId}/snapshots/{snapshotId}, decoding only json snapshots and using markdown/html/text snapshots as-is). Cross-links the new error section from the traces note, and rewords the ~150-200 row ceiling to observed behavior only.
  • api-reference/errors.mdx: new ## Agent section: 10 error rows in ascending HTTP order, each message verbatim from the v2 agent controllers, the upstream snapshot route, or the shipped OpenAPI, plus a subsection mapping the five AgentTraceError.code values to one-sentence remedies and noting retryable.
  • api-reference/v2-openapi.json: corrects the trace events response description, which told callers to order globally by producerSequence.
  • snippets/v2/agent/trace/poll/{python,js,curl}.mdx: new poll-the-trace snippets.
  • snippets/v2/agent/artifacts/{python,js,curl}.mdx: new artifact-retrieval snippets.

No localized files touched.

Verified, but deliberately left out

  • Which content is guaranteed to land in an artifact. The schema verifies artifact kind values and the snapshot handoff, but nothing in the API repo establishes that every scraped page's markdown becomes an artifact: the split between artifacts and tool_call.finished result payloads is decided inside the agent service. The section is therefore written as "how to retrieve the artifacts a run produces", with an explicit caveat that artifacts are the run's output rather than a page-by-page archive. Needs confirmation from the agent-service owner before we can promise more.
  • The model default fix. The task listed "default": "spark-1-pro" on GET /agent/{jobId}, but it is already "default": "spark-2" on main (both the request enum and the status response), fixed by docs: overhaul agent endpoint reference, spark-2 default, trace/snapshot endpoints #1319. Nothing to change; the producerSequence description correction took its place as the mechanical fix.
  • The trace 200 example. Skipped. A 10-event example would have to invent toolName values and result shapes that no schema on main pins down, so field-level accuracy isn't something I can guarantee.
  • The 24-hour retention window and the toolName vocabulary. Both are visible in code (status expiresAt is created_at + 24h) but belong to the agent service's contract, so they're held pending confirmation from its owner.
  • Snapshot encoding for non-json artifacts. The shipped snapshot reference states that the snapshot field is the artifact content as a string and that only json artifacts are JSON-encoded, and both SDKs hand the field back as a raw string without decoding it. The recipes now decode json only and use the string directly for markdown/html/text. That is correct for json under either encoding and matches the documented behavior for the text kinds, but the encoding is owned upstream, so the text-kind half is documented-not-tested.
  • Invalid snapshot ID / Snapshot not found. Raised in review and now verified, so both rows are in the table. The snapshot route returns 400 {"error": "Invalid snapshot ID"} for a non-UUID id and 404 {"error": "Snapshot not found"} when no such snapshot exists, at https://github.com/firecrawl/extract-v3/blob/bf55aeae869d9c22ec8d87115c63bbc1b07eb687/src/routers/internal_extract.ts#L240-L251, on the route that GET /v2/agent/{jobId}/snapshots/{snapshotId} proxies verbatim. Those bodies carry no success field, which is also what the section's shape caveat now documents.
  • Threat-protection 403s on POST /agent. Real and verified, but they're an org-policy behavior shared with /scrape and /crawl rather than agent-specific, so they belong in a threat-protection row, not the Agent section.

🤖 Generated with Claude Code

… agent errors

Adds a "Getting the agent's source data" recipe (artifact.updated ->
snapshot fetch), a "Following a run in progress" section stating plainly
that delivery is polling plus webhooks, and an Agent section in the error
catalog with messages read verbatim from the v2 agent controllers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mintlify

mintlify Bot commented Aug 27, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
firecrawl 🟢 Ready View Preview Aug 27, 2026, 1:17 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review follow-up. The snapshot endpoint returns markdown, html, and text
artifacts as the content itself and only JSON-encodes json artifacts, so
the artifact recipes no longer decode every kind. The poll examples now
keep polling past terminal status until the trace goes quiet, and the
cURL one dedupes on eventId instead of reprinting the whole trace.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… files

Both messages are returned by the upstream snapshot route and proxied
verbatim by GET /v2/agent/{jobId}/snapshots/{snapshotId}, so callers see
them as-is. The cURL poll example now keeps its scratch files in a
mktemp directory it removes on exit instead of writing to the cwd.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant