Skip to content

docs(social): authenticated Scraper Results polling + scrape persistence note (chat#1840)#262

Closed
sweetmantech wants to merge 2 commits into
mainfrom
feat/apify-runs-account-scope
Closed

docs(social): authenticated Scraper Results polling + scrape persistence note (chat#1840)#262
sweetmantech wants to merge 2 commits into
mainfrom
feat/apify-runs-account-scope

Conversation

@sweetmantech

@sweetmantech sweetmantech commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

Contract-first PR for recoupable/chat#1840 — both items. The api PRs implementing this follow and link back here. Merge order: this docs PR → api PRs.

GET /api/apify/runs/{runId} — auth contract (item 1)

Today the endpoint is admin-only, while these docs, the recoup-platform-api-access skill, and customer task prompts all tell account-key agents to poll it (the can-start/can't-read asymmetry in the issue). New contract (decision 2026-07-03, superseding the earlier owner-or-admin draft): any valid API key or Bearer token can poll any run id. Scrape results are public social content and run ids are unguessable; authentication still gates the endpoint so anonymous callers can't use it as a free Apify proxy. No new response codes — 200/400/500 unchanged.

POST /api/socials/{id}/scrape — persistence note (item 2)

One sentence documenting what completed runs persist: profile stats to the social row, and on Instagram/TikTok/X the scraped post URLs (surfaced via Get Artist Posts). TikTok/X post persistence is api#753.

JSON re-validated; targeted edits only.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Clarified API documentation for social endpoints, including authentication requirements for polling runs.
    • Added details on what data is saved after a social scrape completes, including persisted profile stats and scraped post URLs for supported platforms.

…e note (chat#1840)

GET /api/apify/runs/{runId} is moving from admin-only to owner-or-admin:
the account that started the run via POST /api/socials/{id}/scrape can
poll it. Document the new 403 (foreign run, non-admin) and 404 (no
recorded owner, non-admin) responses and the ownership rules in the
description. Runs with no recorded owner (pre-tracking, or started from
other endpoints) stay admin-only.

Also note on Social Scrape that completed runs persist profile stats,
and post URLs on Instagram/TikTok/X — surfaced via Get Artist Posts.

Contract for recoupable/chat#1840 (both items); api PRs follow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mintlify

mintlify Bot commented Jul 3, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
recoup-docs 🟢 Ready View Preview Jul 3, 2026, 6:50 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@coderabbitai

coderabbitai Bot commented Jul 3, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Updated OpenAPI documentation descriptions for two Social endpoints in social.json: clarified authentication requirements for polling run status, and detailed post-completion persistence and visibility behavior for scraped profile stats and post URLs.

Changes

Social API Documentation

Layer / File(s) Summary
Endpoint description clarifications
api-reference/openapi/social.json
Expanded descriptions for GET /api/apify/runs/{runId} (authentication/polling behavior) and POST /api/socials/{id}/scrape (post-completion persistence of stats and platform-specific post URL visibility).

Estimated code review effort: 1 (Trivial) | ~2 minutes

Possibly related issues

Possibly related PRs

  • recoupable/docs#258: Both PRs modify the description of POST /api/socials/{id}/scrape in the same social.json file.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly captures the main docs changes: authenticated polling and scrape persistence notes.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/apify-runs-account-scope

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Re-trigger cubic

…ision)

Design decision 2026-07-03: scrape datasets are public social content and
Apify run ids are unguessable, so possession of a runId plus any valid
API key / Bearer token is sufficient — no ownership scoping. Drops the
403/404 responses; keeps the authentication requirement and the scrape
persistence note.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@sweetmantech sweetmantech changed the title docs(social): account-scoped Scraper Results auth + scrape persistence note (chat#1840) docs(social): authenticated Scraper Results polling + scrape persistence note (chat#1840) Jul 3, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 1 file (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="api-reference/openapi/social.json">

<violation number="1" location="api-reference/openapi/social.json:485">
P1: The GET /api/apify/runs/{runId} endpoint description says "Requires authentication: any valid API key or Bearer token can poll any run id" but the endpoint has no `security` field. Other authenticated endpoints in this file (e.g., GET /api/artists/{id}/posts, GET /api/connectors) explicitly declare `security: [{ "apiKeyAuth": [] }, { "bearerAuth": [] }]`. Without a `security` declaration, the OpenAPI spec tells consumers that this endpoint accepts unauthenticated requests, contradicting both the description and the new contract.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

"/api/apify/runs/{runId}": {
"get": {
"description": "Check the status and retrieve results from Apify scraper runs. This endpoint uses the Apify API Client to fetch the current status of a scraper run and its results if available. Use the runId returned from endpoints like Instagram Comments, Instagram Profiles, Social Scrape, or Artist Socials Scrape to poll for results.",
"description": "Check the status and retrieve results from Apify scraper runs. This endpoint uses the Apify API Client to fetch the current status of a scraper run and its results if available. Use the runId returned from endpoints like Instagram Comments, Instagram Profiles, Social Scrape, or Artist Socials Scrape to poll for results. Requires authentication: any valid API key or Bearer token can poll any run id — scrape results are public social content and run ids are unguessable, so possession of a runId plus a valid credential is sufficient.",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: The GET /api/apify/runs/{runId} endpoint description says "Requires authentication: any valid API key or Bearer token can poll any run id" but the endpoint has no security field. Other authenticated endpoints in this file (e.g., GET /api/artists/{id}/posts, GET /api/connectors) explicitly declare security: [{ "apiKeyAuth": [] }, { "bearerAuth": [] }]. Without a security declaration, the OpenAPI spec tells consumers that this endpoint accepts unauthenticated requests, contradicting both the description and the new contract.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At api-reference/openapi/social.json, line 485:

<comment>The GET /api/apify/runs/{runId} endpoint description says "Requires authentication: any valid API key or Bearer token can poll any run id" but the endpoint has no `security` field. Other authenticated endpoints in this file (e.g., GET /api/artists/{id}/posts, GET /api/connectors) explicitly declare `security: [{ "apiKeyAuth": [] }, { "bearerAuth": [] }]`. Without a `security` declaration, the OpenAPI spec tells consumers that this endpoint accepts unauthenticated requests, contradicting both the description and the new contract.</comment>

<file context>
@@ -482,7 +482,7 @@
     "/api/apify/runs/{runId}": {
       "get": {
-        "description": "Check the status and retrieve results from Apify scraper runs. This endpoint uses the Apify API Client to fetch the current status of a scraper run and its results if available. Use the runId returned from endpoints like Instagram Comments, Instagram Profiles, Social Scrape, or Artist Socials Scrape to poll for results. Authorization is account-scoped: the account that started the run via the [Social Scrape](/api-reference/social/scrape) endpoint (or an admin) can poll it. Runs started by other accounts return 403; run ids with no recorded owner — including runs started before ownership tracking existed and runs started from other endpoints — return 404 for non-admin callers.",
+        "description": "Check the status and retrieve results from Apify scraper runs. This endpoint uses the Apify API Client to fetch the current status of a scraper run and its results if available. Use the runId returned from endpoints like Instagram Comments, Instagram Profiles, Social Scrape, or Artist Socials Scrape to poll for results. Requires authentication: any valid API key or Bearer token can poll any run id — scrape results are public social content and run ids are unguessable, so possession of a runId plus a valid credential is sufficient.",
         "parameters": [
           {
</file context>

@sweetmantech

Copy link
Copy Markdown
Collaborator Author

Closing unmerged — after the chat#1840 design flip to the capability model (any valid key can poll any run), this PR no longer fixes any contract drift: the existing page never documented the admin gate, and api#752 makes the code match what the docs already imply (poll with your key, 200/400/500). The one-sentence Social Scrape persistence note is a nice-to-have that can ride along the next time that page is touched.

@sweetmantech sweetmantech deleted the feat/apify-runs-account-scope branch July 3, 2026 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant