Skip to content

Declared actors: opt-in human/agent identity on the tracker - #339

Merged
ralyodio merged 1 commit into
masterfrom
worktree-declared-actors
Oct 4, 2026
Merged

ralyodio merged 1 commit into
masterfrom
worktree-declared-actors

Conversation

@ralyodio

@ralyodio ralyodio commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

Why

Nothing on the wire separates a person from an agent driving a real browser. The tracker guesses from the user agent and the scripted cap, so an undeclared agent in Chrome reads as a human visit. This adds the honest half: an opt-in, self-reported declaration, so we can track internally who is who (first two actors: anthony@profullstack.com human, riotcoder@profullstack.com agent operated by Anthony) and offer it to customers.

How it works

  • An account registers actors (email, name, human|agent, optional human operator for an agent) and mints cpa_ tokens per browser or agent. The token is the credential; a bare email is never accepted.
  • The beacon carries the token by:
    • Crawlproof-Actor: cpa_… header (Playwright extraHTTPHeaders, Puppeteer)
    • the beacon body, from the site's localStorage, set by opening a site once with ?crp_actor=cpa_… (stripped from the URL) or crawlproof('actor', token)
  • No cookie channel. The tracker is documented as cookieless and credentials: 'omit' is pinned by tests/contract/stats-js.test.ts. "Declare this browser" in the dashboard hands out one ?crp_actor= link per tracked project plus a bookmarklet.

Built for people gaming it

  • One-way trust (applyDeclaration): a declared agent is believed (bot:declared, and the visitor rollup counts it as a bot). A declared human never overrides bot detection; a mismatch (bot UA or scripted cap) is counted as a contradiction on that actor.
  • Tokens are hashed (API pepper, actor: domain), revocable one by one, looked up at most once a minute per process.
  • A verified address can be claimed by one account only (partial unique index). The owner's login address is verified on creation; anything else gets an emailed link. Unverified addresses are labelled as such.
  • Privacy: names are visible to the actor's owner only unless made public; other site owners get per-kind counts.

Surfaces

  • API: GET/POST /api/tracker/v1/actors, PATCH/DELETE /actors/:id, POST/DELETE /actors/:id/tokens, GET /actors/:id/verify. Bearer crp_ or session.
  • /api/tracker/v1/stats gains declared (per-kind totals + your own actors); crawlproof stats prints a "Declared (self-reported)" block.
  • CLI: crawlproof actors list|add|token|revoke
  • MCP: list_actors, add_actor, mint_actor_token
  • UI: Settings → Declared actors
  • Docs: statistics page section; the privacy bullets now match the code (visitor/session ids have lived in localStorage since tracker: count people, not beacons (visitor rollup + scripted cap) #263; the page still said "No localStorage").

Migration

supabase/migrations/20261004120000_tracker_declared_actors.sql: apply by hand after merge (deploys don't run migrations). Safe in either order: before it, token lookups fail and every beacon is undeclared, actor_id is only written when an actor resolved, and declared is null.

Checks

  • tsc --noEmit clean; vitest run 217 files / 2742 tests pass (24 new in tests/tracker-declared-actors.test.ts, including route-level: declared agent in stock Chrome → p_kind: bot, bot:declared, actor_id on the raw event; declared human with GPTBot UA → stays bot, contradiction).
  • Migration applied twice to a throwaway Postgres 17 with stubs: idempotent; second verified claim on the same address refused; totals correct; RLS hides another account's actors/stats; authenticated cannot call tracker_touch_actor.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown

ThreatCrush Security Scan

49 finding(s)

HIGH/CRITICAL: 2 | MEDIUM: 32 | LOW: 15

Severity Rule Location
HIGH tls-verification-disabled lib/onion.ts:48
HIGH secret-generic-credential lib/sp/platforms/facebook.ts:32
MEDIUM js-unescaped-html-sink app/(app)/dashboard/admin/email-broadcast/EmailBroadcastForm.tsx:125
MEDIUM js-unescaped-html-sink app/(app)/dashboard/projects/[id]/autoblog/articles/[articleId]/page.tsx:214
MEDIUM js-unescaped-html-sink app/(marketing)/blog/[slug]/page.tsx:67
MEDIUM js-unescaped-html-sink app/(marketing)/blog/[slug]/page.tsx:97
MEDIUM js-unescaped-html-sink app/(marketing)/blog/[slug]/page.tsx:104
MEDIUM js-unescaped-html-sink app/(marketing)/blog/[slug]/page.tsx:110
MEDIUM js-unescaped-html-sink app/(marketing)/recent/page.tsx:186
MEDIUM js-unescaped-html-sink app/(marketing)/recent/page.tsx:190
MEDIUM js-unescaped-html-sink app/c/[project]/[slug]/page.tsx:77
MEDIUM js-unescaped-html-sink app/c/[project]/page.tsx:57
MEDIUM js-unescaped-html-sink app/careers.js/route.ts:228
MEDIUM js-unescaped-html-sink app/careers.js/route.ts:285
MEDIUM js-unescaped-html-sink app/layout.tsx:129
MEDIUM js-open-redirect app/login/form.tsx:39
MEDIUM js-unescaped-html-sink app/r/[token]/page.tsx:176
MEDIUM js-open-redirect app/signup/form.tsx:43
MEDIUM js-open-redirect components/billing/buy-credits-modal.tsx:98
MEDIUM js-unescaped-html-sink components/json-ld.tsx:8
MEDIUM js-unescaped-html-sink components/report/markdown-view.tsx:15
MEDIUM js-unescaped-html-sink lib/careers/page-templates.ts:198
MEDIUM js-dynamic-code-execution lib/crawl-limits.ts:67
MEDIUM redos-nested-quantifier lib/emailMarkdown.ts:41
MEDIUM redos-nested-quantifier lib/emailMarkdown.ts:324
MEDIUM redos-nested-quantifier lib/lx/articleGen.ts:99
MEDIUM redos-nested-quantifier lib/tracker/agent-gate.ts:61
MEDIUM sh-predictable-temp-path ops/selfhost/server/setup-supabase.sh:218
MEDIUM sh-remote-script-execution prober/deploy/provision.sh:30
MEDIUM sql-template-interpolation scripts/detect-slot-themes.ts:31
MEDIUM sql-template-interpolation scripts/purge-constructed-keywords.ts:163
MEDIUM sql-template-interpolation scripts/purge-offniche-keywords.ts:124
MEDIUM js-dynamic-code-execution scripts/test-crawl-limits.mjs:14
MEDIUM js-dynamic-code-execution scripts/test-crawl-limits.mjs:24
LOW secret-generic-credential app/(marketing)/docs/autoblog-webhook/page.tsx:145
LOW secret-generic-credential lib/sp/platforms/linkedin.ts:25
LOW js-dynamic-code-execution tests/careers-page-templates.test.ts:21
LOW js-dynamic-code-execution tests/careers-widget-script.test.ts:19
LOW js-dynamic-code-execution tests/careers-widget-script.test.ts:69
LOW js-dynamic-code-execution tests/contract/ad-visitor-id.test.ts:51
LOW js-dynamic-code-execution tests/contract/ad-visitor-id.test.ts:52
LOW js-dynamic-code-execution tests/contract/ads-click-cooldown-redis.test.ts:20
LOW js-dynamic-code-execution tests/contract/ads-click-cooldown-redis.test.ts:24
LOW js-dynamic-code-execution tests/contract/ads-click-cooldown-redis.test.ts:25
LOW js-dynamic-code-execution tests/contract/ads-click-cooldown-redis.test.ts:26
LOW js-dynamic-code-execution tests/contract/ads-click-cooldown-redis.test.ts:31
LOW js-dynamic-code-execution tests/contract/ads-click-cooldown-redis.test.ts:35
LOW secret-generic-credential tests/contract/posthog-integration.test.ts:13
LOW secret-generic-credential tests/lead-campaign.test.ts:16

Snippets are redacted; ThreatCrush never prints matched credential material.

Nothing on the wire separates a person from an agent driving a real
browser, so visitors may now say who they are. An account registers
actors (email, name, kind human|agent, optional human operator for an
agent) and mints cpa_ tokens per browser or agent. The beacon carries the
token by the Crawlproof-Actor header (headless agents) or the body, from
localStorage set by a ?crp_actor= link or crawlproof('actor', token).
A bare email is never accepted.

Built for people gaming it:
- one-way trust: a declared agent is believed (bot:declared, and the
  visitor rollup counts it as a bot); a declared human never overrides
  bot detection and a mismatch counts as a contradiction on the actor
- tokens hashed with the API pepper under an "actor:" domain
- a verified address can be claimed by one account only; the owner's
  login is verified on creation, anything else by an emailed link
- names are private to the owner unless made public; other site owners
  see per-kind counts

The tracker stays cookieless (credentials: 'omit' is pinned by a
contract test), so there is no cookie channel; the dashboard's
"Declare this browser" hands out one ?crp_actor= link per tracked
project plus a bookmarklet for other sites.

Surfaces: /api/tracker/v1/actors (+ tokens, verify), declared totals in
/api/tracker/v1/stats, `crawlproof actors`, MCP list_actors / add_actor
/ mint_actor_token, Settings -> Declared actors, docs section.

Migration 20261004120000_tracker_declared_actors.sql: apply by hand
after merge. Until then the ingest treats every token as undeclared.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ralyodio
ralyodio force-pushed the worktree-declared-actors branch from 2e3a332 to 733d4e0 Compare October 4, 2026 13:41
@ralyodio
ralyodio merged commit 5b797bb into master Oct 4, 2026
10 checks passed
@ralyodio
ralyodio deleted the worktree-declared-actors branch October 4, 2026 13:44
ralyodio added a commit that referenced this pull request Oct 4, 2026
#339 added `actors` to the in-repo CLI (cli/index.ts) only. The
`crawlproof` on PATH is the published @profullstack/crawlproof
(packages/cli, 0.3.0), which answered 'unknown command: actors'.

The command now lives in lib/tracker/actorsCli.ts and both CLIs call it,
the same way lib/emailTracking/cli is shared, so they cannot drift.
Package bumped to 0.4.0.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant