Skip to content

Add a page-wide Humans / Bots / All toggle to the project stats page - #233

Merged
ralyodio merged 1 commit into
masterfrom
stats-who-toggle
Sep 5, 2026
Merged

ralyodio merged 1 commit into
masterfrom
stats-who-toggle

Conversation

@ralyodio

@ralyodio ralyodio commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

What the toggle does

/dashboard/projects/:id/stats gets a page-wide Humans · Bots · All segmented control (same visual language as the per-card timeframe tabs), persisted in the URL as ?who=humans|bots|all, default humans. One value drives everything on the page:

  • the headline Metric tiles (Humans: Human visits + AI referrals; Bots: Bot crawls; All: the three tiles as before)
  • the TrafficPulse series (Humans: the human band with AI referrals inside it; Bots: the bot band alone; All: humans + bots stacked), and its frame total
  • every BreakdownPanel / RankedPanel (event mix, sources, pages, exit pages, referrers, actions, countries, cities, devices, browsers, OS) including their timeframe-tab refetches (use-panel-range sends who, caches per who:range)
  • the Live panel (/api/projects/:id/live-events?who=), filtered on the bucket prefix

Under Humans or Bots a caption reads: "Split recorded from 5 Sep 2026; earlier traffic appears under All." Every label keeps its definition tooltip. The mapping lives once in lib/tracker/who.ts; the tracker-stats and live-events routes reject a junk who with 400 and default to humans when absent.

The kind column, and the "unknown before 5 Sep" caveat

Migration supabase/migrations/20260905190000_tracker_kind_split.sql (idempotent):

  • adds kind text not null default 'unknown' check (kind in ('human','bot','unknown')) to tracker_event_daily_stats, tracker_device_daily_stats, tracker_geo_daily_stats, tracker_exit_daily_stats, plus tracker_exit_sessions (so the exit marker is decremented off the row it actually sits on)
  • rebuilds each rollup's primary key with kind as the last column (the ingest route reads/writes by the full key, so the PK is the conflict target); guarded on whether the current PK already names kind
  • extends the two covering indexes' include lists with kind (page_path stays); new names created first, old ones dropped after, so coverage never lapses
  • ingest (app/api/track/route.ts) derives kind from the bucket once (kindFromBucket: bot iff bucket starts with bot:, AI referrals are human) and writes it into all four rollup upserts and the exit sessions

Rows written before this migration are kind = 'unknown'. Nothing in the old rollups can say what they were, so they are not guessed at: they appear only under All. Under Humans/Bots the pages / referrers / actions / exit pages / countries / cities / devices panels start on the day the migration is applied. The bucket-based figures (tiles, series, Top sources) and the raw tracker_events table have always carried the bucket, so those are split across all history.

RPC signature changes

Every panel RPC gains a trailing p_kind text default null (null = all rows; 'human' / 'bot' filter on kind for the rollups, on the bucket prefix for tracker_daily_stats / tracker_events). Each is drop function if exists <exact old signature> then create function with the old arguments first, so existing callers (the portfolio analytics page, MCP) keep working; grants re-issued for authenticated, service_role; security invoker throughout.

  • singles: tracker_daily_series, tracker_bucket_totals, tracker_event_mix, tracker_top_pages, tracker_top_referrers, tracker_top_actions, tracker_top_exit_pages, tracker_top_countries, tracker_top_cities, tracker_device_totals
  • _multi: the same ten
  • raw: tracker_recent_series, tracker_recent_bucket_totals, tracker_recent_event_mix, tracker_recent_top_pages, tracker_recent_top_referrers, tracker_recent_top_actions, tracker_recent_top_countries, tracker_recent_top_cities

set work_mem = '16MB' is preserved on tracker_top_pages_multi, tracker_top_actions_multi and tracker_top_exit_pages_multi (the drifted live definitions from tracker_reporting_indexes); a contract test pins that it is on exactly those three. Not touched: tracker_first_day, tracker_project_totals, tracker_project_daily_series, dashboard_project_traffic.

lib/tracker/series.ts's legacy events = pageviews + interactions backfill is now only applied to an unfiltered series, so a filtered zero day stays zero instead of borrowing the other side's rows.

Apply the migration via MCP BEFORE merge

The code passes p_kind on every RPC call, so without the migration every stats panel 404s on the missing signature. Apply 20260905190000_tracker_kind_split.sql by hand via the Supabase MCP first, then merge. Note the PK rebuild on tracker_event_daily_stats (~1.2M rows) takes an ACCESS EXCLUSIVE lock for the seconds the index build takes; ingest writes queue for that window. The check constraints are added not valid + validate so they never block writes.

Tests

  • tests/tracker-who.test.ts (new): who parsing / validation, who -> p_kind, kindFromBucket, tile and series-layer selection, panelUrl, p_kind on every panel at rollup and raw ranges, backfill guard
  • tests/contract/tracker-kind-split.test.ts (new): migration contract (kind columns + PKs, index includes, every RPC dropped by exact old signature and re-created with p_kind last, work_mem on exactly the three, never definer)
  • tests/contract/tracker-stats-who.test.ts (new): both API routes default / map / 400
  • tests/contract/tracker-exit.test.ts, tests/tracker-rollup-ranges.test.ts extended
tsc --noEmit: clean
vitest run: Test Files 156 passed | 1 skipped (157), Tests 2046 passed | 7 skipped (2053), 11.76s

🤖 Generated with Claude Code

https://claude.ai/code/session_01WJaXiqE9BDoNfoJBhfXroC

Every number on /dashboard/projects/:id/stats now follows one ?who=
value (humans by default): the headline tiles, the Traffic pulse series,
each breakdown card and the live panel. It maps onto a trailing
`p_kind text default null` on every tracker panel RPC (singles, _multi
and the tracker_recent_* raw twins), and onto a bucket filter on the
live-events route.

The bucket-less rollups (event / device / geo / exit) gain a `kind`
column in their primary key so a human and a bot hit on the same path
are two rows; ingest derives it from the bucket. Rows from before the
migration are `unknown` and appear only under All, which the toggle
says. The three portfolio RPCs that spill keep `set work_mem = '16MB'`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJaXiqE9BDoNfoJBhfXroC
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown

ThreatCrush Security Scan

39 finding(s)

HIGH/CRITICAL: 2 | MEDIUM: 28 | LOW: 9

Severity Rule Location
HIGH tls-verification-disabled lib/onion.ts:48
HIGH secret-generic-credential lib/sp/platforms/facebook.ts:32
MEDIUM js-unescaped-html-sink app/(app)/dashboard/admin/email-broadcast/EmailBroadcastForm.tsx:125
MEDIUM js-unescaped-html-sink app/(app)/dashboard/projects/[id]/autoblog/articles/[articleId]/page.tsx:214
MEDIUM js-unescaped-html-sink app/(marketing)/blog/[slug]/page.tsx:67
MEDIUM js-unescaped-html-sink app/(marketing)/blog/[slug]/page.tsx:97
MEDIUM js-unescaped-html-sink app/(marketing)/blog/[slug]/page.tsx:104
MEDIUM js-unescaped-html-sink app/(marketing)/blog/[slug]/page.tsx:110
MEDIUM js-unescaped-html-sink app/(marketing)/recent/page.tsx:186
MEDIUM js-unescaped-html-sink app/(marketing)/recent/page.tsx:190
MEDIUM js-unescaped-html-sink app/c/[project]/[slug]/page.tsx:77
MEDIUM js-unescaped-html-sink app/c/[project]/page.tsx:57
MEDIUM js-unescaped-html-sink app/careers.js/route.ts:228
MEDIUM js-unescaped-html-sink app/careers.js/route.ts:285
MEDIUM js-unescaped-html-sink app/layout.tsx:129
MEDIUM js-open-redirect app/login/form.tsx:39
MEDIUM js-unescaped-html-sink app/r/[token]/page.tsx:176
MEDIUM js-open-redirect app/signup/form.tsx:43
MEDIUM js-open-redirect components/billing/buy-credits-modal.tsx:98
MEDIUM js-unescaped-html-sink components/json-ld.tsx:8
MEDIUM js-unescaped-html-sink components/report/markdown-view.tsx:15
MEDIUM js-unescaped-html-sink lib/careers/page-templates.ts:198
MEDIUM redos-nested-quantifier lib/emailMarkdown.ts:41
MEDIUM redos-nested-quantifier lib/emailMarkdown.ts:324
MEDIUM redos-nested-quantifier lib/lx/articleGen.ts:99
MEDIUM redos-nested-quantifier lib/tracker/agent-gate.ts:61
MEDIUM sh-remote-script-execution prober/deploy/provision.sh:30
MEDIUM sql-template-interpolation scripts/detect-slot-themes.ts:31
MEDIUM sql-template-interpolation scripts/purge-constructed-keywords.ts:163
MEDIUM sql-template-interpolation scripts/purge-offniche-keywords.ts:124
LOW secret-generic-credential app/(marketing)/docs/autoblog-webhook/page.tsx:145
LOW secret-generic-credential lib/sp/platforms/linkedin.ts:25
LOW js-dynamic-code-execution tests/careers-page-templates.test.ts:21
LOW js-dynamic-code-execution tests/careers-widget-script.test.ts:19
LOW js-dynamic-code-execution tests/careers-widget-script.test.ts:69
LOW js-dynamic-code-execution tests/contract/ad-visitor-id.test.ts:51
LOW js-dynamic-code-execution tests/contract/ad-visitor-id.test.ts:52
LOW secret-generic-credential tests/contract/posthog-integration.test.ts:13
LOW secret-generic-credential tests/lead-campaign.test.ts:16

Snippets are redacted; ThreatCrush never prints matched credential material.

@ralyodio
ralyodio merged commit 7fd8fde into master Sep 5, 2026
10 checks passed
@ralyodio
ralyodio deleted the stats-who-toggle branch September 5, 2026 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant