CrawlProof Alerts — free ValueSERP email alerts (Phase 1)

[ralyodio]

Builds CrawlProof Alerts (PRD: docs/crawlproof-alerts-prd.md): free, near-realtime email alerts for anything Google can see, powered by ValueSERP, with the existing crawler used to confirm backlinks. Extends the product from a one-time audit into recurring engagement.

Core loop

email + category → magic-link (double opt-in) → scheduled ValueSERP poll (recency-filtered) → dedupe by canonical URL → only never-seen URLs email → backlink candidates crawl-confirmed before alerting.

Reuses the app's pg_cron → /api/cron/<job> → worker pattern, the audit crawler (fetchPage/attachRendered), Resend, and the profiles plan model.

Six fixes folded in over the v1 draft

1. Economics — per-account monthly SERP-call budget (consume_alert_serp_budget) as the real cost backstop, not just a 50-alert count. A maxed free user is 10–20× the $0.15 ceiling otherwise; F5Bot's free-API story doesn't transfer to paid SERP.
2. Cold-start — an alert's first poll seeds the dedupe set silently (no email/crawl); value on creation comes from the instant test run (promoted to P0).
3. Email batching — one digest per user, not one email per alert (50 alerts × daily ⇒ up to 50 emails/day otherwise).
4. Canonical dedupe — documented as normalized-URL (SERP gives no canonical tag; per-result crawl would be required).
5. Backlink framing — "newly-indexed pages that mention + link the domain," not a full backlink crawler.
6. Trust & safety — people-tracking templates (name/reputation/impersonation/legal) gated out of the launch picker pending T&S/GDPR.

What's here

• Migration supabase/migrations/20260704120000_alerts.sql: alerts, alert_seen_urls, alert_findings, profile budget columns, budget RPC, crawlproof-alert-checks pg_cron job + global kill-switch (cron_config.alerts_enabled).
• Engine lib/alerts/*: categories/compile, ValueSERP client, canonical dedupe, backlink confirm (+ JS-render fallback), poll+dedupe engine, batched digest, signed pause/unsubscribe tokens, limits/budgets, abuse validation, worker processing.
• Routes/UI: cron route, pause/unsubscribe handlers, server actions, /alerts dashboard, public /get-alerts signup, worker /alerts/check-user.
• Tests: 26 cases (dedupe, template compile, backlink anchor detection, abuse validation). App + worker tsc --noEmit clean.

Deploy config

Set VALUESERP_API_KEY, ALERTS_FROM (warmed subdomain); seed cron_config (site_url, cron_secret). alerts_enabled=false halts all SERP spend.

Still open (per PRD §11)

Paid pricing must clear hourly SERP COGS (~$108–$180/mo @ 250 alerts); ValueSERP plan limits; Legal/T&S before un-gating people-tracking. next lint is broken repo-wide in Next 16 (pre-existing), so lint wasn't run; typecheck + tests are the gate.

🤖 Generated with Claude Code

[github-actions]

vu1nz Security Review

0 finding(s) in PR #?

No security issues found.