Skip to content

fix(sync): drop durable calendar-list discovery failures - #2689

Closed
cursor[bot] wants to merge 1 commit into
mainfrom
cursor/posthog-error-investigation-d1a7
Closed

fix(sync): drop durable calendar-list discovery failures#2689
cursor[bot] wants to merge 1 commit into
mainfrom
cursor/posthog-error-investigation-d1a7

Conversation

@cursor

@cursor cursor Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Stops Google calendar-list durable 4xx refusals (e.g. "The user must be signed up for Google Calendar.") from burning the sync retry ladder and tripping self-heal alarms.

PostHog cluster (7d):

Root cause: google-calendar.adapter mapped every non-410 list failure to discoveryFailed, and syncCalendarList / dispatch treated that as transient → retryableTransient × 20 → failed → self-heal requeue loop.

Fix (mirrors event readFailed / watch watchFailed):

  • Classify Google calendarList errors: cursorExpired (410), transient (429/5xx/network + rate-limit 403 reasons), discoveryFailed (durable 4xx)
  • Enrich cause with HTTP status + Google reason for triage
  • dispatchSyncJob drops durable discoveryFailed, stamps lastReadFailure* on the calendarList resource, frees the coalescing key for later rediscovery/reconnect

Simplicity

Minimal path parallel to existing readFailed settlement — no new job failure class, no health-model redesign. Classification helpers stay local to the Google calendar adapter (same shape as event-reader / notifications).

Automated validation

  • bun test:sync -- packages/sync/src/providers/google/google-calendar.adapter.test.ts packages/sync/src/domain/sync-job-dispatch.service.db.test.ts packages/sync/src/domain/calendar-list-sync.service.db.test.ts → 48 pass
  • bun run lint → no new issues (existing repo warnings only)

Independent review

Self-reviewed against the prod log cause chain and the settled readFailed / watchFailed patterns. Confirmed rate-limit 403s remain transient.

Test plan

  • bun test:sync -- packages/sync/src/providers/google/google-calendar.adapter.test.ts packages/sync/src/domain/sync-job-dispatch.service.db.test.ts packages/sync/src/domain/calendar-list-sync.service.db.test.ts
  • bun run lint

Remaining risk / follow-up

  • After deploy, clear or revive exhausted failed job 6a76454e6a9f4b8649b55e41 (calendarListSync:6a6539741eaf145653f64365) so rediscovery can re-enqueue; self-heal budget is already exhausted for that row.
  • Backend GenericError.NotSure (“Not sure why error occurred. See logs”, 60 occ) from calendar list proxy remains opaque in PostHog because error() puts the useful kind on result, not Error.message — separate triage improvement, not required for this storm.
  • Resolve PostHog issues 019fd9ff-… / 019fdc65-… once volume stops post-deploy.
Open in Web View Automation 

Classify Google calendarList 4xx refusals as discoveryFailed and settle
them as drops instead of retryableTransient, matching the readFailed path.
Keeps 429/5xx/network and rate-limit 403s as transient retries.

Co-authored-by: Tyler Dane <tyler-dane@users.noreply.github.com>
@tyler-dane tyler-dane closed this Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants