Knowledge sync - #19
Open
scibly wants to merge 6 commits into
Open
Conversation
Background work that outlives a request moves to Inngest, self-hosted rather than Inngest Cloud so the Docker deployment and the hosted app run one code path. Replaces the hand-rolled lease-and-chain cron pattern for anything new. - Serve route at /api/inngest, client and functions in apps/app/src/lib/inngest - heartbeat demo function (cron + event trigger, retries: 2) to prove wiring - inngest/inngest container in compose, on its own database on the existing Postgres, created by a one-shot that runs on every `up` - INNGEST_BASE_URL, INNGEST_EVENT_KEY, INNGEST_SIGNING_KEY required with no defaults; INNGEST_DEV switches signing explicitly - ADR 0004 records the decision and the rejected alternatives Closes #8 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Inngest foundation: self-hosted orchestration engine
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
The scheduled poll ran as a Vercel cron that grabbed a singleton lease row and then chained POSTs to itself, hop by hop, until every due connection had had its turn. The lease existed because two overlapping chains would double the provider quota and race the same watermark; the hops existed because one serverless invocation could not outlive the work. Inngest already solves both. A cron function lists the due connections and sends one event each; a per-connection function does the actual poll. Retries, the concurrency cap that keeps a provider from being hammered, and the failure hook are declarations on the function rather than logic in the route. Observable behaviour is unchanged: same 04:00 schedule, same selection order, same backoff tiers, same watermark semantics. Deleted with the mechanism they served: `IntegrationSyncLease`, the `/api/cron/sync-integrations` route and its guard, the `crons` entry in `vercel.json`, and `CRON_SECRET` — nothing reads it once the last cron route is gone. Functions are feature-owned and collected in `src/server/inngest.ts`, the composition root for background work, mirroring how `api/root.ts` collects tRPC routers. `src/lib/` may not import from `src/features/`, so the collection cannot live beside the client. Closes #10 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`refreshTokenEncrypted` and `tokenExpiresAt` were write-only: set once in the OAuth callback and read by nothing. Both were always NULL besides — Notion's `exchangeCode` returns neither, and Notion is the only provider in the registry. `BaseIntegrationProvider.refreshToken` was a stub whose entire behaviour was to throw, with no caller. Notion access tokens do not expire; they die when a user revokes access, and no refresh call fixes that. A failing poll already retries and then backs off, which is the right outcome for a connection that needs reconnecting by hand. A provider that genuinely needs a refresh path can add these back together with the code that reads them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
feat: move integration freshness sync onto Inngest
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
What
Knowledge Sync turns the reasoning buried in an org's merged GitHub PRs — descriptions, review threads with their diff hunks, linked issues — into cited documentation published as Notion sub-pages under a scibly-owned parent page. Every suggestion goes through a review queue where a maintainer edits or rejects it before anything publishes; nothing writes itself into the wiki.
Getting there also lands four pieces of infrastructure the rest of the product now sits on:
Self-hosted Inngest as the org-wide background orchestrator (ADR 0004), replacing the hand-rolled cron plus Postgres-lease pattern. The existing integration freshness sync migrates onto it and IntegrationSyncLease is deleted.
A GitHub App connection with short-lived installation tokens, a second credential shape alongside the existing encrypted OAuth tokens.
Write capability in the Notion provider, which was read-only until now.
Credit metering and plan gating for the new AI actions, on Business, Pro, and Internal.
Ten tickets (#8–#17) integrate on this branch and ship as one reviewed unit.
Why
Almost every engineering org depends on a handful of experts who never wrote anything down, and where docs do exist they're stale enough that nobody trusts them. The knowledge isn't actually missing — it's sitting in PR reviews that get read once and never again. Knowledge Sync harvests it continuously and keeps it current, and the reason the output is trustworthy is that every claim carries a link back to the PR it came from, so a reader can check the receipt instead of taking the doc's word for it.
Checklist
pnpm checkpasses (typecheck + lint + format)pnpm test:unitpassesapps/app/apps/webchanges, ranpnpm --filter <@scibly/app|@scibly/web> run format:check