Skip to content

feat: add Beeper as a third comms chat source (#5872) - #6649

Merged
atomantic merged 7 commits into
atomantic:mainfrom
tzioup:pr/beeper-upstream
Sep 9, 2026
Merged

feat: add Beeper as a third comms chat source (#5872)#6649
atomantic merged 7 commits into
atomantic:mainfrom
tzioup:pr/beeper-upstream

Conversation

@tzioup

@tzioup tzioup commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds Beeper as a third chat source in Comms, alongside iMessage and Signal, reading from the Beeper Desktop local API, as proposed in #5872. This is the proof of concept for your review: a Messages > Beeper page with conversation list, thread view and participants, a machine-local mirror behind a tracked migration, Tribe linking through network-scoped identities, and the Comms feature group you asked for in the issue thread. The existing iMessage and Signal readers are untouched. The feature is off by default; with it off, the tree behaves as it does today.

Three commits, layered so each can be read on its own:

  1. feat: bucket related instance features under a group toggle (14 files). The group toggle from your reply on Add Beeper as a third chat ingestion source in Comms, reading from its local Desktop API #5872. Independently mergeable.
  2. refactor: expose the shared seams the beeper feature hooks into (24 files). Small changes to existing infrastructure the feature reuses, plus catalog rows and one budget re-measure.
  3. feat: add a beeper desktop bridge as a third comms chat source (104 files). The feature itself.

This is large for one PR. If you would rather take it staged, the commit boundaries are the seams I would split on.

Scope against #5872

Delivered as proposed.

  • Optional instance feature beeper, default off, no detector, nav entry and Messages tab gated the same way as every other feature-tagged entry.
  • Two flags kept distinct: the instance feature governs navigation, settings.beeper.enabled governs ingestion. Turning the nav off does not stop the sweep.
  • Storage db-primary. Eight beeper_* tables behind tracked migrations: the six from the issue plus beeper_credentials (the vaulted token) and beeper_outbox (see the send section).
  • Never federates: no peer-subscribable kind, no dataSync category, no schema-version entry, no sync_sequence column, and the beeper data directory is absent from the media-library walk. A guard test with a planted-violation probe pins each of those.
  • Token vaulted with AES-256-GCM in Postgres; OAuth with dynamic client registration and PKCE S256, and pasting a token as a first-class alternative. No refresh grant exists, so expiry is stored beside the token and surfaced early.
  • Correctness from HTTP, promptness from the socket: an interval sweep runs unconditionally with cursor and rows committing in one transaction; the WebSocket has a 75-second silence watchdog and is relayed to the browser as invalidation only, never rows, only to Beeper subscribers.
  • tribe_identities keyed (kind, network, handle) for durable handles, with beeper_participants.tribe_person_id as the cache for counterparties that carry no durable identifier. Linking is idempotent and survives a resweep.
  • Lazy attachment mirror with a disk budget, an eviction guard, and serving from an authenticated route.

Added from your reply. A Comms feature group in Settings > Features. FaceTime Audio, iMessage, Signal, X and Stacker News join it alongside Beeper; the group has its own toggle and each member keeps an individual override. A group's flag defaults to on when unset, so an install that never stored group state resolves every member exactly as before, with no settings migration.

Two decisions the issue left to you, taken as proposed since your reply did not object. The Beeper account dependency stays behind the manual toggle (decision A). Full message bodies are mirrored rather than kept as a 160-character preview (decision B). Both are reversible in review; B is the one that changes the schema.

Beyond the issue's V1: send. The issue listed sending as out of scope and said I would raise it separately. A first cut of it is in this PR, and I want to be plain about why and how far it goes. It was scoped in the design that preceded the issue, and it is the piece that turns Tribe's "you have not spoken to X" into a message on the channel X actually replies on, without copy-pasting into another app. What exists today is the safe core, not the workflow:

  • You send by scrolling to an existing conversation, selecting it, and typing. There is no new-message flow (pick a recipient, new or existing, then type) and no search-for-recipient flow (find the person, choose the network, type). The new-conversation button is rendered disabled so the gap is visible. Both flows are small additions; they stopped here because the PR is large enough to file, test and review before building more on it.
  • Human-gated. Two routes, one records the approved text and one performs the POST. No scheduler, agent, voice tool or Chief-of-Staff tool reaches either, asserted structurally by a test. No AI drafting or review on the path. The Tribe heartbeat draft-to-send streamlining is likewise deferred.
  • Durable: the outbox row is written before the POST; Beeper has no idempotency key on send, so nothing retries automatically, ever; a runaway breaker trips on a burst of sends or three consecutive failures, and only a human clears it.
  • Gated by scope. Send needs the write scope on the token. Connect runs the OAuth flow from PortOS's own UI and Beeper's consent screen shows both scopes; grant read only and Beeper refuses every send with 403, mapped to a terminal error: the row fails in place and nothing goes out while reads keep working. That is the operator's switch for keeping send off entirely.

Where the value already sits is the read side: participants on most major networks can be linked to Tribe people, so PortOS ingests relationship traffic holistically instead of having blind spots wherever a conversation lives outside the readers it ships.

If you would rather review read-first as the issue framed it, send is isolated (outbox service, its two routes, the composer) and can move to its own PR.

Partial in this PR.

  • The Requests and Later system scopes and the Add scope control render disabled rather than missing, so the deferral is visible.
  • Thread search is not wired; the client wraps Beeper's search endpoints with tests but nothing calls them.

Deferred, on record rather than hidden.

  • Cross-source dedupe. Beeper also bridges iMessage and Signal, so an install running a built-in reader and Beeper for the same network sees those conversations twice. Documented, with the per-feature override as the way to pick one reader per network.
  • Writing-style analysis over message text, and an activity-timeline adapter reading the mirror alongside human_activity_events.
  • Replacing or deprecating the built-in readers.
  • Outbound attachments, reactions, read receipts, typing indicators, merged cross-network views, avatars, a transport label in the composer, and a circuit breaker for a sweep against a stalled Beeper Desktop.

Privacy model

  • The mirror holds full message bodies from every connected network and stays on this machine by construction; the never-federates guard test is the contract.
  • The socket relay broadcasts to the Beeper subscriber set only, never instance-wide, and its frames carry ids and kinds, never content.
  • Attachment bytes are served from an authenticated /api/ route with a row-level check, never a static mount.
  • The token value never reaches a response or a log line, asserted by a route test; status reports presence, expiry, provenance and granted scopes only.
  • baseUrl is loopback-only unless an explicit opt-in is set, re-checked on every read, since the token rides on every request.
  • No fixture contains a real handle, name, phone number or message body.

Test plan

Suites at the branch tip, all green:

Suite Result
client 11,098 passed, 8 skipped
server 41,962 passed, 55 skipped
db 331 passed

Guard tests worth reading first: server/services/sharing/beeperNeverFederates.test.js (planted-violation probe), server/services/beeperOutboxHumanGate.test.js (no non-human caller can reach send), the "token value never reaches a response or a log line" block in server/routes/beeper.test.js, and docs/features/product-surfaces.test.js (route inventory for the Comms section).

Live, against Beeper Desktop 4.3.89 on the build that includes the upstream merge:

  • Feature off hides the sidebar row and the Messages pill; the tab strip resolves from the page-nav manifest in order and gated; setting the feature back to Inherit restores it.
  • Comms group card with per-feature overrides in Settings > Features.
  • OAuth connect with consent landing on the UI origin; disconnect revokes at the authorization server.
  • Sweep across nine connected accounts with per-chat cursors, the list header's "Syncing N of 9 accounts" strip moving as it runs, roughly a thousand messages mirrored, then purged.
  • Realtime frames through the shipped transport after an in-session connect.
  • Sends through the outbox on several networks, each resolved by its confirmation frame; the first, checked against the read API, showed exactly one message.
  • Attachments served with reserved dimensions; Low priority rail control PATCH and revert; purge removes every mirrored row.
  • Tribe linking exercised by hand: title chip, search-first picker, create-and-link, change, unlink, and the Tribe page's linked-handles block.

Not verified live, listed in docs/features/beeper.md under "What is not verified live": HEAD content-length and byte fidelity of a mirrored attachment, the in-flight image placeholder in a real browser, the conversation-not-found warning path, and a durable identity claim surviving purge plus resweep by hand.

To try it: enable the Comms group and Beeper in Settings > Features, open /messages/beeper?settings=1, Connect (or paste a token), enable scheduled sync, Sync now.

Docs

docs/features/beeper.md is the deep dive (what it talks to, setup, the mirror, sending, privacy model, API surface, files). AGENTS.md documents the feature-group mechanism and its parity guarantee. Catalog rows in docs/API.md, docs/README.md, docs/STORAGE.md and docs/features/product-surfaces.md. No changelog file or version bump, per the release convention.

Closes #5872

Settings > Features gains a Comms group so FaceTime Audio, iMessage,
Signal, X and Stacker News can be switched on or off together, while
each one can still be overridden individually. A group's own flag
defaults to on when unset, so introducing a group never hides a feature
that was already enabled and needs no settings migration.

- Feature registry gains a group registry and a group tag on a feature
  descriptor; iMessage, Signal, X and Stacker News join FaceTime Audio
  in the group. X and Stacker News become instance features for the
  first time, default on, so nothing changes for an existing install.
- Feature resolution now checks the feature's own stored override, then
  the group flag, then the detector, then the default. Clearing an
  override back to inherit deletes the stored key instead of writing a
  third sentinel.
- New settings endpoint to update a group, plus validation for the group
  slice on the generic settings PUT.
- The iMessage, Signal, X and Stacker News nav commands are tagged with
  their feature so they gate the same way every other feature-tagged nav
  entry does.
- The Features tab renders a group card with its own toggle above the
  member features.
Small, non-Beeper-specific changes to existing infrastructure that the
Beeper source (next commit) reuses instead of duplicating, plus the
catalog and barrel entries a new feature has to register.

- serveLocalFile accepts a caller-declared content type, bounded to a
  declared set, for a content-addressed store whose filenames carry no
  extension. Two more risky MIME variants (application/xhtml+xml,
  text/xml) are treated as unsafe to inline.
- The peer media-library sync exports its directory list so a
  federation-boundary test can assert against it directly.
- Tribe exports its touchpoint auto-create so a caller resolving a
  person through a different axis can reuse the same dedupe-keyed
  insert.
- Routine paths, backup-exclude, data-manager category and storage-doc
  entries for a new on-disk store.
- A shared ConnectionStatusDot component, adopted by the existing
  agents World tab in place of its inline version.
- Docs catalog rows (API, README, product surfaces) and a route
  inventory guard for the Comms section.
- The static-import budget test keeps upstream's latest narrowed entry
  and is re-measured on this branch's final tree (99,168 instantiations,
  +144 over the previous measurement), with the same ~1.5k allowance
  restored above it.
Adds Beeper as a chat source alongside iMessage and Signal, reading
from the Beeper Desktop local API. WhatsApp, Discord, Telegram and any
other network Beeper bridges are mirrored into a machine-local store
and shown on a new Messages > Beeper page: conversation list, thread
view, participants, and Tribe linking. Off by default, gated by the
Comms feature group.

What it adds:

- OAuth connect (dynamic registration + PKCE) with pasting a token as an
  alternative; the credential is stored AES-256-GCM encrypted in
  Postgres and decrypted only at the moment of use.
- A local-API client using raw fetch rather than the published SDK,
  which lags the live API and lacks account status, login id and the
  bridges endpoint. Cursor pagination, send-safe retry defaults, typed
  error mapping.
- Eight beeper_* tables and their migration: accounts, credentials,
  conversations, messages, participants, attachment metadata, per-chat
  sync cursors and the outbox.
- A watermark-bounded ingestion sweep whose message rows, attachment
  references and cursor commit in one transaction, on a scheduler the
  user controls.
- A WebSocket transport with a silence watchdog, relayed to the browser
  as invalidation only: ids and kinds, never message content.
- The chat surface: rail, pinned grid, thread, composer, attachments,
  and a lazy attachment byte mirror with a disk budget and eviction
  guard, served from an authenticated route.
- Tribe identity linking for participants: durable handle claims in a
  network-scoped identities table, a person picker with search, a
  create form, and person-scoped views across the Tribe pages.
- A durable send outbox, reply-only to an existing thread: row before
  POST, no automatic retry ever, socket confirmation with a GET
  fallback, and a runaway breaker only a human clears. Sending is only
  possible when the token was granted the write scope.

Privacy model: the mirror holds full message bodies from every
connected network and stays on this machine by construction. No beeper
record kind is exposed to peer sync, no wire-schema or snapshot
category exists for it, and no beeper_* table carries the sequence
column federation is built on; the beeper data directory never enters
the media-library federation walk. All of this is pinned by a test.
The socket relay broadcasts only to Beeper subscribers, never
instance-wide, and its frames carry no content.

Send posture: one send path, and it is a human one. No scheduler,
agent, voice tool or Chief-of-Staff tool can reach the outbox or the
send endpoint, asserted structurally by a test. Beeper has no
idempotency key on send, so nothing retries a send automatically.

Not in this change (deferred): starting a new thread from the composer
(the button is present but disabled), searching across threads, a
read-receipts toggle, and a sync-now job model.
@atomantic
atomantic merged commit c16860b into atomantic:main Sep 9, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Beeper as a third chat ingestion source in Comms, reading from its local Desktop API

2 participants