Add opt-in anonymous PostHog telemetry - #705
Open
time-attack wants to merge 1 commit into
Open
Conversation
Telemetry is a no-op without POSTHOG_API_KEY, and the org-wide admin toggle (default off) is the only way to turn it on; TELEMETRY_DISABLED is a hard kill switch that can only disable. The toggle is re-checked durably on every flush, so opting out reaches every process within one flush cycle and drops anything still buffered. Events carry only enums, counts, durations, and the build SHA, tied to a random per-deployment UUID stored in Postgres; no content, names, or user identifiers ever leave the instance. Events request no GeoIP enrichment and no person profiles; docs/telemetry.md lists every event and property, and only explicit allowlists emit. Most events derive from existing sinks: a decorator over the audit log maps allowlisted actions, wrappers over the metrics sink and error log cover turn outcomes, the harness resolver reports harness/model usage, and one shared trigger runner covers cron, webhook, monitor, and keychain fires. A sweeper flushes batches every 30s and (bounded to 2s) on shutdown; transient failures requeue with exponential backoff, permanent rejections drop the batch, and the daily heartbeat dedupes through the durable instance row so deploys and multiple processes don't inflate it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Opt-in, anonymous product telemetry via PostHog, so we can learn which features matter without collecting anything sensitive. Off by default; a no-op unless the deployment sets
POSTHOG_API_KEYand an org admin turns on the new Usage telemetry toggle.Privacy design
telemetry_instance). No message content, file/skill names, user identifiers, channel/session ids, error messages, or org name — verified against raw stored PostHog payloads, not just by reading the code.TELEMETRY_DISABLED=trueis a hard kill switch that can only disable, never enable — the admin toggle is the single way on.$geoip_disableand$process_person_profile: false; docs note that the receiving project must enable PostHog's "Discard client IP data" (sender can't force it).docs/telemetry.mdlists every event and property; only explicit allowlists emit.How
No new dependency — a ~260-line module (
src/insights/telemetry.ts) POSTs to PostHog's batch API withfetch, flushing viacreateSweeperevery 30s and (bounded to 2s) on shutdown. Transient failures requeue with exponential backoff; permanent 4xx drops the batch. Most events come from existing sinks rather than new call sites:AuditLog.recordmaps an allowlisted subset of audit actions (skills, crons, webhooks, deploys, connectors, keychain, files, memory, projects, grants) plus an explicit admin-resource list →config_updatedMetricsSink/ErrorLog→turn_completed/memory_auto_captured/error_recordedharness_invoked(harness/model adoption)trigger_firedfor cron/webhook/monitor/keychain-ask/secret-dropinstance_heartbeatdeduped through the durable instance row so multiple processes and redeploys don't inflate itThe org toggle follows the existing org-flag pattern in
config-store(incl. a durable getter, mirroringgetInteractiveFastModeDurable) and the standardADMIN_RESOURCESboolean + admin-panel card. Also deletes a stale duplicate org-key list influshScopefound during review.Testing
test/telemetry.test.ts(11 tests): payload shape + privacy flags, no-PII assertion against a poisoned audit event, opt-out drops buffered events, durable confirm overrides a stale cache, backoff + permanent-drop, durable daily heartbeat, bounded shutdown flush.config_updateditself is correctly dropped because consent wasn't yet active), and raw stored payload inspection.Review: 8 independent fresh-context review passes (correctness, reuse, conventions, efficiency, simplification, altitude, removed-behavior, cross-file tracing); all confirmed findings addressed in this diff.
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.