feat: realtime inspect command by filipecabaco · Pull Request #6642 · supabase/cli

filipecabaco · 2026-09-15T22:16:37Z

Summary

PoC for a realtime inspect command that provides mechanisms for users and agents to run checks on realtime.

It support the same as our base inspector ( https://realtime-ng-0.supabase.co/ ) and also provides easier ways to check RLS policies against private channels.

github-actions

🤖 AI Review

Both independent reviews completed. After deduplication and code verification, 22 of 23 findings are confirmed and one is refuted. The confirmed issues include one credential-exposure risk, four major correctness/contract bugs, and several validation, reporting, telemetry, documentation, and test-coverage concerns.

Findings

Severity	Location	Category	Sources	Claim
🔴 CRITICAL	`apps/cli/src/commands/inspect/realtime/presence/presence.handler.ts:117`	`security`	codex	Machine-readable presence output can expose JWTs or Supabase keys embedded in presence metadata.
🟠 MAJOR	`apps/cli/src/commands/inspect/realtime/check/check.handler.ts:120`	`error-handling`	claude+codex	`check` replaces channel join failures with `RealtimeEndpointUnhealthyError`, breaking the documented error-code contract and losing the rejected/timed-out/closed distinction.
🟠 MAJOR	`apps/cli/src/commands/inspect/realtime/realtime.connection.ts:72`	`authentication`	codex	`--service-role` can select a publishable key and can incorrectly report that selected key as elevated.
🟠 MAJOR	`apps/cli/src/commands/inspect/realtime/realtime.connection.ts:188`	`target-resolution`	codex	`--project-ref` with an explicit API key still requires an authenticated Management API key lookup just to derive the URL.
🟠 MAJOR	`apps/cli/src/commands/inspect/realtime/listen/listen.handler.ts:176`	`machine-output`	codex	Bounded JSON output omits received frames from its sole stdout result.
🟡 MINOR	`apps/cli/src/commands/inspect/realtime/realtime.connection.ts:91`	`target-resolution`	claude	When an explicit URL is combined with a key resolved from the local project, the target is incorrectly labeled as the local stack at the explicit URL.
🟡 MINOR	`apps/cli/src/commands/inspect/realtime/listen/listen.handler.ts:64`	`validation`	claude+codex	JSON listen mode rejects a tail bounded by `--events` unless `--duration` is also supplied.
🟡 MINOR	`apps/cli/src/commands/inspect/realtime/listen/listen.handler.ts:201`	`correctness`	claude	The final frame count can exceed the number of frames actually consumed and disagree with `byCategory`.
🟡 MINOR	`apps/cli/src/commands/inspect/realtime/realtime.format.ts:116`	`ux`	claude	Default filtering is misleadingly attributed to an explicitly supplied `--categories` flag.
🟡 MINOR	`apps/cli/src/commands/inspect/realtime/realtime.prelude.ts:188`	`telemetry`	claude	Several declared flags are absent from the handler telemetry map, causing safe boolean and choice values to be recorded as redacted placeholders.
🟡 MINOR	`apps/cli/src/commands/inspect/realtime/listen/listen.handler.ts:87`	`validation`	claude+codex	Replay constraints are not enforced: public replay is accepted, non-positive limits pass through, and a limit without a replay start is silently ignored.
🟡 MINOR	`apps/cli/src/commands/inspect/realtime/realtime.flags.ts:74`	`documentation`	claude	Non-zero/default-choice Realtime flag defaults are absent from the generated-reference override table.
🟡 MINOR	`apps/cli/package.json:63`	`dependencies`	claude	The CLI build resolves and bundles two versions of `@<!---->supabase/realtime-js`.
🟡 MINOR	`apps/cli/src/commands/inspect/realtime/realtime.events.ts:69`	`correctness`	claude	Payload cleaning removes null and empty-string fields before `--full-payload` or machine output can render them.
🟡 MINOR	`apps/cli/src/commands/inspect/realtime/realtime.connection.ts:110`	`error-handling`	claude	Automatic target resolution does not fall back to the linked project when local configuration resolves without a usable API key.
🟡 MINOR	`apps/cli/src/commands/inspect/realtime/listen/listen.handler.ts:190`	`signal-handling`	claude	The handler's graceful signal arm races the global signal wrapper, so Ctrl-C can interrupt the command before it emits its summary and returns exit 130.
🟡 MINOR	`apps/cli/docs/inspect-realtime.md:49`	`test-coverage`	codex	No test exercises the real RealtimeClient websocket session, listener registration, authentication ordering, or cleanup lifecycle.
⚪ NIT	`apps/cli/src/commands/inspect/realtime/realtime.connection.ts:45`	`naming`	claude	New Realtime-only helpers and public flag types use misleading `legacy`/`Legacy` prefixes.
⚪ NIT	`apps/cli/src/auth/http-debug.layer.ts:30`	`documentation`	claude	The credential-query-key comment discusses only presigned object-store URLs after API-key and access-token parameters were added.
⚪ NIT	`apps/cli/src/command-internal/db-target-flags.ts:60`	`maintainability`	claude	The new value-consuming flag names are unsorted and separate the log-level explanation from the alphabetized general registry it precedes.
⚪ NIT	`apps/cli/src/commands/inspect/realtime/realtime.layers.ts:6`	`documentation`	claude	Realtime reuses a runtime layer whose documentation still says it is only for Storage gateway commands.
⚪ NIT	`apps/cli/docs/inspect-realtime.md:27`	`documentation`	codex	The developer guide uses direct Bun invocations instead of the repository's pnpm-owned scripts.

Refuted findings (kept for transparency, not posted as review comments)

apps/cli/src/commands/inspect/realtime/listen/listen.handler.ts:188 (correctness): An unbounded production tail can finish and be incorrectly reported as stopped by an --events limit.
Refuted: The scripted test uses a finite mock stream, but the production stream cannot end independently: its queue is ended only during scope cleanup, which occurs after the race has already completed. Thus the claimed unbounded production path is unreachable in the current implementation.

Stats

Claude findings: 17 · Codex findings: 9 · Confirmed: 22 · Refuted: 1 · Uncertain: 0

Models: claude-opus-5 + gpt-5.6-sol · Trigger: auto · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

github-actions · 2026-09-15T22:37:31Z

+  const resolved = yield* resolveRealtimeProject(request);
+  const effectiveUrl = url ?? resolved.url;
+  yield* legacyValidateRealtimeUrl(effectiveUrl);
+
+  return {
+    url: effectiveUrl,
+    apiKey: apiKey === undefined ? resolved.apiKey : Redacted.make(apiKey),
+    source: resolved.source,
+    projectRef: resolved.projectRef,
+    elevated: elevated || resolved.elevated,
+  } satisfies RealtimeTarget;


🟡 MINOR · target-resolution · source: claude

When an explicit URL is combined with a key resolved from the local project, the target is incorrectly labeled as the local stack at the explicit URL.

Evidence: realtime.connection.ts:92 uses the explicit URL, while lines 97-100 retain the resolved target's source and key. describeRealtimeTarget at lines 226-235 consequently renders a remote flag URL as local stack at ....

Suggested fix: Represent URL and credential provenance separately, or mark the effective target as flag-sourced when an explicit URL overrides the resolved URL.

github-actions · 2026-09-15T22:37:31Z

+        if (output.format === "json" && Option.isNone(prepared.duration)) {
+          return yield* new RealtimeInvalidOptionError({
+            message:
+              "listen needs --duration with --output-format json, which emits one object once the tail ends; use --output-format stream-json to stream frames as they arrive.",
+          });
+        }


🟡 MINOR · validation · source: claude+codex

JSON listen mode rejects a tail bounded by --events unless --duration is also supplied.

Evidence: listen.handler.ts:64 checks only prepared.duration, although lines 183-186 bound the stream with prepared.events.

Suggested fix: Reject JSON mode only when both duration and event limits are absent, and mention both alternatives in the error.

github-actions · 2026-09-15T22:37:31Z

+            const counts = yield* session.counts;
+            const summary = {
+              emitted: counts.emitted,
+              suppressed: counts.suppressed,
+              byCategory: yield* Ref.get(byCategory),
+            };
+
+            if (output.format === "text") {
+              if (prepared.postgres !== undefined && (summary.byCategory["postgres"] ?? 0) === 0) {
+                yield* output.warn(
+                  realtimeNoChangesHint({
+                    table: `${prepared.postgres.schema}.${prepared.postgres.table}`,
+                    filtered: prepared.postgres.filter !== undefined,
+                    elevated: connection.target.elevated,
+                    asUser: connection.userToken !== undefined,
+                  }),
+                );
+              }
+              yield* output.outro(`${realtimeSummaryLine(summary)} ${describeRealtimeStop(stop)}`);
+              return;
+            }
+
+            yield* output.success("Realtime tail complete.", {
+              channel: spec.channel,
+              url: spec.url,
+              source: connection.target.source,
+              stoppedBy: stop,
+              frames: summary.emitted,
+              suppressed: summary.suppressed,
+              byCategory: summary.byCategory,
+            });


🟡 MINOR · correctness · source: claude

The final frame count can exceed the number of frames actually consumed and disagree with byCategory.

Evidence: listen.handler.ts:203 takes emitted from session.counts, while lines 167-180 separately count consumed frames. realtime.session.ts:112 increments the session count whenever a selected frame is queued, before Stream.take limits consumption.

Suggested fix: Derive the emitted count from the handler-side category totals and use session counts only for suppressed frames.

github-actions · 2026-09-15T22:37:31Z

+    .join(", ");
+
+  const suppressed =
+    summary.suppressed > 0 ? ` (${summary.suppressed} filtered out by --categories)` : "";


🟡 MINOR · ux · source: claude

Default filtering is misleadingly attributed to an explicitly supplied --categories flag.

Evidence: realtime.format.ts:116 always says filtered out by --categories. The default set excludes channel and transport at realtime.events.ts:10, while realtime.session.ts:195-197 records channel subscription events that increment the suppressed count.

Suggested fix: Use neutral wording for default filtering, or mention --categories only when the user supplied it.

github-actions · 2026-09-15T22:37:31Z

+        if (!attempt.joined.ok) {
+          yield* report({ name: "join", ok: false, detail: attempt.joined.message });
+          return yield* new RealtimeEndpointUnhealthyError({
+            kind: "handshake_refused",
+            message: attempt.joined.message,
+          });
+        }


🟠 MAJOR · error-handling · source: claude+codex

check replaces channel join failures with RealtimeEndpointUnhealthyError, breaking the documented error-code contract and losing the rejected/timed-out/closed distinction.

Evidence: check.handler.ts:104 catches RealtimeJoinFailedError, retains only its message, and lines 122-125 fail with RealtimeEndpointUnhealthyError{kind:"handshake_refused"}. inspect-realtime.md:132-134 documents separate endpoint and join error codes.

Suggested fix: Retain the original RealtimeJoinFailedError and its reason after recording the failed step.

github-actions · 2026-09-15T22:37:32Z

+  const flagKey = flagSecret ?? Option.getOrUndefined(request.apiKey);
+  const envUrl = legacyEnvValue(URL_ENV_KEY);
+  const envKey = KEY_ENV_KEYS.map(legacyEnvValue).find((value) => value !== undefined);
+
+  const url = flagUrl ?? envUrl;
+  const apiKey = flagKey ?? envKey;
+  const elevated = flagSecret !== undefined;
+
+  if (url !== undefined && apiKey !== undefined) {
+    yield* legacyValidateRealtimeUrl(url);
+    return {
+      url,
+      apiKey: Redacted.make(apiKey),
+      source: flagUrl !== undefined && flagKey !== undefined ? "flag" : "env",
+      projectRef: undefined,
+      elevated,
+    } satisfies RealtimeTarget;
+  }
+
+  const resolved = yield* resolveRealtimeProject(request);
+  const effectiveUrl = url ?? resolved.url;
+  yield* legacyValidateRealtimeUrl(effectiveUrl);
+
+  return {
+    url: effectiveUrl,
+    apiKey: apiKey === undefined ? resolved.apiKey : Redacted.make(apiKey),
+    source: resolved.source,
+    projectRef: resolved.projectRef,
+    elevated: elevated || resolved.elevated,


🟠 MAJOR · authentication · source: codex

--service-role can select a publishable key and can incorrectly report that selected key as elevated.

Evidence: realtime.connection.ts:72 falls back to request.apiKey even when serviceRole is true. When project resolution is also needed, line 97 retains that explicit key while line 100 inherits resolved.elevated from the unused resolved secret key.

Suggested fix: When service-role mode is requested, require or resolve a secret/service-role key and derive elevated solely from the key actually selected.

github-actions · 2026-09-15T22:37:32Z

+const legacyLinkedRealtimeTarget = Effect.fnUntraced(function* (
+  projectRef: Option.Option<string>,
+  serviceRole: boolean,
+) {
+  const cliSettings = yield* CommandSettings;
+  const resolver = yield* ProjectRefResolver;
+  const ref = yield* resolver.resolve(projectRef);
+
+  const api = yield* (yield* CommandPlatformApiFactory).make;
+  const keys = extractServiceKeys(
+    yield* api.v1.getProjectApiKeys(serviceRole ? { ref, reveal: true } : { ref }).pipe(


🟠 MAJOR · target-resolution · source: codex

--project-ref with an explicit API key still requires an authenticated Management API key lookup just to derive the URL.

Evidence: realtime.connection.ts:91 resolves a complete project target whenever the URL is missing; lines 188-198 unconditionally construct the platform client and call getProjectApiKeys, although line 218 derives the URL directly from the resolved ref.

Suggested fix: Resolve URL and API key independently and derive a hosted URL directly when a project ref and key are already available.

github-actions · 2026-09-15T22:37:32Z

+                if (output.format === "text") {
+                  yield* output.raw(`${formatRealtimeLine(event, render)}\n`);
+                  return;
+                }
+                yield* output.event(realtimeFrameEvent(event, render));
+              });
+
+            const tail = Option.match(prepared.events, {
+              onNone: () => session.events,
+              onSome: (limit) => Stream.take(session.events, limit),
+            }).pipe(Stream.runForEach(emit));
+
+            const stop = yield* Effect.raceAll([
+              tail.pipe(Effect.as<LegacyListenStop>("events")),
+              processControl
+                .awaitSignal(["SIGINT", "SIGTERM"])
+                .pipe(Effect.as<LegacyListenStop>("interrupted")),
+              ...Option.match(prepared.duration, {
+                onNone: () => [],
+                onSome: (limit) => [
+                  Effect.sleep(limit).pipe(Effect.as<LegacyListenStop>("duration")),
+                ],
+              }),
+            ]);
+
+            const counts = yield* session.counts;
+            const summary = {
+              emitted: counts.emitted,
+              suppressed: counts.suppressed,
+              byCategory: yield* Ref.get(byCategory),
+            };
+
+            if (output.format === "text") {
+              if (prepared.postgres !== undefined && (summary.byCategory["postgres"] ?? 0) === 0) {
+                yield* output.warn(
+                  realtimeNoChangesHint({
+                    table: `${prepared.postgres.schema}.${prepared.postgres.table}`,
+                    filtered: prepared.postgres.filter !== undefined,
+                    elevated: connection.target.elevated,
+                    asUser: connection.userToken !== undefined,
+                  }),
+                );
+              }
+              yield* output.outro(`${realtimeSummaryLine(summary)} ${describeRealtimeStop(stop)}`);
+              return;
+            }
+
+            yield* output.success("Realtime tail complete.", {
+              channel: spec.channel,
+              url: spec.url,
+              source: connection.target.source,
+              stoppedBy: stop,
+              frames: summary.emitted,
+              suppressed: summary.suppressed,
+              byCategory: summary.byCategory,


🟠 MAJOR · machine-output · source: codex

Bounded JSON output omits received frames from its sole stdout result.

Evidence: listen.handler.ts:176-180 sends non-text frames through output.event, while lines 223-230 emit only summary fields. jsonOutputLayer writes events to stderr at output.layer.ts:423 and only success data to stdout at lines 455-456.

Suggested fix: Accumulate frames in bounded JSON mode and include them in the final result; continue immediate event emission only for stream-json.

github-actions · 2026-09-15T22:37:32Z

+The handler tests run against a scripted session rather than a socket
+(`RealtimeSessions`, see `realtime-session.service.ts`), so they need no
+stack, no project and no credentials.


🟡 MINOR · test-coverage · source: codex

No test exercises the real RealtimeClient websocket session, listener registration, authentication ordering, or cleanup lifecycle.

Evidence: inspect-realtime.md:49-51 states that handler tests use scripted sessions without a socket, and the Realtime directory contains no test targeting realtime.session.ts.

Suggested fix: Add a focused local websocket or stack-backed integration test covering open, authentication, join, frame receipt, and scoped cleanup.

github-actions · 2026-09-15T22:37:32Z

+bun src/main.ts inspect realtime check
+
+# the real artifact (bundles the Realtime SDK into a single binary)
+pnpm run build:binary
+./dist/supabase inspect realtime check
+```
+
+> `pnpm exec turbo run supabase#build` also builds the Go sidecar and
+> `@supabase/config`. Use `build:binary` when you only want the CLI binary.
+
+Tests:
+
+```sh
+cd apps/cli
+
+# everything for this family (61 unit + 57 integration, no network, no server)
+bun --bun vitest run --project unit --project integration src/commands/inspect/realtime


⚪ NIT · documentation · source: codex

The developer guide uses direct Bun invocations instead of the repository's pnpm-owned scripts.

Evidence: inspect-realtime.md:27 runs the source entrypoint with Bun and line 43 invokes Vitest directly. package.json provides dev, test:unit, and test:integration scripts, and trusted/CLAUDE.md says pnpm scripts are the source of truth.

Suggested fix: Rewrite the examples using pnpm dev and the relevant pnpm test scripts with path filters.

feat: realtime inspect command

152a577

filipecabaco requested a review from a team as a code owner September 15, 2026 22:16

filipecabaco added the do not merge Approve to apply; do not merge. label Sep 15, 2026

github-actions Bot reviewed Sep 15, 2026

View reviewed changes

Navigation Menu

feat: realtime inspect command - #6642

feat: realtime inspect command#6642
filipecabaco wants to merge 1 commit into
developfrom
feat/realtime-inspect

filipecabaco commented Sep 15, 2026

Uh oh!

github-actions Bot left a comment

Uh oh!

github-actions Bot Sep 15, 2026

Uh oh!

github-actions Bot Sep 15, 2026

Uh oh!

github-actions Bot Sep 15, 2026

Uh oh!

github-actions Bot Sep 15, 2026

Uh oh!

github-actions Bot Sep 15, 2026

Uh oh!

github-actions Bot Sep 15, 2026

Uh oh!

github-actions Bot Sep 15, 2026

Uh oh!

github-actions Bot Sep 15, 2026

Uh oh!

github-actions Bot Sep 15, 2026

Uh oh!

github-actions Bot Sep 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

filipecabaco commented Sep 15, 2026

Summary

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

🤖 AI Review

Findings

Stats

Uh oh!

github-actions Bot Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant