Skip to content

fix(stack): ship local stack service logs to Analytics through Vector - #6864

Closed
avallete wants to merge 1 commit into
developfrom
avallete/fervent-mclaren-e91807
Closed

avallete wants to merge 1 commit into
developfrom
avallete/fervent-mclaren-e91807

Conversation

@avallete

Copy link
Copy Markdown
Member

Summary

The experimental stack's Vector recipe launched Vector with its bundled upstream demo config whenever no configPath was given, which is always the case from the CLI. That config emits a fake syslog line every second to the console and leaves Vector's API disabled, so the recipe's /health readiness never passed in docker and nothing reached Analytics.

The recipe now writes a generated pipeline to <root>/<instanceId>/vector/vector.json in prepare:

  • Docker / Podman: a docker_logs source filtered to the stack's com.supabase.stack label, routed by a new com.supabase.service container label to per-service remaps ported from the legacy vector.yaml template (Kong omitted; the stack has none), and HTTP sinks to ${LOGFLARE_URL}/api/logs?source_name=… authenticated with ${LOGFLARE_PRIVATE_ACCESS_TOKEN}. Vector interpolates both from the env vars the recipe already set.
  • Native: an API-only pipeline. Native service output is only held in stack memory, so there is no source Vector can read; this is an explicit limitation.

Supporting changes:

  • ContainerSpec gains a service label (set by process recipes and the database) and an opt-in engineApi flag. The flag mounts the engine socket read-only at /var/run/docker.sock, or sets DOCKER_HOST to host.docker.internal for TCP or named-pipe endpoints. The endpoint comes from docker context inspect (which honors DOCKER_HOST) or podman info, and the Docker Desktop/Colima socket mapping mirrors the legacy CLI.
  • Podman without an active API socket fails Vector's launch with an actionable message instead of a bind-mount error.
  • experimental stack start SIDE_EFFECTS updated.

A new Vector.integration.test.ts runs the database, Analytics and Vector in docker and asserts that a database RAISE LOG marker lands in Logflare's postgres.logs source with LOG severity.

Reviewer notes

  • Vector still receives engine access when a custom configPath is supplied; nothing in the CLI passes one today.
  • TCP engine endpoints are assumed to be a local, plaintext daemon, as in the legacy CLI.

🤖 Generated with Claude Code

The stack's default Vector pipeline enabled the health API but forwarded
no service logs, so nothing reached Analytics.

With Docker or Podman the default pipeline now tails the stack's
labelled service containers through the engine socket and posts Auth,
REST, Realtime, Storage, Functions and database logs to their Logflare
sources. Containers carry a service label for routing, and Vector gets
read-only engine API access. Native Vector keeps forwarding nothing
because native service output is only held in stack memory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@avallete

Copy link
Copy Markdown
Member Author

Superseded by #6893, which persists the stack's service logs to files and ships them to Analytics directly from the stack host (no Vector in the path), catching up after Analytics has slept.

@avallete avallete closed this Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant