Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 71 additions & 32 deletions .github/scripts/sync-artifacts-catalog.test.ts

Large diffs are not rendered by default.

168 changes: 50 additions & 118 deletions .github/scripts/sync-artifacts-catalog.ts

Large diffs are not rendered by default.

70 changes: 12 additions & 58 deletions .github/workflows/slim-release-published.yml
Original file line number Diff line number Diff line change
@@ -1,23 +1,7 @@
name: Slim Release Published

# supabase/slim-services sends this dispatch after it mints an immutable release
# `<service>-<upstream_version>-r<revision>`. The stack catalog
# (packages/stack/src/Artifacts.ts) is the single version table for the service; this workflow
# treats the dispatch as a trigger and reconciles the catalog against every committed
# (published, non-draft) `<service>-...-r<N>` release, opening or updating one pull request per
# hotfix and/or upgrade it finds (`.github/scripts/sync-artifacts-catalog.ts`'s `planSlimUpdates`).
#
# Plan and apply run from the same checkout of the default branch, in a single job: a re-run
# always recomputes from the latest develop, so a stale plan can never be applied, and a
# superseded PR's branch is rewritten in place (force-pushed) instead of racing a fresh one. A
# backlog republish of an older upstream version naturally plans nothing.
#
# The payload arrives with whatever authority holds the dispatch token, so it is treated as
# untrusted: fields are pattern- and shape-checked before use
# (`validateSlimReleasePublishedPayload`), and never interpolated directly into a `run:` script
# (only passed through `env:`). Release tag names come from an external API too, so every value
# `planSlimUpdates` emits is re-validated against the same anchored patterns before it can reach a
# branch name or PR title.
# Reconciles the stack catalog against supabase/slim-services releases. See
# docs/adr/0026-slim-artifact-mirrors.md, "Hotfix and upgrade pickup".

on:
repository_dispatch:
Expand Down Expand Up @@ -46,9 +30,7 @@ jobs:
ref: ${{ github.event.repository.default_branch }}
persist-credentials: false

# Installs the locked workspace dependencies the scripts need: `Artifacts.ts` imports
# `effect`, and `render-service-dockerfile.ts` imports `@supabase/stack/internal/artifacts`.
# Replaces the bare mise step the pre-single-table version of this workflow used.
# `Artifacts.ts` and `render-service-dockerfile.ts` import locked workspace dependencies.
- name: Setup
uses: ./.github/actions/setup
with:
Expand All @@ -58,13 +40,7 @@ jobs:
id: base
run: echo "sha=$(git rev-parse HEAD)" >>"$GITHUB_OUTPUT"

# Delegates to `validateSlimReleasePublishedPayload` (sync-artifacts-catalog.ts) so every
# field — including `upstream_version` and `release_version`, not just `service` and
# `revision` — is checked against an anchored charset before it is ever written to
# `$GITHUB_OUTPUT`, a branch name, or a PR title. A value that fails validation makes the
# script exit non-zero and print a single `::error ::…` line to stdout (the rejected value
# is JSON-escaped in that message, so it can never smuggle in a newline or workflow command),
# which this step re-echoes so it still surfaces as an annotation.
# Re-echoes a validate-payload failure so its `::error ::…` line surfaces as an annotation.
- name: Validate payload
id: validate
env:
Expand All @@ -81,17 +57,7 @@ jobs:
fi
echo "$output" >>"$GITHUB_OUTPUT"

# Reconciles the service against every committed slim-services release (`planSlimUpdates`),
# not just the release that triggered this dispatch — this run is idempotent and safe to
# receive out of order. Records go only to `--output` (never stdout): each line is
# field-separated with `\x1f` (see `updateLine`), which a later step reads directly. Any
# `::warning ::…` the planner emits (an ignored tag, say) prints to this step's own stdout
# instead, so it can never be mistaken for a malformed record.
#
# `--expect-release` guards against the releases API lagging behind this very dispatch: it
# requires `<service>-<release_version>` to be a listed tag before planning runs at all,
# re-listing a bounded number of times first (`waitForExpectedRelease`). Without it, a slow
# API would make this run plan without the release that triggered it, and pass quietly.
# `--expect-release` waits for the releases API to list the release behind this dispatch.
- name: Plan updates
id: plan
env:
Expand Down Expand Up @@ -134,18 +100,9 @@ jobs:
permission-contents: write
permission-pull-requests: write

# One branch per planned item, built fresh from the base commit recorded above (the default
# branch's head at the start of this run) — never from wherever a previous loop iteration
# left HEAD. Pins exactly the planned release (`--release`, never "highest at apply time"),
# so plan and pin agree by construction: a revision minted a second later arrives with its
# own dispatch, not by racing this one.
#
# The app token (contents + pull-requests write) never reaches third-party code or disk:
# `git push` takes it only as part of an explicit URL, computed once as `push_url`
# (overridable via `PUSH_REMOTE_URL`, the seam a dry run uses to target a local bare repo
# instead), and `gh` gets it inline per call. Every `bun`/`pnpm` command below — the sync
# script, the Dockerfile generator, the formatter — runs under `env -u APP_TOKEN` so a
# compromised transitive dependency of any of them (they import `effect`) can't read it.
# One branch per planned item, rebuilt from the recorded base commit, never from a previous
# iteration's HEAD; `--release` pins the planned release, never "highest at apply time".
# `bun`/`pnpm` run under `env -u APP_TOKEN`: only `git push` and `gh` ever see the app token.
- name: Apply planned updates
if: steps.plan.outputs.count != '0'
env:
Expand All @@ -162,9 +119,8 @@ jobs:
push_url="${PUSH_REMOTE_URL:-https://x-access-token:${APP_TOKEN}@github.com/${GITHUB_REPOSITORY}.git}"
manual_prefix="bun .github/scripts/sync-artifacts-catalog.ts --service ${SERVICE} --release"

# Read the records on fd 3, not stdin: every command the loop body runs (`bun`, `gh`,
# `git`) otherwise shares stdin with the `read`, and any of them consuming a byte of it
# would swallow the rest of the file's records.
# Reads records on fd 3, not stdin, so commands the loop runs (`bun`, `gh`, `git`) never
# consume bytes meant for `read`.
while IFS=$'\x1f' read -r -u 3 kind branch title release from; do
if [ -z "$kind" ] || [ -z "$branch" ] || [ -z "$title" ] || [ -z "$release" ] || [ -z "$from" ]; then
echo "::error ::Malformed plan-updates line: kind='${kind}' branch='${branch}' title='${title}' release='${release}' from='${from}'."
Expand Down Expand Up @@ -200,10 +156,8 @@ jobs:
else
action="pins"
fi
# Names the release this PR actually pins as the subject; the dispatch's triggering
# release (RELEASE_VERSION) can differ from it (a hotfix dispatch commonly also
# yields an unrelated upgrade on the same line), so it's only mentioned, not implied
# to be what changed.
# Names the release this PR actually pins, not necessarily RELEASE_VERSION: postgres
# can plan a hotfix on one line and an unrelated upgrade on another in one dispatch.
body="$(printf 'This %s %s from %s to %s.\n\nhttps://github.com/supabase/slim-services/releases/tag/%s-%s\n\nPlanned after supabase/slim-services published %s. This branch is rewritten from %s whenever a newer relevant release arrives.\n' \
"$action" "$SERVICE" "$from" "$release" "$SERVICE" "$release" "$RELEASE_VERSION" "$DEFAULT_BRANCH")"

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ import { describe, expect, test, vi } from "vitest";
import { renderDockerfile } from "./render-service-dockerfile.ts";

// A minimal fixture catalog covering every slim-capable alias, so `renderDockerfile`'s
// "every alias needs exactly one line" check is satisfied by default; each test only mutates
// what it's exercising. `vi.mock` factories are hoisted above every other top-level statement,
// so the fixture pins are built inline here rather than imported from an outer module.
// one-line-per-alias check is satisfied by default; each test only mutates what it's
// exercising. `vi.mock` factories are hoisted above every other top-level statement, so the
// fixture pins are built inline here rather than imported from an outer module.
vi.mock("@supabase/stack/internal/artifacts", () => {
const nativePin = { archive: "a".repeat(64), manifest: "b".repeat(64) };
const natives = { "darwin-arm64": nativePin, "linux-amd64": nativePin, "linux-arm64": nativePin };
Expand Down
3 changes: 1 addition & 2 deletions apps/cli/scripts/render-service-dockerfile.unit.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,7 @@ describe("renderDockerfile against the real catalog and Dockerfile", () => {
});

// The Go tree still `go:embed`s its own copy for a dependency that hasn't been removed yet;
// this is the single place that keeps the two copies in sync (folded from the former
// dockerfile-go-sync.unit.test.ts), until apps/cli-go is deleted.
// this is the single place that keeps the two copies in sync until apps/cli-go is deleted.
test("the Go tree's embedded Dockerfile is a byte copy of the TS-owned one", () => {
const goDockerfile = readFileSync(GO_DOCKERFILE_PATH, "utf8");
expect(goDockerfile).toBe(currentTsDockerfile);
Expand Down
11 changes: 3 additions & 8 deletions apps/cli/src/commands/start/services/vector.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -222,14 +222,9 @@ const VECTOR_HEALTHCHECK = {
} as const;

/**
* Creates `/etc/vector` (absent from Vector 0.58's images, both slim and upstream), writes the
* rendered `vector.yaml` via a `cat <<'EOF'` heredoc, waits on Logflare's `/health` (sinks would
* otherwise start too early), then `exec`s Vector so it stays PID 1. A TERM trap covers the wait
* so `docker stop` does not burn 10s if Logflare is still down; `-T 2` bounds each probe so a hung
* health endpoint can't defer the trap. Slim Vector ships BusyBox wget, so the wait uses
* `-q --spider` instead of GNU's `--no-verbose --tries`. Both images run as root, so `mkdir -p`
* needs no separate ownership handling, and `/var/lib/vector` (the `docker_logs` source's
* checkpoint `data_dir`) already exists in both.
* Vector 0.58's images (slim and upstream) have no `/etc/vector`, so the script creates it. The
* TERM trap keeps `docker stop` fast while Logflare's `/health` is still down, and `-T 2` bounds
* each probe so a hung endpoint can't defer the trap.
*/
export function buildVectorEntrypointScript(
vectorYaml: string,
Expand Down
129 changes: 99 additions & 30 deletions apps/cli/src/shared/services/services.shared.unit.test.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { describe, expect, it, test } from "@effect/vitest";
import { Effect, Redacted } from "effect";
import { FetchHttpClient } from "effect/unstable/http";
import { vi } from "vitest";
import { afterEach, beforeEach, vi } from "vitest";
import serviceImagesDockerfile from "./Dockerfile" with { type: "text" };
import { dockerfileServiceImageRaw } from "./dockerfile-images.ts";
import {
Expand All @@ -15,36 +15,40 @@ import {
renderServicesWarning,
} from "./services.shared.ts";

// Only `auth` is pinned in this fixture catalog, at the current Dockerfile-independent version
// `v2.197.0-r0`, with a realistic (non-placeholder) fixture digest built to the real
// `ArtifactPin`/`NativePin` shape from `@supabase/stack/internal/artifacts`. Every other service
// is deliberately absent, so `toSlimImage` falls through to the upstream image for them — the
// permanent state for a non-slim-capable alias (kong, `pg14`, the job images): the Dockerfile's
// slim-capable lines are generated from the catalog now, so they never disagree with it.
// `vi.mock` factories are hoisted above every other top-level statement, so the fixture is
// inlined rather than referencing an outer const.
vi.mock("@supabase/stack/internal/artifacts", () => {
const digest = "260e94edb8d402555791146fcf70b8e90efdc6a81877a04e5aa26f0f416a5dd7";
const nativePin = { archive: digest, manifest: digest };
return {
catalogPins: () => [
{
service: "auth",
sourceService: "auth",
pin: {
upstreamVersion: "v2.197.0",
revision: 0,
image: `ghcr.io/supabase/cli/auth:v2.197.0-r0@sha256:${digest}`,
natives: {
"darwin-arm64": nativePin,
"linux-amd64": nativePin,
"linux-arm64": nativePin,
},
},
// `catalogPins` defaults to an auth-only fixture; the real-catalog tests swap in the original.
const { mockCatalogPins } = vi.hoisted(() => ({ mockCatalogPins: vi.fn() }));

vi.mock("@supabase/stack/internal/artifacts", (importOriginal) =>
importOriginal<typeof import("@supabase/stack/internal/artifacts")>().then((actual) => ({
...actual,
catalogPins: mockCatalogPins,
})),
);

const { catalogPins: actualCatalogPins } = await vi.importActual<
typeof import("@supabase/stack/internal/artifacts")
>("@supabase/stack/internal/artifacts");

const FIXTURE_DIGEST = "260e94edb8d402555791146fcf70b8e90efdc6a81877a04e5aa26f0f416a5dd7";
const FIXTURE_NATIVE_PIN = { archive: FIXTURE_DIGEST, manifest: FIXTURE_DIGEST };
const FIXTURE_CATALOG_PINS = [
{
service: "auth",
sourceService: "auth",
pin: {
upstreamVersion: "v2.197.0",
revision: 0,
image: `ghcr.io/supabase/cli/auth:v2.197.0-r0@sha256:${FIXTURE_DIGEST}`,
natives: {
"darwin-arm64": FIXTURE_NATIVE_PIN,
"linux-amd64": FIXTURE_NATIVE_PIN,
"linux-arm64": FIXTURE_NATIVE_PIN,
},
],
};
});
},
},
];

mockCatalogPins.mockImplementation(() => FIXTURE_CATALOG_PINS);

const ACCESS_TOKEN = Redacted.make(`sbp_${"a".repeat(40)}`);
const PROJECT_REF = "abcdefghijklmnopqrst";
Expand Down Expand Up @@ -122,6 +126,71 @@ describe("services shared", () => {
expect(postgresImageForDbMajorVersion(14)).toBe(pg14);
});

describe("against the real slim-services catalog", () => {
beforeEach(() => {
mockCatalogPins.mockImplementation(actualCatalogPins);
});

afterEach(() => {
mockCatalogPins.mockImplementation(() => FIXTURE_CATALOG_PINS);
});

test("lists slim images with versions derived from the catalog's default pins", () => {
const expectedNames = [
"postgres",
"auth",
"postgrest",
"realtime",
"storage",
"edge-runtime",
"studio",
"pgmeta",
"analytics",
"pooler",
];
const rows = listLocalServiceVersions({ slim: true });

expect(rows.map((row) => row.name)).toEqual(
expectedNames.map((service) => `ghcr.io/supabase/cli/${service}`),
);

for (const row of rows) {
const service = row.name.replace("ghcr.io/supabase/cli/", "");
const defaultPin = actualCatalogPins().find(
(entry) => entry.sourceService === service && entry.isDefault,
);
expect(defaultPin).toBeDefined();
expect(row.local).toBe(defaultPin?.pin.upstreamVersion);
expect(row.remote).toBe("");
}
});

test("slim-translates a serviceVersions override to a non-default catalog pin", () => {
const nonDefaultPostgresPin = actualCatalogPins().find(
(entry) => entry.sourceService === "postgres" && !entry.isDefault,
);
if (nonDefaultPostgresPin === undefined) {
throw new Error("Expected the catalog to carry a non-default postgres pin.");
}
const version = nonDefaultPostgresPin.pin.upstreamVersion;

// The Dockerfile's `pg` stage pins the default line, not this one — establishing that the
// override below actually changes the resolved version instead of matching it by accident.
expect(dockerfileServiceImageRaw("pg").split(":").at(-1)).not.toBe(version);

expect(
listLocalServiceVersions({
slim: true,
serviceVersions: { postgres: version },
}),
).toContainEqual({
name: "ghcr.io/supabase/cli/postgres",
local: version,
remote: "",
});
});
});

test("slim-translates a version override that matches a catalog pin", () => {
expect(
listLocalServiceVersions({ slim: true, serviceVersions: { auth: "v2.197.0" } }),
Expand Down
39 changes: 7 additions & 32 deletions apps/cli/src/shared/services/slim-images.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,7 @@ import { Config, ConfigProvider, Effect, Option } from "effect";
const SLIM_IMAGES_ENV = "SUPABASE_USE_SLIM_IMAGES";
const SLIM_IMAGE_PREFIX = "ghcr.io/supabase/cli/";

/**
* Maps embedded-Dockerfile aliases onto the slim service catalog. Aliases with
* no slim build (kong, `pg14`, the `differ`/`migra`/`pgprove` job images) are
* absent and keep their docker.io reference. OrioleDB tags are excluded in
* `slimCatalogPin`.
*/
/** Dockerfile alias to slim service; kong, `pg14`, and the job images have no slim build. */
const SLIM_SERVICE_BY_ALIAS = {
pg: "postgres",
pg15: "postgres",
Expand Down Expand Up @@ -46,19 +41,12 @@ const V_PREFIXED_SERVICES: ReadonlySet<SlimServiceName> = new Set([
"pooler",
]);

/**
* Reads the ambient slim-image flag for callers without an explicit project value: always the
* process environment, never the active `ConfigProvider`. That lookup cannot fail.
*/
/** The slim-image flag from the process environment, never the active `ConfigProvider`. */
export const slimImagesEnabled = Effect.suspend(() =>
Config.option(Config.string(SLIM_IMAGES_ENV)).parse(ConfigProvider.fromEnv()),
).pipe(Effect.map(Option.exists((value) => value === "true" || value === "1")), Effect.orDie);

/**
* Catalog-normalized slim tag under `ghcr.io/supabase/cli/<service>`. The
* published slim catalog uses a `v` prefix for application services while
* postgres, studio, and vector retain their unprefixed tags.
*/
/** Catalog tag: `v`-prefixed for application services; postgres, studio, and vector stay bare. */
function slimTagForService(service: SlimServiceName, rawTag: string): string {
const tag = rawTag.trim();
if (V_PREFIXED_SERVICES.has(service)) {
Expand Down Expand Up @@ -100,12 +88,7 @@ export function slimCatalogPin(alias: string, image: string): SlimCatalogPin | u
return { service, version: slimTagForService(service, tag) };
}

/**
* Looks up the pinned catalog image (with its published `@sha256` digest) for
* `service` whose `upstreamVersion` equals `version`. Reads the same catalog
* `apps/cli`'s stack-independent clients use, keyed by the slim-services
* `sourceService` name (which matches this module's `SlimServiceName`).
*/
/** Keyed by the slim-services `sourceService`, which matches `SlimServiceName`. */
function catalogImageFor(service: SlimServiceName, upstreamVersion: string): string | undefined {
for (const entry of catalogPins()) {
if (entry.sourceService === service && entry.pin.upstreamVersion === upstreamVersion) {
Expand All @@ -116,17 +99,9 @@ function catalogImageFor(service: SlimServiceName, upstreamVersion: string): str
}

/**
* Resolves the catalog's pinned slim image (repository, release version and
* digest) whose `upstreamVersion` normalizes to `image`'s tag for `alias`.
* This owns tag normalization (`v`-prefixing, `tagPrefix`, vector's `-alpine`
* strip) via {@link slimCatalogPin}, so pins that differ only in prefix
* between the two registries (`supavisor`, `logflare`) still match. Returns `undefined` whenever
* no catalog pin matches `alias` and `image`'s tag — callers then keep the upstream (non-slim)
* image instead of guessing a slim tag. That covers more than "no slim build": an alias with no
* slim build at all (kong, `pg14`, the one-shot job images); an excluded tag on an alias that
* does have one (an OrioleDB `pg` tag, which {@link slimCatalogPin} always excludes); and a tag
* the catalog simply doesn't pin (an upstream version the catalog hasn't caught up to yet, or a
* hosted-project override from `supabase link` that doesn't match the pinned upstream version).
* Resolves the catalog's pinned slim image matching `alias`'s normalized tag (via
* {@link slimCatalogPin}), or `undefined` when no catalog pin matches — callers then keep the
* upstream image instead of guessing a slim tag.
*/
export function toSlimImage(alias: string, image: string): string | undefined {
const pin = slimCatalogPin(alias, image);
Expand Down
Loading
Loading