feat(items): deploy litellm with its own postgres and redis

ICONS.md

@@ -27,7 +27,7 @@ to storefront visitors — nominative use — and remains the property of its
 respective owner. Inclusion here is not an endorsement by those projects, and
 these files are **not** covered by this repository's `LICENSE`. See `NOTICE`.
 
-All 13 items carry an icon; none currently ship screenshots.
+All 14 items carry an icon; none currently ship screenshots.
 
 | Item | Source | Method | License note |
 |---|---|---|---|
@@ -36,6 +36,7 @@ All 13 items carry an icon; none currently ship screenshots.
 | `label-studio` | <https://github.com/HumanSignal.png?size=512> | GitHub org avatar | Label Studio maintainer (HumanSignal) GitHub org avatar; nominative use. |
 | `langflow` | <https://github.com/langflow-ai.png?size=512> | GitHub org avatar | Langflow mark via the langflow-ai GitHub org avatar; nominative use. |
 | `litellm` | <https://github.com/BerriAI.png?size=512> | GitHub org avatar | LiteLLM maintainer (BerriAI) GitHub org avatar; nominative use. |
+| `litellm-stack` | <https://github.com/BerriAI.png?size=512> | GitHub org avatar | Same BerriAI mark as `litellm`, copied byte-for-byte rather than re-fetched, since the item lists the same upstream software; nominative use. |
 | `meilisearch` | <https://github.com/meilisearch.png?size=512> | GitHub org avatar | Meilisearch mark via the meilisearch GitHub org avatar; nominative use. |
 | `mlflow` | <https://github.com/mlflow.png?size=512> | GitHub org avatar | MLflow mark via the mlflow GitHub org avatar; nominative use. |
 | `n8n` | <https://github.com/n8n-io.png?size=512> | GitHub org avatar | n8n mark via the n8n-io GitHub org avatar; nominative use. |

items/litellm-stack/blueprint.yaml

@@ -0,0 +1,39 @@
+specVersion: v1
+kind: BLUEPRINT
+metadata:
+  slug: litellm-stack
+  version: 1
+spec:
+  components:
+    # Producers first: map order is graph order, and reading the two nodes the
+    # wires come from before the node they arrive at is what makes the
+    # connections below resolvable on a single pass down the file.
+    db:
+      component: ./components/postgres.yaml
+      size: general.standard.small
+      connections: {}
+    cache:
+      component: ./components/redis.yaml
+      size: general.standard.small
+      connections: {}
+    web:
+      component: ./components/litellm.yaml
+      # The single-container `litellm` item runs on general.economy.nano because
+      # a stateless proxy is mostly idle sockets. This one is not that: with
+      # DATABASE_URL set, the first boot runs the Prisma schema setup, and every
+      # request afterwards carries a key lookup and a spend write.
+      size: general.standard.small
+      connections:
+        databaseUrl:
+          fromRole: db
+          fromOutput: dsn
+        redisUrl:
+          fromRole: cache
+          fromOutput: url
+  # Derived rather than authored. The merged USER inputs are exactly the install
+  # form this item wants — the database user and name, both defaulted, beside
+  # three secrets the platform mints — and an authored block would have to
+  # restate all five to add nothing. Neither wired input appears there, which is
+  # the point: a CONNECTION input is satisfied by its wire and never reaches the
+  # form. What is left is a form with no required field on it.
+  parameters: {}

items/litellm-stack/components/litellm.yaml

@@ -0,0 +1,177 @@
+specVersion: v1
+kind: COMPONENT
+metadata:
+  version: 1
+spec:
+  workload:
+    kind: SERVICE
+    source:
+      type: IMAGE
+      # Same pin as the single-container `litellm` item, and it is the tip of
+      # LiteLLM's stable track rather than merely a recent tag: the project
+      # publishes `vX.Y.Z-stable` releases beside its ordinary ones, and
+      # v1.83.14-stable.patch.3 is the highest of them.
+      ref: ghcr.io/berriai/litellm:v1.83.14-stable.patch.3
+    # No `command`. The image's ENTRYPOINT is `docker/prod_entrypoint.sh`, whose
+    # last line is `exec litellm "$@"`, and a Musher `command` overrides the
+    # image's CMD while inheriting its ENTRYPOINT — so every word written here
+    # arrives as a `litellm` CLI argument. The stock CMD (`--port 4000`) is
+    # already the one this component wants. See the `redisUrl` input below for
+    # what that argument-only surface costs.
+    endpoints:
+      primary:
+        containerPort: 4000
+        protocol: HTTP
+        visibility: PUBLIC
+    health:
+      readiness:
+        # /health/liveliness (LiteLLM's spelling) stays the gate even though this
+        # deployment now has the database that would make /health/readiness
+        # meaningful. That route is declared with
+        # `dependencies=[Depends(user_api_key_auth)]`, so once LITELLM_MASTER_KEY
+        # is set it answers 401 to a probe carrying no key, and the deployment
+        # would never come up. /health/liveliness carries no such dependency.
+        path: /health/liveliness
+        endpoint: primary
+        # Longer than the stateless item's 30s. With DATABASE_URL set, the CLI
+        # runs the Prisma schema setup *before* uvicorn binds the port
+        # (proxy_cli.run_server), so on a first deploy nothing answers on 4000
+        # until the migration chain has finished against an empty database.
+        initialDelaySeconds: 120
+        timeoutSeconds: 5
+        successThreshold: 1
+        failureThreshold: 10
+    envVars:
+      # Lets the proxy keep model definitions, and the config rows below, in the
+      # Postgres this blueprint deploys rather than in a config file this
+      # container has no way to receive. It is also what makes the Admin UI's
+      # "Add model" flow work at all.
+      - key: STORE_MODEL_IN_DB
+        value:
+          type: LITERAL
+          value: 'True'
+      # LITELLM_MODE defaults to "DEV", and "DEV" is the one value that makes
+      # the CLI call load_dotenv() on startup — reading a stray .env out of the
+      # working directory. Setting it leaves the platform as the only thing that
+      # can put settings into this container.
+      - key: LITELLM_MODE
+        value:
+          type: LITERAL
+          value: PRODUCTION
+      # DISABLE_SCHEMA_UPDATE is deliberately NOT set. Upstream sets it on proxy
+      # replicas because their Helm chart runs migrations in a separate job;
+      # this blueprint has no such job, so the proxy applying its own schema at
+      # startup is the only thing that creates the tables the database exists
+      # for.
+  contract:
+    inputs:
+      masterKey:
+        schema:
+          type: STRING
+          isSensitive: true
+        isRequired: true
+        suppliedBy: USER
+        ui:
+          label: Master key
+        target:
+          envVarKey: LITELLM_MASTER_KEY
+        generator:
+          byteLength: 32
+          encoding: ALPHANUMERIC
+        description: >-
+          Auto-generated proxy admin credential. It authenticates the admin API,
+          mints virtual keys, and is the Admin UI password. Copy it from the
+          deployment's Configuration tab.
+      # Distinct from the master key and far less forgiving. LITELLM_SALT_KEY
+      # encrypts the provider credentials the proxy stores in Postgres, so
+      # changing it after any credential has been saved leaves every stored
+      # credential undecryptable — upstream's own guidance is "set once, never
+      # change it". It is generated rather than typed for the same reason the
+      # master key is, and it exists in this item and not in the single-container
+      # `litellm` item because there is no encrypted-at-rest credential without a
+      # database to hold it.
+      saltKey:
+        schema:
+          type: STRING
+          isSensitive: true
+        isRequired: true
+        suppliedBy: USER
+        ui:
+          label: Salt key
+        target:
+          envVarKey: LITELLM_SALT_KEY
+        generator:
+          byteLength: 32
+          encoding: ALPHANUMERIC
+        description: >-
+          Auto-generated key encrypting provider API keys at rest in the
+          database. Rotating it makes every credential already stored
+          unreadable, so treat it as fixed for the life of the deployment.
+      # The wire that turns a stateless proxy into a re-billable one. Virtual
+      # keys, teams, budgets and the spend ledger are all rows; without this
+      # input none of the tables exist and the proxy is a router with a single
+      # shared credential.
+      #
+      # DATABASE_URL is the whole of the database configuration this container
+      # needs — the CLI reads it directly, appends its connection-pool query
+      # params, and hands it to Prisma. (LiteLLM will also assemble one from
+      # DATABASE_HOST/PORT/USER/PASSWORD/NAME, which is the same value spelled
+      # in five inputs instead of one wire.)
+      databaseUrl:
+        schema:
+          type: STRING
+          format: CONNECTION_STRING
+          semanticType: POSTGRES
+          isSensitive: true
+        isRequired: true
+        suppliedBy: CONNECTION
+        ui: null
+        target:
+          envVarKey: DATABASE_URL
+        description: >-
+          Postgres connection string the proxy stores virtual keys, teams,
+          budgets and spend in. Wired from the database node in this blueprint.
+      # WHAT THIS WIRE DOES, AND THE ONE THING IT DOES NOT.
+      #
+      # It does deliver the address. REDIS_URL is read by
+      # `_redis_kwargs_from_environment()`, which maps every `REDIS_<ARG>`
+      # variable onto a redis-py argument, and RedisCache drops a null host,
+      # port and password before calling `get_redis_client()` — so this URL
+      # alone is a complete connection, with no REDIS_HOST/REDIS_PORT beside it.
+      #
+      # It does not, on its own, switch Redis on. `redis_usage_cache` — the
+      # client the proxy shares tpm/rpm counters, spend and router state through
+      # — is assigned in exactly one place, `ProxyConfig._init_cache`, reached
+      # only from the `litellm_settings.cache is True` branch of `load_config`.
+      # No environment variable reaches that flag — not in this release, and not
+      # in the current one, which was read to check whether the gap had closed.
+      #
+      # And a config file cannot be shipped here: as the `command` note above
+      # records, everything this component can pass the container is a `litellm`
+      # CLI argument, so there is nowhere to write a `config.yaml` for `--config`
+      # to name.
+      #
+      # What closes the gap is the other wire. With STORE_MODEL_IN_DB set,
+      # `ProxyConfig.save_config` writes `litellm_settings` to the `config` table
+      # in Postgres and `_update_config_from_db` merges it back at every start,
+      # so a one-time POST of `{"litellm_settings": {"cache": true}}` to
+      # /config/update, authenticated with the master key, is durable and
+      # applies to every replica. `cache_params` is left empty on purpose: an
+      # empty block is what sends LiteLLM to the environment, and the
+      # environment is where this wire has already put the URL.
+      redisUrl:
+        schema:
+          type: STRING
+          format: CONNECTION_STRING
+          semanticType: REDIS
+        isRequired: true
+        suppliedBy: CONNECTION
+        ui: null
+        target:
+          envVarKey: REDIS_URL
+        description: >-
+          Redis connection URL backing the shared response cache, cross-replica
+          rate-limit counters and router state. Wired from the cache node in
+          this blueprint; enable caching once from the Admin UI or
+          /config/update and the setting persists in the database.
+    outputs: {}

items/litellm-stack/components/postgres.yaml

@@ -0,0 +1,147 @@
+specVersion: v1
+kind: COMPONENT
+metadata:
+  version: 1
+spec:
+  workload:
+    kind: SERVICE
+    source:
+      type: IMAGE
+      ref: postgres:17.10-alpine
+    endpoints:
+      primary:
+        containerPort: 5432
+        protocol: TCP
+        visibility: PRIVATE
+    envVars:
+      - key: PGDATA
+        value:
+          type: LITERAL
+          value: /var/lib/postgresql/data/pgdata
+    volumes:
+      data:
+        sizeGib: 10
+        mountPath: /var/lib/postgresql/data
+  contract:
+    inputs:
+      # Generated rather than typed, and the `dsn` output below is the reason.
+      # A `${{ params.<key> }}` token is substituted verbatim — the platform
+      # percent-encodes nothing — so a password carrying `@`, `/`, `:`, `#` or
+      # `?` composes a URI that parses wrong or not at all, and the failure
+      # surfaces inside Prisma minutes into a deploy rather than on the form
+      # that accepted it. ALPHANUMERIC draws from base62, which contains no
+      # character reserved in a URI, so the value is safe in the userinfo
+      # position by construction rather than by a rule the user has to obey.
+      #
+      # Nothing outside the mesh connects to this database, so there is no
+      # password for a person to choose: the proxy receives it over a wire, and
+      # the deployment's Configuration tab is where to read it for a manual
+      # `psql`. That leaves this item's install form asking for nothing at all.
+      postgresPassword:
+        schema:
+          type: STRING
+          isSensitive: true
+        isRequired: true
+        suppliedBy: USER
+        ui:
+          label: Database password
+        target:
+          envVarKey: POSTGRES_PASSWORD
+        generator:
+          byteLength: 32
+          encoding: ALPHANUMERIC
+        description: >-
+          Auto-generated password for this database's user. It reaches the proxy
+          inside the connection string below rather than by being typed twice.
+          Copy it from the deployment's Configuration tab if you want to connect
+          to the database by hand.
+      # Typed, and interpolated into `dsn` just as the password is — so the same
+      # percent-encoding hazard applies, with the difference that this one has a
+      # person on the other end of it. The pattern is the intersection that
+      # costs nothing: every character in it is URL-safe, and it is also exactly
+      # what Postgres accepts as an unquoted identifier, so it rejects no name a
+      # role could legally have.
+      postgresUser:
+        schema:
+          type: STRING
+          default: postgres
+          pattern: '^[A-Za-z_][A-Za-z0-9_]{0,62}$'
+        isRequired: false
+        suppliedBy: USER
+        ui:
+          label: Database user
+        target:
+          envVarKey: POSTGRES_USER
+      postgresDb:
+        schema:
+          type: STRING
+          default: app
+          pattern: '^[A-Za-z_][A-Za-z0-9_]{0,62}$'
+        isRequired: false
+        suppliedBy: USER
+        ui:
+          label: Database name
+        target:
+          envVarKey: POSTGRES_DB
+    outputs:
+      address:
+        schema:
+          type: STRING
+          semanticType: POSTGRES
+        description: >-
+          Mesh-internal `host:port` of the database endpoint (for example
+          `db-myapp.internal:5432`), resolvable only by workloads in the same
+          deployment. Wire it into a consuming component's CONNECTION input to
+          point that workload at this database.
+        valueFrom: DERIVED
+        value: null
+      # The whole URI rather than the address, for a consumer that takes one
+      # DSN and no separate credential settings. LiteLLM is that consumer:
+      # `DATABASE_URL` is the only database setting its container reads without
+      # a config file, and Prisma parses it as a URI.
+      #
+      # DECLARED, and DECLARED does not mean static. The value below is a
+      # template the platform interpolates once the node is placed:
+      # `${{ params.<key> }}` against this node's own resolved inputs,
+      # `${{ self.privateAddress.<endpoint> }}` against its own addressing. So
+      # the host and port that are unknowable at authoring time still arrive
+      # here — writing them is not the same as knowing them now.
+      #
+      # DERIVED is the narrower thing, and the reason `address` above is one and
+      # this is not: the platform assigns every DERIVED output the same value,
+      # the node's public URL if it has one and its private `host:port`
+      # otherwise. Neither is a connection string. `format` and `semanticType`
+      # do not close the gap — they are wiring-compatibility tags the publish
+      # gate reads to check a connection's two ends agree, and they synthesise
+      # nothing. An earlier revision of this file had that backwards, and the
+      # deployment it produced handed LiteLLM a `DATABASE_URL` of
+      # `db-litellm-stack.internal:5432`, which Prisma rejects with P1012
+      # before the proxy binds a port at all.
+      #
+      # Every token resolves from this node and nothing else — its own USER
+      # inputs, its own placement — which is what keeps the output legal:
+      # component §6.2 forbids an output that depends on a value the component
+      # received over an *inbound connection*, and none of these arrive that
+      # way. Blueprint §4.2's own worked example wires a Postgres node's
+      # `connectionString`, so a composed DSN is the shape the contract was
+      # written for.
+      #
+      # Sensitive because it embeds POSTGRES_PASSWORD. `address` is not, and
+      # that difference is the reason to keep both: a consumer that only needs
+      # to reach the server should take the address and not a copy of the
+      # password.
+      dsn:
+        schema:
+          type: STRING
+          format: CONNECTION_STRING
+          semanticType: POSTGRES
+          isSensitive: true
+        description: >-
+          Ready-to-use `postgresql://` URI for this database — the mesh-internal
+          host and port above, carrying the user, password and database name
+          this node was deployed with. Wire it into a consuming component's
+          CONNECTION input where that workload wants one connection string
+          rather than the parts.
+        valueFrom: DECLARED
+        value: >-
+          postgresql://${{ params.postgresUser }}:${{ params.postgresPassword }}@${{ self.privateAddress.primary }}/${{ params.postgresDb }}