diff --git a/packs/gridlight-1.1.2/README.md b/packs/gridlight-1.1.2/README.md deleted file mode 100644 index 0579cd02..00000000 --- a/packs/gridlight-1.1.2/README.md +++ /dev/null @@ -1,194 +0,0 @@ -# Gridlight AI Platform - -Gridlight is an on-premises AI platform that runs entirely inside your cluster — no -data leaves it. A single **gateway** (Rust/Axum, port 8080) is the only endpoint your -applications talk to; it orchestrates **agents** that self-register with it: LLM -inference and embeddings, plus optional specialists for image generation -(diffusers/SDXL), text-to-speech (Piper), and speech-to-text (faster-whisper). It -serves retrieval-augmented generation (RAG) over your own documents using a bundled -PostgreSQL, Qdrant vector store, and Neo4j knowledge graph. Deployed as a Palette -add-on, it gives you private chat, document Q&A, image, and voice from one URL. - -## Prerequisites - -- A **Gridlight license ID** (UUID) — obtain from . -- Kubernetes **>= 1.26** with a default **StorageClass** that provisions - `ReadWriteOnce` volumes (e.g. AWS EBS `gp3`). -- At least one **GPU node** with the **NVIDIA device plugin** for LLM and image - inference. On a single-GPU node, enable NVIDIA **GPU time-slicing** if you want the - LLM and image agents to share it; otherwise put them on separate GPU nodes (the - image agent's output is served back through the gateway, so it does not need to - share a node with the gateway). -- Outbound egress to HuggingFace for first-use model downloads (or pre-stage models on - the data volume for air-gapped installs). Container images come from the public - registry `public.ecr.aws/q1c5c5i2/gridlight` — no pull credentials required. - -## Parameters - -Set under `charts.gridlight-palette.*` in the pack values. Minimum to deploy: - -| Name | Description | Type | Default | Required | -| --- | --- | --- | --- | --- | -| `auth.licenseId` | Gridlight license UUID | String | - | Yes | -| `auth.postgresPassword` | PostgreSQL password | String | - | Yes | -| `auth.neo4jPassword` | Neo4j password | String | - | Yes | -| `auth.gatewayToken` | Bearer token clients send (change from `dev-token`) | String | `dev-token` | No | -| `models.llm.preset` | LLM model: `qwen-7b`, `mistral-7b`, `qwen-32b`, `mixtral-8x7b`, … | String | `qwen-7b` | No | -| `models.image.preset` | Image model: `juggernautxl`, `cogview3-plus`, `auraflow`, `cogview4` | String | `juggernautxl` | No | -| `models.video.preset` | Video model: `wan-2.1` (14B, **default** — needs ~48GB GPU / L40S/A100; runs on 16–24GB via CPU offload), `wan-2.2`, `cogvideox-5b`, `ltx-video`, `wan-2.1-1.3b` (fits 8–24GB), `auto` | String | `wan-2.1` | No | -| `gateway.service.type` | `ClusterIP` / `LoadBalancer` / `NodePort` | String | `ClusterIP` | No | -| `pvc.size` | Shared volume for models + data (raise for large LLMs) | String | `100Gi` | No | -| `imageAgent.deploy`, `sttAgent.deploy`, `voiceAgent.deploy`, `videoAgent.deploy` | Toggle each specialist agent on/off | Bool | `true` (video: `false`) | No | - -Every component (`llmAgent`, `embedAgent`, each specialist, `postgres`, `qdrant`, -`neo4j`) has a `deploy` flag, so you can run any subset — e.g. gateway + LLM + image -only. See `values.yaml` for the full reference. - -## Usage - -### 1. Add the pack to a cluster profile - -Create an [add-on cluster profile](https://docs.spectrocloud.com/profiles/cluster-profiles/create-cluster-profiles/create-addon-profile/), -search for the **Gridlight AI Platform** pack, and overwrite the default configuration -with your values. Minimal example: - -```yaml -charts: - gridlight-palette: - auth: - licenseId: "00000000-0000-0000-0000-000000000000" # your license UUID - postgresPassword: "" - neo4jPassword: "" - gatewayToken: "" # change from dev-token - models: - llm: - preset: "qwen-7b" - image: - preset: "juggernautxl" - gateway: - service: - type: LoadBalancer # expose the gateway; ClusterIP for in-cluster only - pvc: - size: "150Gi" -``` - -To run only a subset of capabilities, set the per-agent toggles, e.g. image off: - -```yaml -charts: - gridlight-palette: - imageAgent: - deploy: false -``` - -Deploy the profile to your cluster. The agents self-register with the gateway and -download their models on first use (the image agent preloads its model at startup). - -### 2. Use the platform - -Everything goes through the **gateway** with `Authorization: Bearer `. -Get the gateway address from the `gridlight-...-gateway` Service (the external -hostname when `gateway.service.type: LoadBalancer`). - -```bash -GW="http://:8080" -TOKEN="" - -# Chat / RAG answer -curl -s "$GW/neon" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{"question": "Summarize our onboarding policy"}' - -# Ingest a document, then ask about it via /neon -curl -s "$GW/upload-structured" -H "Authorization: Bearer $TOKEN" -F file=@handbook.pdf - -# Generate an image (no model field uses models.image.preset) -curl -s "$GW/image" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{"prompt": "a red fox in a snowy forest, photorealistic"}' - -# Text-to-speech (returns WAV audio inline as base64) -curl -s "$GW/voice" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{"text": "Hello from Gridlight", "voice": "en_US-amy-medium"}' -``` - -The `image_url` returned by `/image` is served back **through the gateway**, so it -works even when the image agent runs on a different node — no shared filesystem or -external object store is required. - -> [!CAUTION] -> Image generation is a long synchronous request (a cold model load + generation can -> take 60–90s). When exposing the gateway via a cloud load balancer, the pack sets a -> 600s idle timeout (`gateway.service.idleTimeoutSeconds`); keep it high enough that -> the LB does not cut generation requests. - -> [!CAUTION] -> The specialist Python agents run as non-root and write to the shared data volume; -> the pack sets an `fsGroup` so they can. If you override pod security contexts, retain -> a group that can write the volume. - -### Video generation (optional, GPU-heavy) - -Video is **off by default** (`videoAgent.deploy: false`) and needs its own GPU. The -default model is the production-quality 14B — set expectations before enabling it: - -| `models.video.preset` | Fits | Warm gen (5 s @ 480p) | Quality | Download | -|-----------------------|------|-----------------------|---------|----------| -| `wan-2.1-1.3b` | 8–24 GB | ~1–2 min | Low (warps on motion) | ~17 GB | -| `ltx-video` | 12–24 GB | ~30–60 s | Decent, fast | ~20 GB | -| `cogvideox-5b` | 16 GB+ (offload) | ~3–5 min | Good | ~22 GB | -| `wan-2.1` (14B, **default**) | **48 GB** ideal / 16–24 GB offload | ~3–5 min @ 48 GB · **~30 min** with offload | High | ~40–76 GB | -| `wan-2.2` (A14B) | 48 GB+ | ~5–8 min | Highest | ~55 GB | - -- **First generation is a cold start:** model download (17–76 GB) + GPU load (14B ≈ 8 min). - Pre-stage the model on the PVC to skip it. Warm generations only pay the time above. -- **Single-concurrency:** one clip at a time; concurrent requests get `429`. A client - disconnect does NOT cancel the in-flight job. -- **On a ≥48 GB GPU set `videoAgent.offloadMode: none`** — the default `model` swaps weights - and is ~6× slower; use `model` only on 16–24 GB GPUs. - -```yaml -charts: - gridlight-palette: - models: - video: - preset: "wan-2.1" # quality tier (see table) - defaultQuality: "720p" # default res/fps: proxy|draft|preview|720p|1080p|2k|4k - videoAgent: - deploy: true - offloadMode: "none" # auto | none | model | sequential - replicas: 1 # >1 for parallel gen — also set pvc.enabled=false -``` - -Clients can override per request with `quality_preset` or explicit -`width`/`height`/`fps`/`duration_seconds`/`steps`. - -## Upgrade - -- Initial community release (1.1.2). Keep the pack **name** and **display name** across - future versions so they upgrade in place rather than appearing as a new pack. -- Retain the shared data volume across upgrades so models are not re-downloaded. -- `videoAgent` is **experimental** and `deploy: false` by default; do not rely on it in - production in this release. - -## Security / CVE posture - -All six container images are built on the latest Debian `bookworm` base with security -updates applied at build time (`apt-get upgrade`), which clears every base-OS CVE that -has a released fix. - -The CVEs that remain are predominantly **upstream Debian packages with no available -fix** (marked `will_not_fix` or `affected`/awaiting-patch by Debian) and are present in -essentially every Debian-based image — for example `CVE-2023-45853` (zlib1g, -`will_not_fix`), `CVE-2025-7458` (libsqlite3-0), `CVE-2026-42496` / `CVE-2026-8376` -(perl-base), and `CVE-2019-1010022` (glibc, disputed by Debian/not rated critical). They -are accepted and tracked pending upstream fixes. The image- and video-generation agents -carry a larger dependency surface (PyTorch/CUDA/diffusers), which is inherent to ML -workloads. The speech-to-text and video agents install ffmpeg with -`--no-install-recommends` to exclude the Mesa GL/VA/Vulkan hardware-decode drivers -(unused for headless audio decoding / CUDA generation), removing their `will_not_fix` -CVEs. - -## References - -- Gridlight: -- Gridlight license server: -- Create an add-on cluster profile: diff --git a/packs/gridlight-1.1.2/charts/gridlight-palette-1.1.2.tgz b/packs/gridlight-1.1.2/charts/gridlight-palette-1.1.2.tgz deleted file mode 100644 index 0d876981..00000000 Binary files a/packs/gridlight-1.1.2/charts/gridlight-palette-1.1.2.tgz and /dev/null differ diff --git a/packs/gridlight-1.1.2/logo.png b/packs/gridlight-1.1.2/logo.png deleted file mode 100644 index 2631ae9f..00000000 Binary files a/packs/gridlight-1.1.2/logo.png and /dev/null differ diff --git a/packs/gridlight-1.1.2/pack.json b/packs/gridlight-1.1.2/pack.json deleted file mode 100644 index 7783364d..00000000 --- a/packs/gridlight-1.1.2/pack.json +++ /dev/null @@ -1,17 +0,0 @@ -{ - "addonType": "app-services", - "annotations": { - "source": "community", - "contributor": "Gridlight" - }, - "cloudTypes": ["all"], - "displayName": "Gridlight AI Platform", - "eol": "", - "group": "", - "kubeManifests": [], - "kubeVersion": ">=1.26.0-0", - "layer": "addon", - "name": "gridlight", - "version": "1.1.2", - "charts": ["charts/gridlight-palette-1.1.2.tgz"] -} diff --git a/packs/gridlight-1.1.2/schema.yaml b/packs/gridlight-1.1.2/schema.yaml deleted file mode 100644 index 575d6d98..00000000 --- a/packs/gridlight-1.1.2/schema.yaml +++ /dev/null @@ -1,25 +0,0 @@ -# Spectrocloud Palette schema — controls UI rendering of sensitive fields. -# Fields listed here are rendered as masked password inputs in the Palette UI. -# Keys are charts..; the chart name is gridlight-palette -# (Chart.yaml name), which is also the key these values live under in values.yaml. - -charts.gridlight-palette.auth.postgresPassword: - schema: '{{ format "${password}" | hints "PostgreSQL password for the gridlight user" }}' - -charts.gridlight-palette.auth.neo4jPassword: - schema: '{{ format "${password}" | hints "Neo4j password for the neo4j user" }}' - -charts.gridlight-palette.auth.gatewayToken: - schema: '{{ format "${password}" | hints "Bearer token clients send in Authorization header. Change from dev-token before production." }}' - -charts.gridlight-palette.auth.agentToken: - schema: '{{ format "${password}" | hints "Token agents use to self-register with the gateway. Change from dev-token before production." }}' - -charts.gridlight-palette.auth.licenseId: - schema: '{{ format "${input}" | hints "Gridlight license UUID from https://license.gridlight.ai" }}' - -charts.gridlight-palette.auth.signingSecret: - schema: '{{ format "${password}" | hints "Auto-acquired at startup. Only set when migrating a pre-existing deployment that already has a signing secret." }}' - -charts.gridlight-palette.auth.huggingFaceToken: - schema: '{{ format "${password}" | hints "HuggingFace read token for gated models. Leave blank if using only open models." }}' diff --git a/packs/gridlight-1.1.2/values.yaml b/packs/gridlight-1.1.2/values.yaml deleted file mode 100644 index 4bd785e0..00000000 --- a/packs/gridlight-1.1.2/values.yaml +++ /dev/null @@ -1,607 +0,0 @@ -# GRIDLIGHT Helm Chart — Default Values -# All values can be overridden at install time via --set or --values. -# Required fields (no default): auth.postgresPassword, auth.neo4jPassword. - -# --------------------------------------------------------------------------- -# Pack metadata — consumed by Spectrocloud Palette, not by the Helm chart itself. -# Images here must be publicly pullable (pack-central crane pull CI check). -# Points to AWS Public ECR; production deployments use global.imageRegistry below. -# --------------------------------------------------------------------------- -pack: - namespace: "gridlight" - content: - images: - - image: "public.ecr.aws/q1c5c5i2/gridlight/gateway:1.1.2" - - image: "public.ecr.aws/q1c5c5i2/gridlight/agent:1.1.2" - - image: "public.ecr.aws/q1c5c5i2/gridlight/agent-stt:1.1.2" - - image: "public.ecr.aws/q1c5c5i2/gridlight/agent-image:1.1.2" - - image: "public.ecr.aws/q1c5c5i2/gridlight/agent-voice:1.1.2" - - image: "public.ecr.aws/q1c5c5i2/gridlight/agent-video:1.1.2" - -# --------------------------------------------------------------------------- -# Chart values — passed to the gridlight-palette Helm chart by Palette. -# --------------------------------------------------------------------------- -charts: - gridlight-palette: - - # --------------------------------------------------------------------------- - # Global - # --------------------------------------------------------------------------- - global: - imageRegistry: "public.ecr.aws/q1c5c5i2/gridlight" - imageTag: "1.1.2" - imagePullPolicy: IfNotPresent - imagePullSecrets: [] - - # --------------------------------------------------------------------------- - # Authentication & Secrets - # --------------------------------------------------------------------------- - auth: - # REQUIRED — no default. Provide via --set or a sealed secret. - postgresPassword: "" - neo4jPassword: "" - # Gateway API token — all client requests must include: - # Authorization: Bearer - # IMPORTANT: change this from the default before deploying to production. - gatewayToken: "dev-token" - # Shared token used by Python agents to register with the gateway. - # IMPORTANT: change this from the default before deploying to production. - agentToken: "dev-token" - # Gridlight license ID — required. Obtain from https://license.gridlight.ai - licenseId: "" - # Signing secret — leave blank. The gateway acquires this automatically from - # the license server at first startup using the license ID above. Only set - # this if migrating a deployment that already has a signing secret. - signingSecret: "" - # HuggingFace token — required only if image/video agents need to download gated models. - # Create a read token at https://huggingface.co/settings/tokens - huggingFaceToken: "" - # ECR pull credentials. When set, the chart creates a regcred imagePullSecret. - # Supply the base64-encoded Docker config JSON for your ECR token: - # aws ecr get-login-password | base64 - # Leave blank — public ECR images require no pull credentials. - registryCredentials: "" - - # --------------------------------------------------------------------------- - # Model bootstrap (automatic download on first install) - # --------------------------------------------------------------------------- - # When enabled, an init container downloads models before agents start. - # Skipped if the file already exists (safe to re-run on upgrade). - # Disable for air-gapped environments — pre-stage models on the PVC manually. - bootstrap: - enabled: true - # Embedding model — fixed, not user-selectable (nomic-embed-text v1.5 Q4_0, ~274 MB) - embedModelUrl: "https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/resolve/main/nomic-embed-text-v1.5.Q4_0.gguf" - # LLM model URL is derived automatically from models.llm catalog — no longer set here. - - # --------------------------------------------------------------------------- - # Model Catalog - # --------------------------------------------------------------------------- - # models.llm.category — "general" or "coding" - # models.llm.preset — key from catalog. (see list below) - # models.image.preset — key from catalog (image agent reads IMAGE_MODEL_ID env var) - # models.video.preset — key from catalog (video agent reads DEFAULT_VIDEO_MODEL env var) - # - # To use a model not in the catalog, leave preset blank and set - # llmAgent.modelPath to the full path of a pre-staged GGUF file. - # - # All catalog entries use Apache 2.0 or MIT licenses only. - # --------------------------------------------------------------------------- - models: - llm: - category: "general" # general | coding - preset: "qwen-7b" # see catalog keys below - catalog: - general: - mistral-7b: - name: "Mistral 7B Instruct v0.3" - tier: "entry" - license: "Apache-2.0" - sizeGb: 4.4 - huggingface: "bartowski/Mistral-7B-Instruct-v0.3-GGUF" - filename: "Mistral-7B-Instruct-v0.3-Q4_K_M.gguf" - url: "https://huggingface.co/bartowski/Mistral-7B-Instruct-v0.3-GGUF/resolve/main/Mistral-7B-Instruct-v0.3-Q4_K_M.gguf" - qwen-7b: - name: "Qwen 2.5 7B Instruct" - tier: "entry" - license: "Apache-2.0" - sizeGb: 4.7 - huggingface: "bartowski/Qwen2.5-7B-Instruct-GGUF" - filename: "Qwen2.5-7B-Instruct-Q4_K_M.gguf" - url: "https://huggingface.co/bartowski/Qwen2.5-7B-Instruct-GGUF/resolve/main/Qwen2.5-7B-Instruct-Q4_K_M.gguf" - qwen-32b: - name: "Qwen 2.5 32B Instruct" - tier: "mid" - license: "Apache-2.0" - sizeGb: 19.9 - huggingface: "bartowski/Qwen2.5-32B-Instruct-GGUF" - filename: "Qwen2.5-32B-Instruct-Q4_K_M.gguf" - url: "https://huggingface.co/bartowski/Qwen2.5-32B-Instruct-GGUF/resolve/main/Qwen2.5-32B-Instruct-Q4_K_M.gguf" - mixtral-8x7b: - name: "Mixtral 8x7B Instruct v0.1" - tier: "mid" - license: "Apache-2.0" - sizeGb: 26.4 - huggingface: "TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF" - filename: "mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf" - url: "https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF/resolve/main/mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf" - mixtral-8x22b: - name: "Mixtral 8x22B Instruct v0.1" - tier: "large" - license: "Apache-2.0" - sizeGb: 67.8 - huggingface: "MaziyarPanahi/Mixtral-8x22B-Instruct-v0.1-GGUF" - filename: "Mixtral-8x22B-Instruct-v0.1.Q3_K_M.gguf" - url: "https://huggingface.co/MaziyarPanahi/Mixtral-8x22B-Instruct-v0.1-GGUF/resolve/main/Mixtral-8x22B-Instruct-v0.1.Q3_K_M.gguf" - deepseek-r1: - name: "DeepSeek R1 671B" - tier: "flagship" - license: "MIT" - sizeGb: 161 - shards: 9 - subdir: "DeepSeek-R1-Q2_K" - filenameBase: "DeepSeek-R1-Q2_K" - filename: "DeepSeek-R1-Q2_K-00001-of-00009.gguf" - huggingface: "unsloth/DeepSeek-R1-GGUF" - baseUrl: "https://huggingface.co/unsloth/DeepSeek-R1-GGUF/resolve/main/DeepSeek-R1-Q2_K" - coding: - qwen-coder-7b: - name: "Qwen 2.5 Coder 7B Instruct" - tier: "entry" - license: "Apache-2.0" - sizeGb: 4.7 - huggingface: "bartowski/Qwen2.5-Coder-7B-Instruct-GGUF" - filename: "Qwen2.5-Coder-7B-Instruct-Q4_K_M.gguf" - url: "https://huggingface.co/bartowski/Qwen2.5-Coder-7B-Instruct-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-Q4_K_M.gguf" - granite-8b: - name: "IBM Granite 8B Code Instruct 128K" - tier: "entry" - license: "Apache-2.0" - sizeGb: 4.6 - huggingface: "RichardErkhov/ibm-granite_-_granite-8b-code-instruct-128k-gguf" - filename: "granite-8b-code-instruct-128k.Q4_K_M.gguf" - url: "https://huggingface.co/RichardErkhov/ibm-granite_-_granite-8b-code-instruct-128k-gguf/resolve/main/granite-8b-code-instruct-128k.Q4_K_M.gguf" - qwen-coder-14b: - name: "Qwen 2.5 Coder 14B Instruct" - tier: "mid" - license: "Apache-2.0" - sizeGb: 14.9 - huggingface: "bartowski/Qwen2.5-Coder-14B-Instruct-GGUF" - filename: "Qwen2.5-Coder-14B-Instruct-Q8_0.gguf" - url: "https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/resolve/main/Qwen2.5-Coder-14B-Instruct-Q8_0.gguf" - qwen-coder-32b: - name: "Qwen 2.5 Coder 32B Instruct" - tier: "mid" - license: "Apache-2.0" - sizeGb: 19.9 - huggingface: "bartowski/Qwen2.5-Coder-32B-Instruct-GGUF" - filename: "Qwen2.5-Coder-32B-Instruct-Q4_K_M.gguf" - url: "https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Q4_K_M.gguf" - mixtral-8x22b: - name: "Mixtral 8x22B Instruct v0.1" - tier: "large" - license: "Apache-2.0" - sizeGb: 67.8 - huggingface: "MaziyarPanahi/Mixtral-8x22B-Instruct-v0.1-GGUF" - filename: "Mixtral-8x22B-Instruct-v0.1.Q3_K_M.gguf" - url: "https://huggingface.co/MaziyarPanahi/Mixtral-8x22B-Instruct-v0.1-GGUF/resolve/main/Mixtral-8x22B-Instruct-v0.1.Q3_K_M.gguf" - deepseek-r1: - name: "DeepSeek R1 671B" - tier: "flagship" - license: "MIT" - sizeGb: 161 - shards: 9 - subdir: "DeepSeek-R1-Q2_K" - filenameBase: "DeepSeek-R1-Q2_K" - filename: "DeepSeek-R1-Q2_K-00001-of-00009.gguf" - huggingface: "unsloth/DeepSeek-R1-GGUF" - baseUrl: "https://huggingface.co/unsloth/DeepSeek-R1-GGUF/resolve/main/DeepSeek-R1-Q2_K" - - # Image generation models — downloaded by image-agent via HuggingFace Hub. - # The selected preset's huggingface repo ID is passed as IMAGE_MODEL_ID env var. - image: - preset: "juggernautxl" # juggernautxl (default) | cogview3-plus | auraflow | cogview4 (large GPU) - catalog: - juggernautxl: - name: "JuggernautXL v9" - tier: "balanced" - license: "OpenRAIL-M" - sizeGb: 7 - huggingface: "RunDiffusion/Juggernaut-XL-v9" - cogview3-plus: - name: "CogView3-Plus 3B" - tier: "entry" - license: "Apache-2.0" - sizeGb: 26 - huggingface: "THUDM/CogView3-Plus-3B" - auraflow: - name: "AuraFlow v0.3" - tier: "mid" - license: "Apache-2.0" - sizeGb: 16.5 - huggingface: "fal/AuraFlow-v0.3" - cogview4: - name: "CogView4 6B (large GPU — needs >24GB VRAM / large PVC)" - tier: "large" - license: "Apache-2.0" - sizeGb: 31.1 - huggingface: "THUDM/CogView4-6B" - # hidream-i1 intentionally omitted: the image agent does not yet support - # HiDream-I1's multi-encoder setup (load_dit_pipeline rejects it), and at - # 17B/~47GB it doesn't fit common GPUs. Re-add once the agent supports it. - - # Video generation models — downloaded by video-agent via HuggingFace Hub. - # The preset key is passed to the agent as DEFAULT_VIDEO_MODEL. - # - # DEFAULT IS wan-2.1 (14B) — production-quality output. It needs a ~48GB GPU - # (L40S / A100) to run at full speed; it also fits 16–24GB GPUs (A10G/4090) via - # CPU offload, just slower. On smaller GPUs, switch preset to "wan-2.1-1.3b". - # Set preset: "auto" to let the agent pick by detected VRAM. - video: - preset: "wan-2.1" # wan-2.1(14B) | wan-2.1-1.3b | cogvideox-5b | ltx-video | auto - # Default output quality/resolution when a request omits it (gateway reads this - # as DEFAULT_VIDEO_QUALITY). Higher = sharper + slower + more VRAM. A per-request - # "quality_preset" (or explicit width/height/fps/steps) always overrides. - # proxy(512x320) draft(640x480) preview(768x512) 720p(1280x720) 1080p 2k 4k - defaultQuality: "draft" - catalog: - wan-2.1-1.3b: - name: "Wan 2.1 T2V 1.3B" - tier: "entry" - license: "Apache-2.0" - sizeGb: 17 - huggingface: "Wan-AI/Wan2.1-T2V-1.3B-Diffusers" - ltx-video: - name: "LTX-Video 0.9.7 Distilled" - tier: "fast" - license: "Apache-2.0" - sizeGb: 20 - huggingface: "Lightricks/LTX-Video-0.9.7-distilled" - cogvideox-5b: - name: "CogVideoX 5B" - tier: "mid" - license: "Apache-2.0" - sizeGb: 22 - huggingface: "THUDM/CogVideoX-5b" - wan-2.1: - name: "Wan 2.1 T2V 14B" - tier: "flagship" - license: "Apache-2.0" - sizeGb: 40 - huggingface: "Wan-AI/Wan2.1-T2V-14B-Diffusers" - wan-2.2: - name: "Wan 2.2 T2V A14B" - tier: "flagship" - license: "Apache-2.0" - sizeGb: 55 - huggingface: "Wan-AI/Wan2.2-T2V-A14B-Diffusers" - - # --------------------------------------------------------------------------- - # Data directory - # --------------------------------------------------------------------------- - # All services write to the same PVC mounted at dataDir. - # Override with GRIDLIGHT_DATA_DIR to use a host-path or different mount. - dataDir: "/gridlight-data" - - # Shared persistent volume claim for model files and data. - pvc: - enabled: true - storageClassName: "" # leave blank for cluster default - size: "100Gi" - accessMode: ReadWriteOnce - - # --------------------------------------------------------------------------- - # Gateway - # --------------------------------------------------------------------------- - gateway: - deploy: true - image: - repository: "{{ .Values.global.imageRegistry }}/gateway" - tag: "{{ .Values.global.imageTag }}" - pullPolicy: "{{ .Values.global.imagePullPolicy }}" - replicas: 1 - port: 8080 - service: - type: ClusterIP - port: 8080 - # Idle timeout (seconds) for cloud load balancers. Image/video generation are - # long synchronous requests; the default 60s cuts them off ("Load failed"). - # Applied to AWS ELBs via annotation; for other LBs set service.annotations. - idleTimeoutSeconds: 600 - # Extra Service annotations (e.g. NLB selection, ACM cert, other LB tuning). - annotations: {} - resources: - requests: - cpu: "500m" - memory: "512Mi" - limits: - cpu: "2" - memory: "1Gi" - nodeSelector: {} - tolerations: [] - affinity: {} - # TFLOPs this deployment claims from the license pool (0 = use full license allowance). - tflopRequested: 0 - # Stable license fingerprint. Leave blank to auto-derive a deterministic value from - # the Helm release (recommended) — this keeps one license allocation across pod - # restarts instead of registering phantom gateways. Set explicitly only to share a - # gateway identity across separate installs/clusters. - licenseFingerprint: "" - # License server URL. Override for air-gapped installs pointing at an on-prem server. - licenseServerUrl: "https://license.gridlight.ai" - # Gateway mode: "" = full (Neo4j + Qdrant), "lite" = SQLite-only (no Neo4j entity resolution). - mode: "" - # Comma-separated list of allowed CORS origins for browser clients. - # Leave blank to allow all origins (permissive — safe for VPN-only deployments). - # Set when exposing the gateway via Ingress to restrict to known origins. - # Example: "https://app.example.com,https://admin.example.com" - corsOrigins: "" - # Escape hatch for advanced gateway tuning. Any key/value pair here is - # injected as an env var on the gateway container. Useful for: - # QUERY_EXPANSION_ENABLED: "false" # disable LLM query expansion (reduces latency) - # RAG_INITIAL_CANDIDATES: "30" # more candidates = better recall, more compute - # NEO4J_ENTITY_RESOLUTION_TIMEOUT_MS: "500" # cap Neo4j latency impact - # LATENCY_LOG_BREAKDOWN: "true" # per-component latency logging - # INTERACTIONS_ENABLED: "true" # enable interaction logging for analytics - # RUST_LOG: "debug" # verbose logging for troubleshooting - env: {} - - # --------------------------------------------------------------------------- - # PostgreSQL (in-cluster) - # --------------------------------------------------------------------------- - # WARNING: Setting deploy: false disables the in-cluster PostgreSQL instance. - # PostgreSQL stores all conversation history, uploaded document metadata, - # and TFLOP allocation records. Without it, NO context is saved across - # sessions and the gateway will fail to start. Only set to false if you are - # providing an external PostgreSQL instance via gateway.env (DATABASE_URL). - postgres: - deploy: true - image: "postgres:16-bookworm" - port: 5432 - database: "gridlight" - user: "gridlight" - # Password comes from auth.postgresPassword via Secret - resources: - requests: - cpu: "250m" - memory: "512Mi" - limits: - cpu: "2" - memory: "2Gi" - storage: - size: "20Gi" - storageClassName: "" - - # --------------------------------------------------------------------------- - # Qdrant (in-cluster) - # --------------------------------------------------------------------------- - # WARNING: Setting deploy: false disables vector search (QDRANT_ENABLED and - # SEMANTIC_SEARCH_ENABLED are set to false automatically). RAG queries will - # fall back to keyword search only — semantic similarity matching is lost, - # which significantly reduces answer quality for long documents. - qdrant: - deploy: true - image: "qdrant/qdrant:v1.10.0" - grpcPort: 6334 - httpPort: 6333 - collection: "gridlight" - resources: - requests: - cpu: "250m" - memory: "1Gi" - limits: - cpu: "2" - memory: "4Gi" - storage: - size: "20Gi" - storageClassName: "" - - # --------------------------------------------------------------------------- - # Neo4j (in-cluster) - # --------------------------------------------------------------------------- - # WARNING: Setting deploy: false disables the knowledge graph (NEO4J_ENABLED - # is set to false automatically). Entity resolution, relationship-aware - # retrieval, and graph-hop context expansion are all disabled — the gateway - # falls back to GRIDLIGHT_MODE=lite (SQLite-only). Complex multi-hop queries - # and document relationship graphs will not be available. - neo4j: - deploy: true - image: "neo4j:5.18-community" - boltPort: 7687 - httpPort: 7474 - user: "neo4j" - # Password comes from auth.neo4jPassword via Secret - jvm: - heapInitial: "512m" - heapMax: "1G" - pagecache: "512m" - resources: - requests: - cpu: "500m" - memory: "2Gi" - limits: - cpu: "3" - memory: "4Gi" - storage: - size: "10Gi" - storageClassName: "" - - # --------------------------------------------------------------------------- - # LLM Agent (Rust, port 8001) - # --------------------------------------------------------------------------- - llmAgent: - deploy: true - image: - repository: "{{ .Values.global.imageRegistry }}/agent" - tag: "{{ .Values.global.imageTag }}" - pullPolicy: "{{ .Values.global.imagePullPolicy }}" - port: 8001 - # Override the model path used by llama-server. Leave empty to auto-derive - # from models.llm.catalog. Set this only when using a pre-staged model - # outside the catalog (also set bootstrap.enabled=false in that case). - modelPath: "" - # CPU-only mode: set to "cpu" to bypass GPU detection - # device: "" - resources: - requests: - cpu: "2" - memory: "8Gi" - limits: - cpu: "8" - memory: "16Gi" - nodeSelector: {} - tolerations: [] - affinity: {} - env: {} - - # --------------------------------------------------------------------------- - # Embed Agent (Rust, port 8002) - # --------------------------------------------------------------------------- - embedAgent: - deploy: true - image: - repository: "{{ .Values.global.imageRegistry }}/agent" - tag: "{{ .Values.global.imageTag }}" - pullPolicy: "{{ .Values.global.imagePullPolicy }}" - port: 8002 - # Path to the embedding .gguf model file inside the container. - # Defaults to the bootstrap download destination. - modelPath: "/gridlight-data/models/embeddings/nomic-embed-text-v1.5.Q4_0.gguf" - resources: - requests: - cpu: "1" - memory: "2Gi" - limits: - cpu: "4" - memory: "6Gi" - nodeSelector: {} - tolerations: [] - affinity: {} - env: {} - - # --------------------------------------------------------------------------- - # STT Agent (Python / faster-whisper, port 8004) - # --------------------------------------------------------------------------- - sttAgent: - deploy: true - image: - repository: "{{ .Values.global.imageRegistry }}/agent-stt" - tag: "{{ .Values.global.imageTag }}" - pullPolicy: "{{ .Values.global.imagePullPolicy }}" - port: 8004 - # Whisper model size: tiny (39MB), base (74MB), small (244MB), medium (769MB), large-v3 (1.5GB) - modelSize: "tiny" - # Quantization: auto, float16, int8 (int8 recommended for CPU) - computeType: "auto" - resources: - requests: - cpu: "500m" - memory: "512Mi" - limits: - cpu: "4" - memory: "4Gi" - nodeSelector: {} - tolerations: [] - affinity: {} - env: {} - - # --------------------------------------------------------------------------- - # Image Agent (Python / diffusers, port 8005) - # --------------------------------------------------------------------------- - imageAgent: - deploy: true - image: - repository: "{{ .Values.global.imageRegistry }}/agent-image" - tag: "{{ .Values.global.imageTag }}" - pullPolicy: "{{ .Values.global.imagePullPolicy }}" - port: 8005 - resources: - requests: - cpu: "2" - memory: "8Gi" - limits: - cpu: "8" - memory: "16Gi" - nodeSelector: {} - tolerations: [] - affinity: {} - env: {} - - # --------------------------------------------------------------------------- - # Voice Agent (Python / Piper TTS, port 8006) - # --------------------------------------------------------------------------- - voiceAgent: - deploy: true - image: - repository: "{{ .Values.global.imageRegistry }}/agent-voice" - tag: "{{ .Values.global.imageTag }}" - pullPolicy: "{{ .Values.global.imagePullPolicy }}" - port: 8006 - defaultVoice: "en_US-amy-medium" - resources: - requests: - cpu: "250m" - memory: "512Mi" - limits: - cpu: "2" - memory: "2Gi" - nodeSelector: {} - tolerations: [] - affinity: {} - env: {} - - # --------------------------------------------------------------------------- - # Video Agent (Python / diffusers, port 8007) - # --------------------------------------------------------------------------- - # OFF by default (needs its own GPU) but validated end-to-end. The default model - # (models.video.preset) is wan-2.1 (14B, production quality). - # - # GPU SIZING — this is the important one: - # • wan-2.1 (14B, DEFAULT): best on a ~48GB GPU (L40S / A100). It also runs on a - # 16–24GB GPU (A10G/4090) via CPU offload (offloadMode=model) — slower, and - # needs the larger host-memory limit below for the offloaded weights. - # • On smaller GPUs, set models.video.preset to "wan-2.1-1.3b" or "auto". - # The image bundles the fast plain-GET model downloader (Xet repos otherwise stall). - # The video agent is single-concurrency: it returns 429 while a generation is in - # flight, and a client disconnect does NOT cancel the in-flight job. For concurrent - # video, run multiple replicas. Object storage / async generation tracked in GRD-2354. - videoAgent: - deploy: false - # Replicas >1 enable parallel generation (the agent is single-concurrency and - # returns 429 while busy). Each replica needs its own model storage: set - # pvc.enabled=false (emptyDir per pod — re-downloads the model) or use a - # ReadWriteMany storage class. The default ReadWriteOnce PVC attaches to one pod. - replicas: 1 - image: - repository: "{{ .Values.global.imageRegistry }}/agent-video" - tag: "{{ .Values.global.imageTag }}" - pullPolicy: "{{ .Values.global.imagePullPolicy }}" - port: 8007 - # CPU offload for large models: auto | none | model | sequential. - # "model" keeps the 14B default within VRAM on 16–24GB GPUs; "none" is fastest - # on a 48GB GPU that fits the whole pipeline. - offloadMode: "model" - resources: - requests: - cpu: "2" - memory: "16Gi" - limits: - cpu: "8" - # 56Gi headroom for CPU-offloaded 14B weights; lower to 24Gi if pinning a - # small model (wan-2.1-1.3b) that stays resident on the GPU. - memory: "56Gi" - nvidia.com/gpu: "1" # video generation requires a GPU - nodeSelector: {} - tolerations: [] - affinity: {} - env: {} - - # --------------------------------------------------------------------------- - # Ingress - # --------------------------------------------------------------------------- - ingress: - enabled: false - # ingressClassName: "nginx" - host: "" - # tls: [] - annotations: {} diff --git a/packs/gridlight-1.1.3/README.md b/packs/gridlight-1.1.3/README.md deleted file mode 100644 index 2ac5e64e..00000000 --- a/packs/gridlight-1.1.3/README.md +++ /dev/null @@ -1,308 +0,0 @@ -# Gridlight AI Platform - -On-premises AI platform with LLM inference, semantic search, knowledge graphs, and specialist agents (STT, image generation, voice synthesis) — all running inside your Kubernetes cluster. No data leaves your infrastructure. - -## Prerequisites - -- Kubernetes **1.26+** -- Helm **3.10+** -- A node with at least **16 GB RAM** and **100 GB** available storage for entry-tier models; flagship LLM models require 500 GB+ -- GPU node recommended for inference performance (CPU-only is supported) -- A private container registry reachable from the cluster with the Gridlight images pushed (see [image-build-and-test.md](https://github.com/GRIDLIGHT-INC/gridlight/blob/main/docs/image-build-and-test.md)) -- An `imagePullSecret` or `auth.registryCredentials` (Docker config JSON) if the registry requires authentication - -## Parameters - -| **Parameter** | **Description** | **Type** | **Default** | **Required** | -|---|---|---|---|---| -| `auth.postgresPassword` | Password for the Gridlight PostgreSQL user | String | — | Yes | -| `auth.neo4jPassword` | Password for the Neo4j database user | String | — | Yes | -| `auth.licenseId` | Gridlight license UUID. Required for compute metering. Obtain from [license.gridlight.ai](https://license.gridlight.ai) | String | — | Yes | -| `auth.signingSecret` | License signing secret for heartbeat auth. Leave blank — auto-acquired from the license server at first startup. Only set when migrating a pre-existing deployment. | String | `""` | No | -| `auth.huggingFaceToken` | HuggingFace read token for gated models (e.g. Juggernaut XI). Only required if image or video agents need gated models. | String | `""` | No | -| `auth.gatewayToken` | Bearer token clients include in the `Authorization` header | String | `dev-token` | No | -| `auth.agentToken` | Shared token agents use to register with the gateway | String | `dev-token` | No | -| `auth.registryCredentials` | Docker config JSON for the image registry (base64-encoded). Chart creates the `regcred` secret automatically when set. | String | `""` | No | -| `gateway.tflopRequested` | TFLOPs claimed from the license pool. Set to `0` to use the full licensed allowance. | Integer | `0` | No | -| `gateway.licenseServerUrl` | License server URL. Override only for air-gapped installs pointing at an on-prem license server. | String | `https://license.gridlight.ai` | No | -| `gateway.corsOrigins` | Comma-separated allowed CORS origins for browser clients. Leave blank to allow all origins (permissive). | String | `""` | No | -| `global.imageRegistry` | Container registry prefix for all Gridlight images | String | ECR path | No | -| `global.imageTag` | Image tag for all Gridlight services | String | `1.1.3` | No | -| `global.imagePullPolicy` | Kubernetes image pull policy | String | `IfNotPresent` | No | -| `models.llm.category` | LLM category: `general` (RAG/chat) or `coding` (code generation) | String | `general` | No | -| `models.llm.preset` | LLM model preset key. General: `mistral-7b`, `qwen-7b`, `qwen-32b`, `mixtral-8x7b`, `mixtral-8x22b`, `deepseek-r1`. Coding: `qwen-coder-7b`, `granite-8b`, `qwen-coder-14b`, `qwen-coder-32b`, `mixtral-8x22b`, `deepseek-r1` | String | `qwen-7b` | No | -| `models.image.preset` | Image generation model: `cogview3-plus`, `auraflow`, `cogview4`, `hidream-i1` | String | `cogview3-plus` | No | -| `models.video.preset` | Video generation model: `wan-2.1` (14B, **default** — needs ~48GB GPU / L40S/A100; runs on 16–24GB via CPU offload), `wan-2.2` (A14B), `cogvideox-5b`, `ltx-video`, `wan-2.1-1.3b` (fits 8–24GB), `auto` (pick by VRAM) | String | `wan-2.1` | No | -| `videoAgent.offloadMode` | Large-model CPU offload: `auto`, `none`, `model`, `sequential`. `model` keeps the 14B default within VRAM on 16–24GB GPUs. | String | `model` | No | -| `bootstrap.enabled` | Auto-download LLM and embedding models on first install via init containers. Set `false` for air-gapped deployments — pre-stage models on the PVC. | Bool | `true` | No | -| `pvc.size` | Shared PVC size for models and data | String | `100Gi` | No | -| `pvc.storageClassName` | Kubernetes StorageClass for the PVC. Blank = cluster default. | String | `""` | No | -| `ingress.enabled` | Expose the gateway via an Ingress resource | Bool | `false` | No | -| `ingress.host` | Public hostname for the gateway | String | `""` | No | -| `llmAgent.resources` | CPU/memory requests and limits for the LLM agent pod | Object | see values.yaml | No | -| `llmAgent.nodeSelector` | Node selector for GPU scheduling | Object | `{}` | No | -| `llmAgent.tolerations` | Tolerations for GPU node taints | Array | `[]` | No | - -## Agent fleets - -Each agent type can be declared as a **list of instances** — run several LLMs (different -models), multiple image agents, etc., all behind the one gateway, which load-balances across -them. The list keys are `llmAgents`, `embedAgents`, `imageAgents`, `videoAgents`, `sttAgents`, -`voiceAgents`. When a list is non-empty it **replaces** the matching singular block -(`llmAgent`, …), which remains for backward compatibility: existing values files keep working -unchanged (the singular block is treated as a one-instance fleet named `agent`). - -Per-instance fields: `name` (required, DNS-safe), `deploy`, `replicas`, `model` -(`{category,preset}` or `modelPath`), `storage` (`{mode,size,accessMode,storageClassName}`), -`nodeSelector`, `tolerations`, `affinity`, `resources`, `env`, `port`, and type-specific -`device` (LLM), `fsGroup`/`offloadMode` (image/video), `modelSize`/`computeType` (STT), -`defaultVoice` (voice). - -**Storage modes** (`storage.mode`): `shared` (the single shared PVC — default, backward -compatible), `ownPvc` (a dedicated PVC per instance so it can run on its own node pool), or -`emptyDir` (per-pod ephemeral, small models only). With `ownPvc` + `replicas > 1` the chart -emits a StatefulSet with `volumeClaimTemplates`; otherwise a Deployment. Switching an agent's -`mode` on an existing release is a reinstall, not an in-place upgrade (the workload selector -and kind are immutable). - -A complete worked example is in [`examples/fleet-heterogeneous.yaml`](examples/fleet-heterogeneous.yaml) -(three LLMs across GPU pools, two image agents, and the embed agent isolated onto a CPU pool). - -### Multiple LLM models on SpectroCloud Palette - -To run several LLMs (each a different model) behind one gateway, add an `llmAgents` list to the -**pack values** (the cluster-profile layer for the `gridlight-palette` pack). It is edited as -**raw YAML** — only the `auth.*` fields render as masked inputs in the Palette form; the fleet -lists do not, so type them directly into the pack values editor. - -```yaml -llmAgents: - - name: qwen # required, DNS-safe ^[a-z0-9]([a-z0-9-]*[a-z0-9])?$ - model: { preset: qwen-7b } - storage: { mode: ownPvc, size: 20Gi } - resources: { limits: { nvidia.com/gpu: "1" } } - nodeSelector: { gridlight.ai/pool: gpu-a10g } - - name: big - model: { preset: qwen-32b } - storage: { mode: ownPvc, size: 40Gi } - resources: { limits: { nvidia.com/gpu: "1" } } - nodeSelector: { gridlight.ai/pool: gpu-a100 } -``` - -LLM `preset` values: `mistral-7b`, `qwen-7b`, `qwen-32b`, `mixtral-8x7b`, `mixtral-8x22b`, -`deepseek-r1` (general) and `qwen-coder-7b/14b/32b`, `granite-8b` (coding); or a custom -`model: { modelPath: /data/models/.gguf }`. Use `storage.mode: ownPvc` for a fleet so -each model gets its own volume and can land on its own node pool. - -**Cluster prerequisites — the chart targets node pools, it does not create them:** -1. **Label the node pools** the `nodeSelector`s reference, e.g. - `kubectl label node gridlight.ai/pool=gpu-a10g`. -2. Install the **NVIDIA GPU Operator** so GPU nodes advertise `nvidia.com/gpu`. -3. For `ownPvc` across nodes, have a **StorageClass** that provisions per-node volumes - (EBS `gp3` on EKS). Set `storage.storageClassName` if it is not the cluster default. - -The gateway load-balances across all registered LLM instances automatically. To target a -specific model per request instead, see **Query routing** below. - -## Query routing - -Added in 1.1.3 (gateway image; no chart templates required). - -- **Explicit override (always on):** a request may name a `model` and/or pin an `agent_id` to - route deterministically — e.g. `POST /neon {"question":"…","model":"qwen-32b"}` hits the - `big` agent above. `model_required:true` makes an unavailable target return 404/503 instead - of falling back. Works on `/neon`, `/chat/intelligent`, `/image`, `/video`. No config needed. -- **Intent auto-routing (opt-in, ships dark):** the gateway can infer modality - (text/image/video) from a query and dispatch automatically. Enable per-install with the - gateway env escape hatch: - - ```yaml - gateway: - env: - GATEWAY_ROUTING_MODE: "auto" # default: explicit (off). Fails safe to text. - ``` - - Optional tuning (also via `gateway.env`): `MODALITY_LLM_TIMEOUT_SECS` (3), - `AUTO_ROUTE_MEDIA_TIMEOUT_SECS` (180), `AUTO_ROUTE_MAX_CONCURRENT` (4). - -## Upgrade - -- **1.1.2 → 1.1.3:** gateway query routing (above). Pure gateway-image change — bump the pack to - 1.1.3 (or `--set global.imageTag=1.1.3`) and existing values keep working; explicit override is - immediately available, auto-routing stays off until you set `gateway.env.GATEWAY_ROUTING_MODE=auto`. - The agent images are byte-identical to 1.1.2. To roll back, pin `global.imageTag=1.1.2`. - -- If upgrading from a pre-1.1.2 release, the new `auth.licenseId` field is now required. Provide it via `--set auth.licenseId=` or a values override file. The `auth.signingSecret` is acquired automatically at startup — you do not need to set it manually. -- When upgrading from a version prior to 1.1.1, delete and recreate the `gridlight-credentials` secret before upgrading — the secret key names changed. -- The `bootstrap.llmModelUrl` field was removed in 1.1.1 and replaced by the full `models.llm.catalog` system. If you had a custom model URL, set `llmAgent.modelPath` to the absolute path of the GGUF file on your PVC instead. -- PVC data (downloaded models, ingested documents) is preserved across upgrades. The chart uses `strategy: Recreate` on all agent deployments, so pods terminate cleanly before the new version starts. - -## Usage - -### Minimal install - -```yaml -auth: - postgresPassword: "changeme" - neo4jPassword: "changeme" -``` - -```bash -helm install gridlight oci://registry.spectrocloud.com/community/gridlight \ - --version 1.1.3 \ - --values my-values.yaml \ - --namespace gridlight --create-namespace -``` - -### Selecting a model - -Use the Palette UI dropdowns (driven by `values.schema.json`) or set via values: - -```yaml -models: - llm: - category: "general" - preset: "qwen-32b" # ~19.9 GB — requires 100Gi+ PVC -``` - -All catalog models are **Apache 2.0 or MIT licensed**. - -### Video generation - -Video is **off by default** (`videoAgent.deploy: false`) and needs its own GPU. Set -expectations before enabling it — video is far heavier than image/LLM, and the default -model is the production-quality 14B. - -**Model vs. GPU vs. speed** (measured; warm = model already loaded): - -| `models.video.preset` | Fits | Warm gen (5 s @ 480p) | Quality | Download | -|-----------------------|------|-----------------------|---------|----------| -| `wan-2.1-1.3b` | 8–24 GB | ~1–2 min | Low (warps on motion) | ~17 GB | -| `ltx-video` | 12–24 GB | ~30–60 s | Decent, fast | ~20 GB | -| `cogvideox-5b` | 16 GB+ (offload) | ~3–5 min | Good | ~22 GB | -| `wan-2.1` (14B, **default**) | **48 GB** ideal / 16–24 GB offload | ~3–5 min @ 48 GB · **~30 min** with offload | High | ~40–76 GB | -| `wan-2.2` (A14B) | 48 GB+ | ~5–8 min | Highest | ~55 GB | - -**Latency expectations — read this first:** - -- **First generation is a cold start:** model download (17–76 GB) **+** load into the GPU - (1.3B ≈ 1 min, 14B ≈ 8 min). This is a one-time-per-pod cost but it is *minutes*, not - seconds. Pre-stage the model (below) to avoid it. -- **Warm generations** skip download+load — only the per-clip time in the table applies. -- **Single-concurrency:** the agent generates **one clip at a time**; concurrent requests - get `429 "busy"`. A client timeout/disconnect does **NOT** cancel the in-flight job — it - keeps running and the file lands in `videos/`. - -**Performance knob (important on big GPUs):** on a ≥48 GB GPU the 14B model fits entirely, -so set `videoAgent.offloadMode: none` — the default `model` needlessly swaps weights and is -~6× slower. Use `model` only on 16–24 GB GPUs. - -**Customizing output quality** without code changes: - -```yaml -models: - video: - preset: "wan-2.1" # model (quality tier) — see table above - defaultQuality: "720p" # default resolution/fps when a request omits it: - # proxy 512x320 | draft 640x480 | preview/sd/hd 768x512 | - # 720p 1280x720 | 1080p 1920x1080 | 2k | 4k -videoAgent: - offloadMode: "none" # auto | none | model | sequential - replicas: 1 # raise for parallel generation (see Concurrency below) -``` - -Clients can always override per request with `quality_preset` or explicit -`width`/`height`/`fps`/`duration_seconds`/`steps`. - -**Concurrency (parallel generation):** because the agent is single-concurrency, run -multiple pods to serve clips in parallel: - -```yaml -videoAgent: - replicas: 3 -pvc: - enabled: false # REQUIRED for replicas>1: each pod gets its own emptyDir and - # downloads the model itself. The default ReadWriteOnce PVC can - # only attach to one pod — use emptyDir or a ReadWriteMany class. -``` - -**Pre-staging the model (skip the cold start / air-gapped):** download the model into the -shared PVC once, before agents start, so the first request doesn't pay the download: - -```bash -# into the path the agent reads (MODEL_CACHE_DIR / HF_HOME on the dataDir PVC) -huggingface-cli download Wan-AI/Wan2.1-T2V-14B-Diffusers \ - --local-dir /gridlight-data/models/video/Wan-AI__Wan2.1-T2V-14B-Diffusers -``` - -Run it from a pod that mounts the PVC, or bake the files into a pre-staged volume. With the -model present, the first generation only pays the load time (no download). - -### Air-gapped deployment - -1. Set `bootstrap.enabled: false` -2. Pre-stage the GGUF file on the PVC at the path shown by `helm template` for your chosen preset -3. Set `global.imagePullPolicy: Never` and load images manually - -### Accessing the gateway - -```bash -kubectl port-forward svc/gridlight-gateway 8080:8080 -n gridlight -curl http://localhost:8080/healthz -H "Authorization: Bearer dev-token" -``` - -Primary endpoints: - -| Endpoint | Purpose | -|----------|---------| -| `POST /neon` | RAG query (streaming SSE). Include `"domain"` matching the domain used at ingest time. | -| `POST /chat/intelligent` | Stateful multi-turn conversation | -| `POST /upload-structured` | File ingestion (multipart: `file` + `metadata` JSON with `domain`, `access_tier`, `pii`) | -| `POST /train` | Raw text ingestion | -| `POST /image` | Image generation | -| `POST /voice` | Text-to-speech synthesis | -| `GET /healthz` | Health check | -| `GET /metrics` | Prometheus metrics | - -All requests require `Authorization: Bearer `. - -### Enabling Ingress - -```yaml -ingress: - enabled: true - host: "gridlight.example.com" -``` - -## Troubleshooting - -### LLM/agent never registers — "would exceed TFLOPS ceiling" -Symptom: pods are `Running` but `/agents/list` is empty and the gateway logs -`Agent ... registration rejected: would exceed TFLOPS ceiling. Requested: 125.00, Ceiling: 100.00`. - -Cause: the gateway self-allocates a **default TFLOPS budget (~100)** from the license pool. On a -large GPU (e.g. an A10G reports ~125 TFLOPS) a single agent exceeds that budget and is rejected — -**even when the license has more available** (e.g. 182). In heartbeat-licensing mode the gateway -cannot auto-discover the license ceiling, so it does not raise its own allocation. - -Fix: set the gateway's requested allocation to your license ceiling (from license.gridlight.ai): -```yaml -gateway: - env: - GATEWAY_TFLOPS_REQUESTED: "182" # your license TFLOPS ceiling -``` -Sizing: the budget must cover the **sum** of all GPU agents' reported TFLOPS. Two A10Gs (~250) -need a ceiling ≥ 250; one A10G (~125) needs ≥ 125. - -### Pods stuck `Pending` — PVCs won't bind (raw EKS) -On EKS 1.30+ the in-tree EBS provisioner is gone and `eksctl` clusters have **no default -StorageClass**. Install the EBS CSI driver addon (with an IRSA role) and create a default -`gp3` StorageClass, then reinstall. See the AWS GPU test runbook for the exact commands. - -## References - -- [Gridlight documentation](https://gridlight.ai) -- [Helm chart source](https://github.com/GRIDLIGHT-INC/gridlight/tree/main/helm/gridlight-palette) -- [Image build and kind test guide](https://github.com/GRIDLIGHT-INC/gridlight/blob/main/docs/image-build-and-test.md) -- [Spectrocloud Palette add-on packs](https://docs.spectrocloud.com/integrations/) -- [pack-central community packs](https://github.com/spectrocloud/pack-central) diff --git a/packs/gridlight-1.1.3/charts/gridlight-palette-1.1.3.tgz b/packs/gridlight-1.1.3/charts/gridlight-palette-1.1.3.tgz deleted file mode 100644 index e4409457..00000000 Binary files a/packs/gridlight-1.1.3/charts/gridlight-palette-1.1.3.tgz and /dev/null differ diff --git a/packs/gridlight-1.1.3/logo.png b/packs/gridlight-1.1.3/logo.png deleted file mode 100644 index 2631ae9f..00000000 Binary files a/packs/gridlight-1.1.3/logo.png and /dev/null differ diff --git a/packs/gridlight-1.1.3/pack.json b/packs/gridlight-1.1.3/pack.json deleted file mode 100644 index a46ed4fc..00000000 --- a/packs/gridlight-1.1.3/pack.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "addonType": "app-services", - "annotations": { - "source": "community", - "contributor": "Gridlight" - }, - "cloudTypes": ["all"], - "displayName": "Gridlight AI Platform", - "eol": "", - "group": "", - "kubeManifests": [], - "kubeVersion": ">=1.26.0-0", - "layer": "addon", - "name": "gridlight", - "readme": "## GRIDLIGHT\n\nOn-premises AI platform with LLM inference, semantic search, and specialist agents.\nNo data leaves your cluster.\n\nSee the [README](README.md) for full configuration reference.", - "version": "1.1.3", - "charts": ["charts/gridlight-palette-1.1.3.tgz"] -} diff --git a/packs/gridlight-1.1.3/schema.yaml b/packs/gridlight-1.1.3/schema.yaml deleted file mode 100644 index 575d6d98..00000000 --- a/packs/gridlight-1.1.3/schema.yaml +++ /dev/null @@ -1,25 +0,0 @@ -# Spectrocloud Palette schema — controls UI rendering of sensitive fields. -# Fields listed here are rendered as masked password inputs in the Palette UI. -# Keys are charts..; the chart name is gridlight-palette -# (Chart.yaml name), which is also the key these values live under in values.yaml. - -charts.gridlight-palette.auth.postgresPassword: - schema: '{{ format "${password}" | hints "PostgreSQL password for the gridlight user" }}' - -charts.gridlight-palette.auth.neo4jPassword: - schema: '{{ format "${password}" | hints "Neo4j password for the neo4j user" }}' - -charts.gridlight-palette.auth.gatewayToken: - schema: '{{ format "${password}" | hints "Bearer token clients send in Authorization header. Change from dev-token before production." }}' - -charts.gridlight-palette.auth.agentToken: - schema: '{{ format "${password}" | hints "Token agents use to self-register with the gateway. Change from dev-token before production." }}' - -charts.gridlight-palette.auth.licenseId: - schema: '{{ format "${input}" | hints "Gridlight license UUID from https://license.gridlight.ai" }}' - -charts.gridlight-palette.auth.signingSecret: - schema: '{{ format "${password}" | hints "Auto-acquired at startup. Only set when migrating a pre-existing deployment that already has a signing secret." }}' - -charts.gridlight-palette.auth.huggingFaceToken: - schema: '{{ format "${password}" | hints "HuggingFace read token for gated models. Leave blank if using only open models." }}' diff --git a/packs/gridlight-1.1.3/values.yaml b/packs/gridlight-1.1.3/values.yaml deleted file mode 100644 index 6a3b6c43..00000000 --- a/packs/gridlight-1.1.3/values.yaml +++ /dev/null @@ -1,718 +0,0 @@ -# GRIDLIGHT Helm Chart — Default Values -# All values can be overridden at install time via --set or --values. -# Required fields (no default): auth.postgresPassword, auth.neo4jPassword. - -# --------------------------------------------------------------------------- -# Pack metadata — consumed by Spectrocloud Palette, not by the Helm chart itself. -# Images here must be publicly pullable (pack-central crane pull CI check). -# Points to AWS Public ECR; production deployments use global.imageRegistry below. -# --------------------------------------------------------------------------- -pack: - namespace: "gridlight" - content: - images: - - image: "public.ecr.aws/q1c5c5i2/gridlight/gateway:1.1.3" - - image: "public.ecr.aws/q1c5c5i2/gridlight/agent:1.1.3" - - image: "public.ecr.aws/q1c5c5i2/gridlight/agent-stt:1.1.3" - - image: "public.ecr.aws/q1c5c5i2/gridlight/agent-image:1.1.3" - - image: "public.ecr.aws/q1c5c5i2/gridlight/agent-voice:1.1.3" - - image: "public.ecr.aws/q1c5c5i2/gridlight/agent-video:1.1.3" - -# --------------------------------------------------------------------------- -# Global -# --------------------------------------------------------------------------- -global: - imageRegistry: "public.ecr.aws/q1c5c5i2/gridlight" - imageTag: "1.1.3" - imagePullPolicy: IfNotPresent - imagePullSecrets: [] - -# --------------------------------------------------------------------------- -# Authentication & Secrets -# --------------------------------------------------------------------------- -auth: - # REQUIRED — no default. Provide via --set or a sealed secret. - postgresPassword: "" - neo4jPassword: "" - # Gateway API token — all client requests must include: - # Authorization: Bearer - # IMPORTANT: change this from the default before deploying to production. - gatewayToken: "dev-token" - # Shared token used by Python agents to register with the gateway. - # IMPORTANT: change this from the default before deploying to production. - agentToken: "dev-token" - # Gridlight license ID — required. Obtain from https://license.gridlight.ai - licenseId: "" - # Signing secret — leave blank. The gateway acquires this automatically from - # the license server at first startup using the license ID above. Only set - # this if migrating a deployment that already has a signing secret. - signingSecret: "" - # HuggingFace token — required only if image/video agents need to download gated models. - # Create a read token at https://huggingface.co/settings/tokens - huggingFaceToken: "" - # ECR pull credentials. When set, the chart creates a regcred imagePullSecret. - # Supply the base64-encoded Docker config JSON for your ECR token: - # aws ecr get-login-password | base64 - # Leave blank — public ECR images require no pull credentials. - registryCredentials: "" - -# --------------------------------------------------------------------------- -# Model bootstrap (automatic download on first install) -# --------------------------------------------------------------------------- -# When enabled, an init container downloads models before agents start. -# Skipped if the file already exists (safe to re-run on upgrade). -# Disable for air-gapped environments — pre-stage models on the PVC manually. -bootstrap: - enabled: true - # Embedding model — fixed, not user-selectable (nomic-embed-text v1.5 Q4_0, ~274 MB) - embedModelUrl: "https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/resolve/main/nomic-embed-text-v1.5.Q4_0.gguf" - # LLM model URL is derived automatically from models.llm catalog — no longer set here. - -# --------------------------------------------------------------------------- -# Model Catalog -# --------------------------------------------------------------------------- -# models.llm.category — "general" or "coding" -# models.llm.preset — key from catalog. (see list below) -# models.image.preset — key from catalog (image agent reads IMAGE_MODEL_ID env var) -# models.video.preset — key from catalog (video agent reads DEFAULT_VIDEO_MODEL env var) -# -# To use a model not in the catalog, leave preset blank and set -# llmAgent.modelPath to the full path of a pre-staged GGUF file. -# -# All catalog entries use Apache 2.0 or MIT licenses only. -# --------------------------------------------------------------------------- -models: - llm: - category: "general" # general | coding - preset: "qwen-7b" # see catalog keys below - catalog: - general: - mistral-7b: - name: "Mistral 7B Instruct v0.3" - tier: "entry" - license: "Apache-2.0" - sizeGb: 4.4 - huggingface: "bartowski/Mistral-7B-Instruct-v0.3-GGUF" - filename: "Mistral-7B-Instruct-v0.3-Q4_K_M.gguf" - url: "https://huggingface.co/bartowski/Mistral-7B-Instruct-v0.3-GGUF/resolve/main/Mistral-7B-Instruct-v0.3-Q4_K_M.gguf" - qwen-7b: - name: "Qwen 2.5 7B Instruct" - tier: "entry" - license: "Apache-2.0" - sizeGb: 4.7 - huggingface: "bartowski/Qwen2.5-7B-Instruct-GGUF" - filename: "Qwen2.5-7B-Instruct-Q4_K_M.gguf" - url: "https://huggingface.co/bartowski/Qwen2.5-7B-Instruct-GGUF/resolve/main/Qwen2.5-7B-Instruct-Q4_K_M.gguf" - qwen-32b: - name: "Qwen 2.5 32B Instruct" - tier: "mid" - license: "Apache-2.0" - sizeGb: 19.9 - huggingface: "bartowski/Qwen2.5-32B-Instruct-GGUF" - filename: "Qwen2.5-32B-Instruct-Q4_K_M.gguf" - url: "https://huggingface.co/bartowski/Qwen2.5-32B-Instruct-GGUF/resolve/main/Qwen2.5-32B-Instruct-Q4_K_M.gguf" - mixtral-8x7b: - name: "Mixtral 8x7B Instruct v0.1" - tier: "mid" - license: "Apache-2.0" - sizeGb: 26.4 - huggingface: "TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF" - filename: "mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf" - url: "https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF/resolve/main/mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf" - mixtral-8x22b: - name: "Mixtral 8x22B Instruct v0.1" - tier: "large" - license: "Apache-2.0" - sizeGb: 67.8 - huggingface: "MaziyarPanahi/Mixtral-8x22B-Instruct-v0.1-GGUF" - filename: "Mixtral-8x22B-Instruct-v0.1.Q3_K_M.gguf" - url: "https://huggingface.co/MaziyarPanahi/Mixtral-8x22B-Instruct-v0.1-GGUF/resolve/main/Mixtral-8x22B-Instruct-v0.1.Q3_K_M.gguf" - deepseek-r1: - name: "DeepSeek R1 671B" - tier: "flagship" - license: "MIT" - sizeGb: 161 - shards: 9 - subdir: "DeepSeek-R1-Q2_K" - filenameBase: "DeepSeek-R1-Q2_K" - filename: "DeepSeek-R1-Q2_K-00001-of-00009.gguf" - huggingface: "unsloth/DeepSeek-R1-GGUF" - baseUrl: "https://huggingface.co/unsloth/DeepSeek-R1-GGUF/resolve/main/DeepSeek-R1-Q2_K" - coding: - qwen-coder-7b: - name: "Qwen 2.5 Coder 7B Instruct" - tier: "entry" - license: "Apache-2.0" - sizeGb: 4.7 - huggingface: "bartowski/Qwen2.5-Coder-7B-Instruct-GGUF" - filename: "Qwen2.5-Coder-7B-Instruct-Q4_K_M.gguf" - url: "https://huggingface.co/bartowski/Qwen2.5-Coder-7B-Instruct-GGUF/resolve/main/Qwen2.5-Coder-7B-Instruct-Q4_K_M.gguf" - granite-8b: - name: "IBM Granite 8B Code Instruct 128K" - tier: "entry" - license: "Apache-2.0" - sizeGb: 4.6 - huggingface: "RichardErkhov/ibm-granite_-_granite-8b-code-instruct-128k-gguf" - filename: "granite-8b-code-instruct-128k.Q4_K_M.gguf" - url: "https://huggingface.co/RichardErkhov/ibm-granite_-_granite-8b-code-instruct-128k-gguf/resolve/main/granite-8b-code-instruct-128k.Q4_K_M.gguf" - qwen-coder-14b: - name: "Qwen 2.5 Coder 14B Instruct" - tier: "mid" - license: "Apache-2.0" - sizeGb: 14.9 - huggingface: "bartowski/Qwen2.5-Coder-14B-Instruct-GGUF" - filename: "Qwen2.5-Coder-14B-Instruct-Q8_0.gguf" - url: "https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/resolve/main/Qwen2.5-Coder-14B-Instruct-Q8_0.gguf" - qwen-coder-32b: - name: "Qwen 2.5 Coder 32B Instruct" - tier: "mid" - license: "Apache-2.0" - sizeGb: 19.9 - huggingface: "bartowski/Qwen2.5-Coder-32B-Instruct-GGUF" - filename: "Qwen2.5-Coder-32B-Instruct-Q4_K_M.gguf" - url: "https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/resolve/main/Qwen2.5-Coder-32B-Instruct-Q4_K_M.gguf" - mixtral-8x22b: - name: "Mixtral 8x22B Instruct v0.1" - tier: "large" - license: "Apache-2.0" - sizeGb: 67.8 - huggingface: "MaziyarPanahi/Mixtral-8x22B-Instruct-v0.1-GGUF" - filename: "Mixtral-8x22B-Instruct-v0.1.Q3_K_M.gguf" - url: "https://huggingface.co/MaziyarPanahi/Mixtral-8x22B-Instruct-v0.1-GGUF/resolve/main/Mixtral-8x22B-Instruct-v0.1.Q3_K_M.gguf" - deepseek-r1: - name: "DeepSeek R1 671B" - tier: "flagship" - license: "MIT" - sizeGb: 161 - shards: 9 - subdir: "DeepSeek-R1-Q2_K" - filenameBase: "DeepSeek-R1-Q2_K" - filename: "DeepSeek-R1-Q2_K-00001-of-00009.gguf" - huggingface: "unsloth/DeepSeek-R1-GGUF" - baseUrl: "https://huggingface.co/unsloth/DeepSeek-R1-GGUF/resolve/main/DeepSeek-R1-Q2_K" - - # Image generation models — downloaded by image-agent via HuggingFace Hub. - # The selected preset's huggingface repo ID is passed as IMAGE_MODEL_ID env var. - image: - preset: "juggernautxl" # juggernautxl (default) | cogview3-plus | auraflow | cogview4 (large GPU) - catalog: - juggernautxl: - name: "JuggernautXL v9" - tier: "balanced" - license: "OpenRAIL-M" - sizeGb: 7 - huggingface: "RunDiffusion/Juggernaut-XL-v9" - cogview3-plus: - name: "CogView3-Plus 3B" - tier: "entry" - license: "Apache-2.0" - sizeGb: 26 - huggingface: "THUDM/CogView3-Plus-3B" - auraflow: - name: "AuraFlow v0.3" - tier: "mid" - license: "Apache-2.0" - sizeGb: 16.5 - huggingface: "fal/AuraFlow-v0.3" - cogview4: - name: "CogView4 6B (large GPU — needs >24GB VRAM / large PVC)" - tier: "large" - license: "Apache-2.0" - sizeGb: 31.1 - huggingface: "THUDM/CogView4-6B" - # hidream-i1 intentionally omitted: the image agent does not yet support - # HiDream-I1's multi-encoder setup (load_dit_pipeline rejects it), and at - # 17B/~47GB it doesn't fit common GPUs. Re-add once the agent supports it. - - # Video generation models — downloaded by video-agent via HuggingFace Hub. - # The preset key is passed to the agent as DEFAULT_VIDEO_MODEL (must match a key - # in the agent's model catalog). Keys/repos below mirror agents/video_agent/main.py. - # - # DEFAULT IS wan-2.1 (14B) — production-quality output. It needs a ~48GB GPU - # (L40S / A100) to run at full speed; it also fits 16–24GB GPUs (A10G/4090) via - # CPU offload, just slower. On smaller GPUs, switch preset to "wan-2.1-1.3b". - # Set preset: "auto" to let the agent pick by detected VRAM. - video: - preset: "wan-2.1" # wan-2.1(14B) | wan-2.1-1.3b | cogvideox-5b | ltx-video | auto - # Default output quality/resolution when a request omits it (the gateway reads this - # as DEFAULT_VIDEO_QUALITY). Higher = sharper + slower + more VRAM. A per-request - # "quality_preset" (or explicit width/height/fps/steps) always overrides this. - # proxy(512x320) draft(640x480) preview(768x512) sd/hd(768x512) 720p(1280x720) - # 1080p(1920x1080) 2k(2560x1440) 4k(3840x2160) - defaultQuality: "draft" - catalog: - wan-2.1-1.3b: - name: "Wan 2.1 T2V 1.3B" - tier: "entry" # fits 8–24GB GPUs; fast but low fidelity - license: "Apache-2.0" - sizeGb: 17 - huggingface: "Wan-AI/Wan2.1-T2V-1.3B-Diffusers" - ltx-video: - name: "LTX-Video 0.9.7 Distilled" - tier: "fast" # distilled, low VRAM, quick - license: "Apache-2.0" - sizeGb: 20 - huggingface: "Lightricks/LTX-Video-0.9.7-distilled" - cogvideox-5b: - name: "CogVideoX 5B" - tier: "mid" # ~16GB VRAM with offload; better coherence - license: "Apache-2.0" - sizeGb: 22 - huggingface: "THUDM/CogVideoX-5b" - wan-2.1: - name: "Wan 2.1 T2V 14B" - tier: "flagship" # DEFAULT — needs ~48GB GPU for speed (offloads on 16–24GB) - license: "Apache-2.0" - sizeGb: 40 - huggingface: "Wan-AI/Wan2.1-T2V-14B-Diffusers" - wan-2.2: - name: "Wan 2.2 T2V A14B" - tier: "flagship" # MoE 14B-active; highest quality, needs 48GB+ - license: "Apache-2.0" - sizeGb: 55 - huggingface: "Wan-AI/Wan2.2-T2V-A14B-Diffusers" - -# --------------------------------------------------------------------------- -# Data directory -# --------------------------------------------------------------------------- -# All services write to the same PVC mounted at dataDir. -# Override with GRIDLIGHT_DATA_DIR to use a host-path or different mount. -dataDir: "/gridlight-data" - -# Shared persistent volume claim for model files and data. -pvc: - enabled: true - storageClassName: "" # leave blank for cluster default - size: "100Gi" - accessMode: ReadWriteOnce - -# --------------------------------------------------------------------------- -# Shared model cache (FLEET-001b) — OPTIONAL, default OFF -# --------------------------------------------------------------------------- -# When enabled, LLM models are downloaded ONCE by a pre-install/pre-upgrade "model-warmer" -# Job into a shared ReadWriteMany volume, and every llm pod mounts that volume read-only and -# reads its model from it — no per-pod download, no duplicate storage, and no N-pod download -# race on a shared path (the single warmer Job is the only writer). -# -# Requires a ReadWriteMany PVC you provision yourself (the chart references it, it does not -# create it) — e.g. an EFS-backed PVC on EKS. Leave disabled for single-node / EBS-only -# installs (llm agents then use their own storage.mode, as before). -# -# Scope: LLM agents (large GGUF files — the case where download dedup matters). embed -# (~274 MB) and the HuggingFace-cached image/video/stt/voice agents are out of scope here. -modelCache: - enabled: false - existingClaim: "" # REQUIRED when enabled: name of a pre-provisioned RWX PVC - path: "/gridlight-cache" # mount path for the shared cache inside agent + warmer pods - -# --------------------------------------------------------------------------- -# Gateway -# --------------------------------------------------------------------------- -gateway: - deploy: true - image: - repository: "{{ .Values.global.imageRegistry }}/gateway" - tag: "{{ .Values.global.imageTag }}" - pullPolicy: "{{ .Values.global.imagePullPolicy }}" - replicas: 1 - port: 8080 - service: - type: ClusterIP - port: 8080 - # Idle timeout (seconds) for cloud load balancers. Image/video generation are - # long synchronous requests; the default 60s cuts them off ("Load failed"). - # Applied to AWS ELBs via annotation; for other LBs set service.annotations. - idleTimeoutSeconds: 600 - # Extra Service annotations (e.g. NLB selection, ACM cert, other LB tuning). - annotations: {} - resources: - requests: - cpu: "500m" - memory: "512Mi" - limits: - cpu: "2" - memory: "1Gi" - nodeSelector: {} - tolerations: [] - affinity: {} - # Volume fsGroup for the shared data PVC. The mount root is group-1000-owned (the - # Python agents init it with fsGroup 1000); the gateway runs non-root (uid/gid 999), - # so it needs this gid as a supplemental group to persist its activation signing - # secret under $GRIDLIGHT_DATA_DIR/.gridlight. Keep aligned with the agents' fsGroup. - fsGroup: 1000 - # TFLOPs this deployment claims from the license pool (0 = use full license allowance). - tflopRequested: 0 - # Stable license fingerprint. Leave blank to auto-derive a deterministic value from - # the Helm release (recommended) — this keeps one license allocation across pod - # restarts instead of registering phantom gateways. Set explicitly only to share a - # gateway identity across separate installs/clusters. - licenseFingerprint: "" - # License server URL. Override for air-gapped installs pointing at an on-prem server. - licenseServerUrl: "https://license.gridlight.ai" - # Gateway mode: "" = full (Neo4j + Qdrant), "lite" = SQLite-only (no Neo4j entity resolution). - mode: "" - # Comma-separated list of allowed CORS origins for browser clients. - # Leave blank to allow all origins (permissive — safe for VPN-only deployments). - # Set when exposing the gateway via Ingress to restrict to known origins. - # Example: "https://app.example.com,https://admin.example.com" - corsOrigins: "" - # Escape hatch for advanced gateway tuning. Any key/value pair here is - # injected as an env var on the gateway container. Useful for: - # QUERY_EXPANSION_ENABLED: "false" # disable LLM query expansion (reduces latency) - # RAG_INITIAL_CANDIDATES: "30" # more candidates = better recall, more compute - # NEO4J_ENTITY_RESOLUTION_TIMEOUT_MS: "500" # cap Neo4j latency impact - # LATENCY_LOG_BREAKDOWN: "true" # per-component latency logging - # INTERACTIONS_ENABLED: "true" # enable interaction logging for analytics - # RUST_LOG: "debug" # verbose logging for troubleshooting - # GATEWAY_ROUTING_MODE: "auto" # 1.1.3: enable intent auto-routing (default: explicit/off) - # GATEWAY_TFLOPS_REQUESTED: "182" # REQUIRED on big GPUs: set to your license TFLOPS ceiling. - # # The gateway otherwise self-allocates a default (~100) and - # # rejects agents whose hardware exceeds it ("would exceed - # # TFLOPS ceiling"), even if the license has more available. - # # In heartbeat-licensing mode the gateway cannot auto-discover - # # the ceiling, so set this to the value shown at license.gridlight.ai. - env: {} - -# --------------------------------------------------------------------------- -# PostgreSQL (in-cluster) -# --------------------------------------------------------------------------- -# WARNING: Setting deploy: false disables the in-cluster PostgreSQL instance. -# PostgreSQL stores all conversation history, uploaded document metadata, -# and TFLOP allocation records. Without it, NO context is saved across -# sessions and the gateway will fail to start. Only set to false if you are -# providing an external PostgreSQL instance via gateway.env (DATABASE_URL). -postgres: - deploy: true - image: "postgres:16-bookworm" - port: 5432 - database: "gridlight" - user: "gridlight" - # Password comes from auth.postgresPassword via Secret - resources: - requests: - cpu: "250m" - memory: "512Mi" - limits: - cpu: "2" - memory: "2Gi" - storage: - size: "20Gi" - storageClassName: "" - -# --------------------------------------------------------------------------- -# Qdrant (in-cluster) -# --------------------------------------------------------------------------- -# WARNING: Setting deploy: false disables vector search (QDRANT_ENABLED and -# SEMANTIC_SEARCH_ENABLED are set to false automatically). RAG queries will -# fall back to keyword search only — semantic similarity matching is lost, -# which significantly reduces answer quality for long documents. -qdrant: - deploy: true - image: "qdrant/qdrant:v1.10.0" - grpcPort: 6334 - httpPort: 6333 - collection: "gridlight" - resources: - requests: - cpu: "250m" - memory: "1Gi" - limits: - cpu: "2" - memory: "4Gi" - storage: - size: "20Gi" - storageClassName: "" - -# --------------------------------------------------------------------------- -# Neo4j (in-cluster) -# --------------------------------------------------------------------------- -# WARNING: Setting deploy: false disables the knowledge graph (NEO4J_ENABLED -# is set to false automatically). Entity resolution, relationship-aware -# retrieval, and graph-hop context expansion are all disabled — the gateway -# falls back to GRIDLIGHT_MODE=lite (SQLite-only). Complex multi-hop queries -# and document relationship graphs will not be available. -neo4j: - deploy: true - image: "neo4j:5.18-community" - boltPort: 7687 - httpPort: 7474 - user: "neo4j" - # Password comes from auth.neo4jPassword via Secret - jvm: - heapInitial: "512m" - heapMax: "1G" - pagecache: "512m" - resources: - requests: - cpu: "500m" - memory: "2Gi" - limits: - cpu: "3" - memory: "4Gi" - storage: - size: "10Gi" - storageClassName: "" - -# --------------------------------------------------------------------------- -# LLM Agent fleet (FLEET-002) -# --------------------------------------------------------------------------- -# Declare MULTIPLE LLM agents, each with its own model, storage, and node pool. All -# instances register with the one gateway, which load-balances across them. When this list -# is non-empty it REPLACES the singular `llmAgent:` block below (which is deprecated and kept -# only for backward compatibility — existing installs keep working untouched). -# -# Each entry fields: name (required, DNS-safe), deploy, model{category,preset|modelPath}, -# replicas, storage{mode,size,accessMode,storageClassName}, resources, nodeSelector, -# tolerations, affinity, device, env, port. (replicas>1 + storage.mode ownPvc → StatefulSet.) -# -# Example — three different models across GPU pools: -# llmAgents: -# - name: qwen -# model: { preset: qwen-7b } -# storage: { mode: ownPvc, size: 20Gi } -# nodeSelector: { gridlight.ai/pool: gpu-a10g } -# - name: mistral -# model: { preset: mistral-7b } -# storage: { mode: ownPvc, size: 20Gi } -# nodeSelector: { gridlight.ai/pool: gpu-a10g } -# - name: big -# model: { preset: qwen-32b } -# storage: { mode: ownPvc, size: 40Gi } -# resources: { limits: { nvidia.com/gpu: "1" } } -# nodeSelector: { gridlight.ai/pool: gpu-a100 } -llmAgents: [] - -# --------------------------------------------------------------------------- -# LLM Agent (Rust, port 8001) — DEPRECATED singular form; use llmAgents[] above for fleets. -# Used as the single "agent" instance when llmAgents is empty (backward-compatible default). -# --------------------------------------------------------------------------- -llmAgent: - deploy: true - image: - repository: "{{ .Values.global.imageRegistry }}/agent" - tag: "{{ .Values.global.imageTag }}" - pullPolicy: "{{ .Values.global.imagePullPolicy }}" - port: 8001 - # Override the model path used by llama-server. Leave empty to auto-derive - # from models.llm.catalog. Set this only when using a pre-staged model - # outside the catalog (also set bootstrap.enabled=false in that case). - modelPath: "" - # CPU-only mode: set to "cpu" to bypass GPU detection - # device: "" - # Storage for the model volume (FLEET-001): - # shared — mount the shared {release}-data PVC (DEFAULT; current/backward-compatible). - # ownPvc — dedicated PVC for this agent so it can run on its own node pool. - # emptyDir — per-pod ephemeral volume; re-downloads on restart (small models only). - # Leave as "shared" for single-node installs. Switch to "ownPvc" when spreading a fleet - # across node pools. (replicas>1 + ownPvc → StatefulSet arrives in FLEET-003.) - # CAVEATS when changing mode on an existing install: (1) switching TO ownPvc creates a - # fresh empty PVC, so the model re-downloads (multi-GB for the LLM); (2) switching AWAY - # from ownPvc leaves the old per-agent PVC behind (it carries resource-policy: keep, so - # delete it manually if you don't want the orphaned storage). - storage: - mode: shared - size: "100Gi" # ownPvc PVC request / emptyDir sizeLimit - accessMode: ReadWriteOnce # ownPvc only — ReadWriteOnce | ReadWriteOncePod (EBS-only) - # storageClassName: "" # ownPvc only; falls back to pvc.storageClassName - resources: - requests: - cpu: "2" - memory: "8Gi" - limits: - cpu: "8" - memory: "16Gi" - nodeSelector: {} - tolerations: [] - affinity: {} - env: {} - -# --------------------------------------------------------------------------- -# Embed Agent fleet (FLEET-002) -# --------------------------------------------------------------------------- -# Multiple embed agents (same fields as llmAgents[]). When non-empty this REPLACES the -# singular embedAgent: block below. The embed↔llm co-location affinity only applies to the -# legacy "agent" instance in shared mode; list instances on ownPvc/emptyDir schedule freely. -embedAgents: [] - -# --------------------------------------------------------------------------- -# Embed Agent (Rust, port 8002) — DEPRECATED singular form; use embedAgents[] for fleets. -# Used as the single "agent" instance when embedAgents is empty (backward-compatible default). -# --------------------------------------------------------------------------- -embedAgent: - deploy: true - image: - repository: "{{ .Values.global.imageRegistry }}/agent" - tag: "{{ .Values.global.imageTag }}" - pullPolicy: "{{ .Values.global.imagePullPolicy }}" - port: 8002 - # Path to the embedding .gguf model file inside the container. - # Defaults to the bootstrap download destination. - modelPath: "/gridlight-data/models/embeddings/nomic-embed-text-v1.5.Q4_0.gguf" - # Storage for the embedding model volume (FLEET-001): - # shared — share the {release}-data PVC with llm-agent (DEFAULT; co-locates on one - # node under RWO — current/backward-compatible behavior). - # ownPvc — dedicated PVC so embed can move to a cheap CPU node pool (drops the - # llm co-location affinity). The nomic model is ~274 MB, so a small PVC. - # emptyDir — per-pod ephemeral; re-downloads the ~274 MB model on restart. - # To isolate embed onto CPU nodes: set mode "ownPvc" (or "emptyDir") AND a CPU nodeSelector. - storage: - mode: shared - size: "2Gi" # ownPvc PVC request / emptyDir sizeLimit (~274 MB model) - accessMode: ReadWriteOnce # ownPvc only — ReadWriteOnce | ReadWriteOncePod (EBS-only) - # storageClassName: "" # ownPvc only; falls back to pvc.storageClassName - resources: - requests: - cpu: "1" - memory: "2Gi" - limits: - cpu: "4" - memory: "6Gi" - nodeSelector: {} - tolerations: [] - affinity: {} - env: {} - -sttAgents: [] # FLEET-002/005 fleet form (fields + modelSize/computeType per instance); replaces sttAgent when non-empty -# --------------------------------------------------------------------------- -# STT Agent (Python / faster-whisper, port 8004) — DEPRECATED singular; use sttAgents[]. -# --------------------------------------------------------------------------- -sttAgent: - deploy: true - image: - repository: "{{ .Values.global.imageRegistry }}/agent-stt" - tag: "{{ .Values.global.imageTag }}" - pullPolicy: "{{ .Values.global.imagePullPolicy }}" - port: 8004 - # Whisper model size: tiny (39MB), base (74MB), small (244MB), medium (769MB), large-v3 (1.5GB) - modelSize: "tiny" - # Quantization: auto, float16, int8 (int8 recommended for CPU) - computeType: "auto" - resources: - requests: - cpu: "500m" - memory: "512Mi" - limits: - cpu: "4" - memory: "4Gi" - nodeSelector: {} - tolerations: [] - affinity: {} - env: {} - -# --------------------------------------------------------------------------- -# Image Agent fleet (FLEET-002/004) -# --------------------------------------------------------------------------- -# Multiple image agents, each its own model + storage + GPU node. Fields like llmAgents[], -# plus model.preset (image catalog key) and fsGroup. When non-empty this REPLACES imageAgent:. -# Example: -# imageAgents: -# - { name: jugg, model: { preset: juggernautxl }, storage: { mode: ownPvc, size: 20Gi } } -# - { name: flux, model: { preset: auraflow }, storage: { mode: ownPvc, size: 30Gi } } -imageAgents: [] - -# --------------------------------------------------------------------------- -# Image Agent (Python / diffusers, port 8005) — DEPRECATED singular; use imageAgents[] for fleets. -# --------------------------------------------------------------------------- -imageAgent: - deploy: true - image: - repository: "{{ .Values.global.imageRegistry }}/agent-image" - tag: "{{ .Values.global.imageTag }}" - pullPolicy: "{{ .Values.global.imagePullPolicy }}" - port: 8005 - resources: - requests: - cpu: "2" - memory: "8Gi" - limits: - cpu: "8" - memory: "16Gi" - nodeSelector: {} - tolerations: [] - affinity: {} - env: {} - -voiceAgents: [] # FLEET-002/005 fleet form (fields + defaultVoice per instance); replaces voiceAgent when non-empty -# --------------------------------------------------------------------------- -# Voice Agent (Python / Piper TTS, port 8006) — DEPRECATED singular; use voiceAgents[]. -# --------------------------------------------------------------------------- -voiceAgent: - deploy: true - image: - repository: "{{ .Values.global.imageRegistry }}/agent-voice" - tag: "{{ .Values.global.imageTag }}" - pullPolicy: "{{ .Values.global.imagePullPolicy }}" - port: 8006 - defaultVoice: "en_US-amy-medium" - resources: - requests: - cpu: "250m" - memory: "512Mi" - limits: - cpu: "2" - memory: "2Gi" - nodeSelector: {} - tolerations: [] - affinity: {} - env: {} - -# --------------------------------------------------------------------------- -# Video Agent (Python / diffusers, port 8007) -# --------------------------------------------------------------------------- -# OFF by default (needs its own GPU) but validated end-to-end. The default model -# (models.video.preset) is wan-2.1 (14B, production quality). -# -# GPU SIZING — this is the important one: -# • wan-2.1 (14B, DEFAULT): best on a ~48GB GPU (L40S / A100). It also runs on a -# 16–24GB GPU (A10G/4090) via CPU offload (VIDEO_OFFLOAD_MODE=model) — slower, -# and needs the larger host-memory limit below for the offloaded weights. -# • On smaller GPUs, set models.video.preset to "wan-2.1-1.3b" (fits 8–24GB) or -# "auto" (agent picks by detected VRAM). -# The image already bundles the fast plain-GET model downloader (Xet repos otherwise -# stall in huggingface_hub). Models are large (17–55 GB) and slow on first generate. -# The video agent is single-concurrency: it returns 429 while a generation is in -# flight, and a client disconnect does NOT cancel the in-flight job. For concurrent -# video, run multiple replicas. Object storage / async generation tracked in GRD-2354. -videoAgents: [] # FLEET-002/004 fleet form (fields like imageAgents[] + offloadMode); replaces videoAgent when non-empty -videoAgent: - deploy: false - # Replicas >1 enable parallel generation (the agent is single-concurrency and returns - # 429 while busy). Each replica needs its own model storage: set pvc.enabled=false - # (emptyDir per pod — re-downloads the model) or use a ReadWriteMany storage class. - # The default ReadWriteOnce PVC can only attach to a single pod. - replicas: 1 - image: - repository: "{{ .Values.global.imageRegistry }}/agent-video" - tag: "{{ .Values.global.imageTag }}" - pullPolicy: "{{ .Values.global.imagePullPolicy }}" - port: 8007 - # CPU offload mode for large models: auto | none | model | sequential. - # "model" keeps the 14B default within VRAM on 16–24GB GPUs; "none" is fastest on - # a 48GB GPU that fits the whole pipeline. - offloadMode: "model" - resources: - requests: - cpu: "2" - memory: "16Gi" - limits: - cpu: "8" - # 56Gi headroom for CPU-offloaded 14B weights; lower to 24Gi if pinning a - # small model (wan-2.1-1.3b) that stays resident on the GPU. - memory: "56Gi" - nvidia.com/gpu: "1" # video generation requires a GPU - # Schedule onto a large-VRAM GPU node, e.g.: - # nodeSelector: { node.kubernetes.io/instance-type: g6e.2xlarge } # L40S 48GB - nodeSelector: {} - tolerations: [] - affinity: {} - env: {} - -# --------------------------------------------------------------------------- -# Ingress -# --------------------------------------------------------------------------- -ingress: - enabled: false - # ingressClassName: "nginx" - host: "" - # tls: [] - annotations: {}