Support inference servers other than Triton, starting with Nereid

[kondratyevd]

Extends SuperSONIC to run inference servers other than Triton, and adds
Nereid as the second supported type.

Nereid speaks the same KServe v2 gRPC protocol as Triton, and its bind_addr is
configurable, so the rendered nereid.yaml keeps Triton's port numbers. Envoy, the
Service, the ServiceMonitor, the Prometheus scrape config and the default scaling
metric therefore need no per-server branching. What genuinely differs is the
image, the model configuration and the probe endpoints.

Commits

ec1cf9e	Rename Triton-specific chart identifiers to inference-server
8a6709f	Make the chart server-agnostic and add the Nereid server type
c7cc61c	Add CI coverage for the Nereid inference server
9213539	Document inference server selection and generalize Triton wording

Breaking change: triton: → inferenceServer:

	before	after
values key	triton:	inferenceServer:
templates	templates/triton/	templates/inference-server/
helper	supersonic.tritonName	supersonic.inferenceServerName
resources	<release>-triton	<release>-inference-server
label	component: triton	component: inference-server
Envoy clusters	triton_grpc_service, triton_admission	inference_server_*
sidecar env	GPU_DEPLOYMENT	INFERENCE_SERVER_DEPLOYMENT

Every downstream values file needs its top-level triton: key renamed. A stale
key now fails the render with a message naming the replacement — Helm would otherwise
ignore it and silently deploy chart defaults. Note that helm lint reports template
fail calls as INFO, so this surfaces on helm install/helm template, not in the
lint job.

Upgrading renames the Deployment, so Helm replaces rather than patches it: expect a
brief interruption, not an immutable-selector error.

What's new

• inferenceServer.type (triton | nereid); an unknown value fails the render.

• Probes accept any one of command (exec shorthand), exec, httpGet or
  tcpSocket. Nereid's image ships no curl, so it needs httpGet against the
  KServe v2 health endpoints.

  Setting two handlers is an error rather than a precedence rule. Helm merges values
  with the chart defaults, so adding httpGet does not remove the inherited
  command; picking a winner would silently probe the wrong endpoint. Switching
  handler is explicit: command: null.

• command/args are omitted when empty, so a file-configured server uses its
  image's entrypoint.

• inferenceServer.nereid.config renders to a ConfigMap mounted at nereid.yaml
  via subPath, so it does not shadow the model folders sharing Nereid's working
  directory.

• A configMap model-repository storageType, for small models and testing.

• values/values-nereid.yaml — a complete worked example.

Testing

.github/workflows/ci-nereid.yaml installs the chart with type: nereid on kind and
drives one inference through Envoy.

The client is a pip-installed tritonclient rather than the one in the Triton SDK
image: the claim under test is that an unmodified KServe v2 client works against
Nereid, and building it from Triton's own proto stubs is what makes that a
cross-implementation check rather than a self-consistency one. Tensor names and shapes
come from ModelMetadata instead of being hardcoded. perf_analyzer is deliberately
not used — it needs the model-config and statistics RPCs, which Nereid does not
implement.

The model is nereid-server's own ml-backends/model3 example (a 10 KB TorchScript
MLP) vendored into a ConfigMap, so the job needs no model volume, no CVMFS and no
network fetch.

Verification of the rename

All 8 pre-existing values files plus chart defaults were rendered before and after
every commit and compared as parsed Kubernetes objects, so key reordering is not
mistaken for change. The only intended render difference in the whole PR is the
dashboard panel retitling, confirmed by diffing the rendered dashboard ConfigMap to be
titles only, no query changes. helm lint passes on all 11 values files.

Upstream dependencies — please read before merging

1. Depends on Triton-compatible Prometheus metrics and KServe v2 HTTP health endpoints ngpaladi/nereid-server#19 for the /v2/health/{live,ready} endpoints
   the probes use and the Triton-named nv_inference_* metrics that the ServiceMonitor
   scrapes and the default serverLoadMetric reads. Until it merges, Nereid has no
   working probe endpoint and no scaling metric.
2. Depends on Implement the KServe v2 RepositoryIndex RPC ngpaladi/nereid-server#20 for the RepositoryIndex RPC that
   scaleFromZero triggers on. Nothing in that path is Triton-specific: Nereid vendors
   the same inference.GRPCInferenceService proto, so
   /inference.GRPCInferenceService/RepositoryIndex — the path the Envoy route and the
   Lua filter match on — is identical for both server types, and the chart needs no
   change. Until it merges, scaleFromZero cannot be used with type: nereid.
3. The Nereid image is a placeholder. The repository has no releases or tags and
   its GHCR package is unpublished, so the CI job is gated on the image resolving and
   skips with a GitHub notice until then. Remove the gate and pin a tag once a release
   exists.

Not addressed, documented as Triton-only

Nereid exports no nv_gpu_* metrics (explicitly out of scope of #19), so the GPU
panels of the Grafana dashboard and the optional metricsCollector stay blank under
Nereid. Reworking the dashboard belongs in its own PR.

Incidental fixes

• yaml-to-schema.py now drops genson's inferred required lists. genson only ever
  sees values.yaml, so required was a copy of the defaults — which Helm merges into
  every release, making the constraint unable to catch a missing key while rejecting a
  values file that deliberately clears one. Nulling a probe handler was blocked by
  exactly this. This is a repo-wide change beyond the Nereid work.
• The configuration guide had two sections numbered 1..

Known sharp edge, not fixed

The same generator types inferenceServer.resources.requests.cpu as integer from the
single sample in values.yaml, so cpu: 500m — or even cpu: 0.5 — is rejected there.
Worked around with cpu: 1 in the CI values rather than widening this PR's scope.

Rebased onto #87 and #86

Rebased onto main after both merged. Resolutions, in each case taking the upstream
content and applying this branch's rename to it:

From #87 (OpenTelemetry/Tempo removal):

• values.yaml — the envoy.tracing_sampling_rate knob I had reworded is gone with
  tracing support; took the deletion.
• dashboards/with-server-map.json — deleted upstream, so my panel retitling in it went
  away with the file. dashboards/default.json still carries the retitling.
• Dropped the now-dead OpenTelemetry repo line from the Nereid CI job.

From #86 (dependency upgrades):

• Ten conflicts, all "upstream's content, this branch's names": the 26.08 Triton image
  bumps, the Envoy admin.access_log / typed_extension_protocol_options migrations,
  and the FQDN form of the upstream address in the external-config test.
• Upgrade all dependencies to latest versions #86 introduced new Triton-named identifiers that this rename now also covers:
  the triton_admission calls in its new KEDA-hold refresh path, .tritonName in the
  Envoy cluster address, and supersonic-triton.cms.svc.cluster.local. in the test
  fixture, plus its new user-facing strings (reject_reason, comments).
• Adopted Upgrade all dependencies to latest versions #86's shared CI helpers in the Nereid job: scripts/wait-for-job.sh (polls
  for Complete or Failed, so a failed Job reports immediately instead of idling to
  the timeout) and the k8s-diagnostics composite action. This job was the only one
  left hand-rolling both.

values.schema.json and docs/.values-table.md are generated, so they were regenerated
at each step using the generator version in effect at that commit, rather than
hand-merged.

Re-verified against the current main: all 8 pre-existing values files plus chart
defaults render identically once the rename map is applied, compared field by field as
parsed Kubernetes objects. The only differences are this branch's own wording changes —
confirmed by diffing the rendered Lua, Envoy and dashboard blobs to be comments, log
strings and panel titles, with no logic or query changes. helm lint passes on all 10
values files, both guard-rails fire, and the Envoy cluster names the Lua filter calls
all resolve against the generated config.

🤖 Generated with Claude Code