Skip to content

Split sidecar listeners and tighten kube-rbac-proxy to /tasks-only #268

@bdchatham

Description

@bdchatham

Problem

Every /v0/* endpoint on the sidecar is reachable via the proxy on :8443. SAR authz protects only /tasks/* because kube-rbac-proxy's --ignore-paths carves out /v0/healthz, /v0/livez, /v0/startupz, and /v0/metrics (see internal/noderesource/noderesource.go bypassPaths() + buildRBACProxyContainer). The single-port model conflates two distinct posture decisions (authn-required tasks vs unauthenticated health/metrics) onto the same surface, and makes per-purpose NetworkPolicy authoring awkward.

Impact

  • Today: any pod in the cluster can scrape sidecar metrics at :8443/v0/metrics without authn or authz. Content audit tracked at Audit /v0/metrics content; now unauthenticated cluster-wide #266; if any field is sensitive, this is a passive disclosure channel cluster-wide.
  • NetworkPolicy authoring can't cleanly split "lock down task traffic" from "allow scraping" because both share a port. NetworkPolicy: restrict :8443 ingress to the controller SA #265 captures the broad-stroke NetworkPolicy ask but is blocked on this listener split for clean per-purpose rules.
  • Operational posture is acceptable on the current internal-only ~10-node fleet but does not scale to a less-trusted cluster posture.

Relevant experts

  • kubernetes-specialist — pod-spec second container port, Service ports, kubelet probe wiring
  • platform-engineer — label-contract documentation + GitOps-managed NetworkPolicy template
  • security-specialist — trust-boundary review of the listener split + bypass-paths removal

Proposed approach

Three-step sequencing across two repos:

  1. Sidecar (sei-protocol/seictl) splits listeners. Two http.Servers:

    • Tasks: 127.0.0.1:7777 (loopback), /tasks/* only, trusted-header authn
    • Public: 0.0.0.0:9100 (pod IP), /v0/healthz/livez/startupz/metrics, no auth
      New env contract: SEI_SIDECAR_PUBLIC_PORT (defaults to 9100).
  2. Controller (this repo) consumes the new contract.

    • Pod-spec adds the public container port + env var on the sidecar
    • Headless Service exposes both :8443 (proxy) and :9100 (public)
    • Kubelet probes move from :8443 (via proxy) to :9100 (sidecar direct)
    • --ignore-paths removed from buildRBACProxyContainer — everything through :8443 requires SAR
  3. Platform (sei-protocol/platform) ships per-namespace NetworkPolicy.

    • Selects pods by sei.io/node label (controller already publishes this)
    • :8443 ingress: controller SA pod only
    • :9100 ingress: monitoring namespace + kubelet host network
      No controller-side generation — NetworkPolicy is externally provisioned, following the same pattern as signing-key / node-key / operator-keyring Secrets.

Acceptance criteria

  • Sidecar listener split lands in seictl with the SEI_SIDECAR_PUBLIC_PORT env contract
  • Controller pod-spec includes the public container port + env wiring
  • Headless Service exposes both ports
  • --ignore-paths removed from kube-rbac-proxy args
  • Kubelet startup/liveness/readiness probes target the new public port (not the proxy)
  • Doc comment on SelectorLabels/ResourceLabels in internal/noderesource/noderesource.go flags the label set as a load-bearing public contract for platform-side NetworkPolicy
  • Platform-side NetworkPolicy template shipped in sei-protocol/platform GitOps
  • Subsumes NetworkPolicy: restrict :8443 ingress to the controller SA #265

Out of scope

  • /v0/metrics content audit — tracked at Audit /v0/metrics content; now unauthenticated cluster-wide #266. If the audit surfaces sensitive fields, the public-port NetworkPolicy in step 3 tightens to monitoring-only.
  • Per-SeiNode NetworkPolicy generation by the controller — explicitly rejected. Same externalization pattern as signing-key / TLS Secret (now removed).
  • Sidecar-side env-contract changes beyond the public-port split — captured in the seictl issue if/when filed.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions