Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/verify-generated.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ jobs:
- name: Verify RBAC is up-to-date
run: make verify-rbac

- name: Verify feature status is up-to-date
run: make verify-feature-status

# Save PR number so the comment workflow (running with write access in the
# base-repo context) can look it up — artifact is the only safe channel
# across the pull_request / workflow_run boundary for fork PRs.
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Binaries for programs and plugins
/feature-status-gen
*.exe
*.exe~
*.dll
Expand Down
10 changes: 10 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,16 @@ verify-rbac: ## Verify RBAC matches generated (for CI)
@rm -f /tmp/generated-rbac.yaml
@echo "✓ RBAC is up-to-date"

.PHONY: generate-feature-status
generate-feature-status: ## Generate feature status table and JSON from metadata
@echo "Generating feature status from metadata..."
@go run cmd/feature-status-gen/main.go

.PHONY: verify-feature-status
verify-feature-status: ## Verify feature status matches generated (for CI)
@echo "Verifying feature status is up-to-date..."
@go run cmd/feature-status-gen/main.go --dry-run

.PHONY: update-crds
update-crds: ## Update CRD collection from upstream
hack/update-crds.sh
Expand Down
24 changes: 24 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,30 @@ The autopilot applies opinionated best practices and manages your platform autom

**Install once. Run VMs. Customize via GitOps when needed.**

## Features

<!-- BEGIN FEATURE STATUS -->
> **Note:** The autopilot framework is currently **TP** and requires `platform.kubevirt.io/autopilot=true` on the HCO CR. Feature maturity levels below are relative to an enabled autopilot.

| Feature | Maturity | Install | Opt-in | Requires | Recommended |
|---------|----------|---------|--------|----------|-------------|
| In-Flight Operations | GA | always | - | - | - |
| Kubelet Performance | GA | always | - | - | - |
| Load-Aware Descheduler | GA | always | - | Kube Descheduler Operator | - |
| Observability | GA | always | - | Cluster Observability Operator | - |
| SWAP | GA | always | - | - | - |
| CPU Manager | DP | opt-in | `featureGate:CPUManager` | - | - |
| Kernel Samepage Merging (KSM) zero pages only | DP | opt-in | `platform.kubevirt.io/enable-ksm-zero-only=true, hcoUnconfigured:spec.virtualization.ksmConfiguration` | - | - |
| KubeVirt Metrics Exporter | DP | opt-in | `platform.kubevirt.io/enable-metrics-exporter=true` | - | Cluster Observability Operator |
| Logging | DP | opt-in | `platform.kubevirt.io/enable-logging=true, platform.kubevirt.io/enable-audit-logging=true` | Loki Operator, Red Hat OpenShift Logging Operator | - |
| MTV Operator | DP | opt-in | `platform.kubevirt.io/enable-mtv=true` | - | - |
| MetalLB Operator | DP | opt-in | `platform.kubevirt.io/enable-metallb=true` | - | - |
| NHC with FAR remediator and SBR detection-mode config | DP | opt-in | `platform.kubevirt.io/enable-node-remediation=true` | Node Health Check Operator, Fence Agents Remediation Operator, Storage Based Remediation Operator | - |
| PCI Passthrough | DP | opt-in | `platform.kubevirt.io/openshift=true, platform.kubevirt.io/enable-pci-passthrough=true` | - | - |
| Transparent Huge Pages (THP) Tuning | DP | opt-in | `platform.kubevirt.io/enable-thp-tuning=true` | - | - |
| Troubleshooting Panel | DP | opt-in | `platform.kubevirt.io/enable-korrel8r=true` | Cluster Observability Operator | - |
<!-- END FEATURE STATUS -->

## Quick Start

### Prerequisites
Expand Down
105 changes: 105 additions & 0 deletions assets/active/metadata.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,9 @@ assets:
value: "true"
- type: hardware-detection
detector: pciDevicesPresent
- type: annotation

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was it missing? does it belong to this PR?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was not gated although not ready to be named GA, and this PR made it visible which is exactly its goal. So let's also gate it.

key: platform.kubevirt.io/enable-pci-passthrough
value: "true"

- name: psi-enable
group: descheduler-loadaware
Expand Down Expand Up @@ -898,3 +901,105 @@ assets:
# - node-maintenance-operator (operators/node-maintenance.yaml.tpl)
# - fence-agents-operator (operators/fence-agents.yaml.tpl)
# - usb-passthrough (machine-config/05-usb-passthrough.yaml.tpl)

# Framework-level maturity. Remove opt_in once the autopilot graduates to GA.

@rlobillo rlobillo Jul 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment implies that when the framework graduates to GA, the opt_in annotation will be removed — but it's not clear what that means in practice for existing clusters. Two possible paths: (1) HCO adds the annotation automatically on upgrade, or (2) the autopilot stops requiring it altogether. Both would silently activate autopilot on existing clusters during an upgrade, potentially triggering MachineConfig rollouts without admin consent.

I'd suggest that platform.kubevirt.io/autopilot=true should be added by default only on new installations, not injected automatically on upgrades. Could you clarify the intended upgrade path when the framework reaches GA?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for raising this.

Intended GA behavior: at GA, we will drop the framework activation gate (path 2). The autopilot is on by default without platform.kubevirt.io/autopilot=true (honoring just platform.kubevirt.io/autopilot=false to completely disable it). HCO will not inject that annotation on upgrade nor the autopilot will touch the HCO CR. This matches the opt-out plan already in ARCHITECTURE.md.

fresh install only? That splits one product into two behaviors simply according to when you deployed it for the first time, doubles test/doc burden... on my opinion is an anti-pattern and we should try to avoid it if not really justified.

MachineConfig concern: only GA features become unconditional at framework GA; DP/TP features keep their own opt-ins. Admins can still opt out per resource (mode: unmanaged, disabled-resources, feature annotations). More than that, framework GA is tied to a minor (or major, e.g. 5.0) release, not a patch one. Admins already expect relevant change at that cadence (node configs refresh, virt-launcher rolls, etc). MachineConfig delivery on that boundary is consistent with that expectation, not a surprise patch-level side effect.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on avoiding fresh install differentiation
MC - I worry mostly about potential troubles when user defined their own conflicting MCs already. I agree we should try to push on it and deliver ours out of the box though...so don't know if we have anything better than a big fat warning in docs.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Machine Config Operator (MCO) is expected to merge all machine configs into a single rendered configuration before applying it. While unexpected conflicts can still occur, sticking to what we've previously documented in product docs or KB articles keeps the risk lower (though never zero). The MCO also uses a rolling update strategy, so it won't move to the next node if the first one fails to come back up.
Ultimately, the risk isn't substantially higher than asking customers to trust and apply a published snippet from our documentation.

framework:
maturity: tp
opt_in: platform.kubevirt.io/autopilot=true

# Feature catalog: maps user-facing features to assets/groups.
# - maturity is required only for opt-in features ("dp" or "tp")
# - features whose assets are all install: always are auto-classified as GA
# - opt-in conditions are derived from the referenced assets (no duplication)
# - every asset must be covered by a feature entry or listed in excluded_assets

# Assets intentionally excluded from feature tracking (e.g. internal-only,
# not yet user-facing, or managed externally).
excluded_assets:
- hco-golden-config

features:
- name: SWAP
description: OpenShift worker node swap support
assets: [swap-enable]

- name: Kubelet Performance
description: Optimized kubelet settings for virtualization workloads
assets: [kubelet-perf-settings]

- name: Observability
description: Prometheus metrics, alerting rules, and Perses dashboards in the OpenShift console
assets: [metrics-service, metrics-servicemonitor, prometheus-alerts, monitoring-ui-plugin, autopilot-dashboard]
requires:
- Cluster Observability Operator

- name: Load-Aware Descheduler
description: Load-aware VM balancing based on CPU/memory utilization and pressure stall metrics
assets: [descheduler-loadaware, psi-enable, psi-enable-master]
requires:
- Kube Descheduler Operator

- name: PCI Passthrough
description: GPU/PCI device passthrough via VFIO
maturity: dp
assets: [pci-passthrough]
Comment thread
tiraboschi marked this conversation as resolved.

- name: CPU Manager
description: Dedicated CPU pinning for guaranteed QoS workloads
maturity: dp
assets: [kubelet-cpu-manager]

- name: MTV Operator
description: Migration Toolkit for Virtualization
maturity: dp
assets: [mtv-operator]

- name: MetalLB Operator
description: Bare-metal load balancer for services
maturity: dp
assets: [metallb-operator]

- name: Logging
description: Integrated logging stack with LokiStack and ClusterLogForwarder
maturity: dp
groups: [logging, audit-logging]
requires:
- Loki Operator
- Red Hat OpenShift Logging Operator

- name: Troubleshooting Panel
description: Korrel8r observability signal correlation in the console
maturity: dp
assets: [troubleshooting-panel-ui-plugin]
requires:
- Cluster Observability Operator

- name: KubeVirt Metrics Exporter
description: Per-node VM storage I/O latency collection via QMP, QGA, and eBPF, and detailed KVM and memory statistics
maturity: dp
groups: [metrics-exporter]
recommended:
- Cluster Observability Operator

- name: In-Flight Operations
Comment thread
tiraboschi marked this conversation as resolved.
description: OperationRuleSet-based coordination for safe concurrent operations
groups: [inflightoperations]

- name: Kernel Samepage Merging (KSM) zero pages only
description: Node-level KSM tuning that enables zero-pages-only deduplication with adaptive scan rate
maturity: dp
groups: [ksm-zero-only]

- name: Transparent Huge Pages (THP) Tuning
description: Node-level THP tuning that sets madvise mode and khugepaged scan rate for KVM guest memory
maturity: dp
groups: [thp-tuning]

- name: NHC with FAR remediator and SBR detection-mode config
description: Node health checks with fence-agents remediation and storage-based remediation in detection-only mode
maturity: dp
assets: [nhc-far, sbr-detection-config]
requires:
- Node Health Check Operator
- Fence Agents Remediation Operator
- Storage Based Remediation Operator
Loading