✨ One pager for KubeVM: A pluggable, provider agnostic APIs for Virtual Machines on Kubernetes - #1893
Conversation
Reference the actual review PR now that it exists, instead of a generic pointer to open PRs against this file.
Point straight at PR vmware-tanzu#1893 without explaining why the file was split out.
* Move KubeVM one-pager content out for PR review The one-pager landed as a single large file, which made it unreviewable as a diff. Replace it with a pointer while the content is reintroduced through a follow-up pull request that shows it as a reviewable addition. * Point one-pager placeholder at PR #1893 Reference the actual review PR now that it exists, instead of a generic pointer to open PRs against this file. * Simplify one-pager placeholder text Point straight at PR #1893 without explaining why the file was split out. * Add please to one-pager placeholder text
Reintroduce the full one-pager content on top of the placeholder so this PR shows the proposal as a reviewable diff instead of a single opaque file.
f09156b to
26ba100
Compare
github-actions
Bot
added
size/L
|
|
||
| ## Business Problem | ||
|
|
||
| Kubernetes has become the default control plane for modern infrastructure, yet the ecosystem still lacks a cross-platform, VM-centric API: a single declarative surface through which any hypervisor or cloud can expose both the full lifecycle of a virtual machine and the hardware capabilities that demanding workloads depend on. |
There was a problem hiding this comment.
How do we prevent this from being a least common denominator API?
Also what are the thoughts on leaky abstractions because folks will ALWAYS want to configure/tweak things below especially calling out security or performance
There was a problem hiding this comment.
How do we prevent this from being a least common denominator API?
As with any new standard, I expect some challenges with this early on. But, as we start to show value by bringing in abstractions on top of the virtual machine layer, I expect more and more providers to converge into integrating into this API.
Taking the same example of VirtualMachineDeployments: If the KubeVM orchestration layer handles creation, rollout and deletion of the VMs of the Deployment, every provider gains this feature without doing any work at all.
Also what are the thoughts on leaky abstractions because folks will ALWAYS want to configure/tweak things below especially calling out security or performance
This is a fair point and this will always be a discussion. For a provider that does a certain feature in a completely bespoke manner, they can continue to handle that via the provider APIs. However, in doing so, they would lose the benefits offered by the KubeVM API.
|
|
||
| Kubernetes has become the default control plane for modern infrastructure, yet the ecosystem still lacks a cross-platform, VM-centric API: a single declarative surface through which any hypervisor or cloud can expose both the full lifecycle of a virtual machine and the hardware capabilities that demanding workloads depend on. | ||
| This gap is becoming urgent because a new class of workload is arriving faster than the tooling to run it. | ||
| Agentic workloads — long-running processes that execute model-generated code and orchestrate tools — are increasingly deployed inside virtual machines, both for the strong isolation a VM provides around untrusted code and for direct access to the hardware accelerators, such as GPUs, SR-IOV network functions, and passthrough devices, that hypervisors already virtualize well. |
There was a problem hiding this comment.
Is the effort here then ... consistent / exact / replicatable environment across providers?
There was a problem hiding this comment.
Precisely. The goal is to get to a model of what I am calling "thick core with thin provider". Which basically means most of the fields land up in the generic KubeVM API layer with very minor platform specific things landing in the provider resources.
The value proposition is a portable resource. A much stronger value proposition is abstractions built on top of these resource templates. E.g., a VirtualMachineReplicaSet resource that knows how to orchestrate VM replicas from templates.
| Kata Containers provides VM-strength isolation for individual workloads by wrapping a Pod in a lightweight micro-VM — well suited to isolating untrusted code at the granularity of a container. | ||
| KubeVirt takes a different approach, converging the virtual machine into the container model by running a QEMU/KVM process inside a Pod, which is an excellent fit when Kubernetes is the sole infrastructure layer and rich, device-level VM modeling on Kubernetes nodes is the goal. | ||
| Both are strong at their design point. | ||
| What neither sets out to be is a portable, Kubernetes-native front door to a full-blown, hypervisor-native estate — an existing vSphere deployment or a public-cloud VM service — that exposes that platform's own lifecycle and hardware capabilities (GPUs, SR-IOV, passthrough) through one vendor-neutral API. |
There was a problem hiding this comment.
Here are we saying we don't care where we are running? (like virtual kubelet creating things outside of the nodes in the k8s cluster itself?)
There was a problem hiding this comment.
Exactly. It is up to the provider to figure out where and how the virtual machines are instantiated. For a provider like Kubevirt, it will be on one of the nodes. For a provider like vSphere / EC2 / GCP, it can be a "true" VM directly on the hypervisor.
| KubeVirt takes a different approach, converging the virtual machine into the container model by running a QEMU/KVM process inside a Pod, which is an excellent fit when Kubernetes is the sole infrastructure layer and rich, device-level VM modeling on Kubernetes nodes is the goal. | ||
| Both are strong at their design point. | ||
| What neither sets out to be is a portable, Kubernetes-native front door to a full-blown, hypervisor-native estate — an existing vSphere deployment or a public-cloud VM service — that exposes that platform's own lifecycle and hardware capabilities (GPUs, SR-IOV, passthrough) through one vendor-neutral API. | ||
| That is the gap KubeVM fills, and it is complementary to both. |
There was a problem hiding this comment.
What's the stance on making kubernetes resources being made available to things in the VM? (secrets may be handy? env variables? how about CSI volumes?)
Are we limited to what is available in the underlying provider API?
There was a problem hiding this comment.
Good point. Thank you for calling this out. KubeVM will provide the abstraction that lets providers accomplish these tasks. For example: a spec.volume reference with a SecretRef can be consumed by the provider to mount the secret as a volume (however they choose to do so).
Are we limited to what is available in the underlying provider API?
Yes, to some extent we will be limited by what a provider (and the underlying platform / hypervisor) can support.
There are opportunities to standardize the API for common lifecycle operations across all providers dealing with VMs. Bootstrapping (cloud-init / Sysprep) comes to mind. We will work with providers on this.
| That is the gap KubeVM fills, and it is complementary to both. | ||
|
|
||
| This document proposes **KubeVM**, a generic and vendor-neutral `VirtualMachine` API served under the `kube-vm.io` group, together with a provider model that lets hypervisors and cloud VM services expose their machines — and, critically, their accelerators — through one portable, Kubernetes-native interface. | ||
| KubeVM is intended to complement KubeVirt, not to replace it: it addresses the hypervisor-native design point that the VM-as-Pod model leaves unaddressed. |
There was a problem hiding this comment.
Do the requested VM(s) run inside existing kubernetes nodes? do we have a stance on that?
There was a problem hiding this comment.
The proposal is agnostic to that. If KubeVM is managing KubeVirt resources, then it will leave the Kubernetes layer orchestrate the scheduling of the VM. If it is another provider such as vSphere, or AWS, the underlying hypervisor layer may want to orchestrate the provisioning and management of VMs.
2 participants
What does this PR do, and why is it needed?
KubeVM is a generic set of Kubernetes-native, hypervisor agnostic APIs to deploy and manage the lifecycle of virtual machines from a Kubernetes cluster. This project provides a consistent interface allowing infrastructure providers to expose core platform capabilities, including scheduling, performance, storage, migration and more. The APIs also allow the platform team to define guardrails and policies around the compute, storage, networking, placement of virtual machines.
Are there any special notes for your reviewer:
N / A