A container's runtime specification carries hooks and a device list, and neither reaches the guest. The consequence is concrete: a nested container runtime cannot start inside a container here, rootless or rootful. I have this implemented and would like to check the shape before opening the pull request.
What is missing, in four parts
1. The spec's hooks have a field and no way to reach it. Spec.hooks exists on the type, the configuration has nowhere to put them, and the spec a container is built from never carries any. Whether anything runs them is the launching runtime's business, which the field's own documentation says; the gap here is that a caller cannot set them at all.
2. A container in a pod cannot reach an OCI runtime. It is spawned with the runtime path hardcoded to nil, so the runtime a standalone container reaches through ociRuntimePath is out of reach for the same container placed in a pod, and the spec it is built from carries no hooks for that runtime to run. Nothing about a pod prevents either: the runtime path is passed per process, and the pause process is the only one with a reason to stay on the default.
3. vmexec builds the container's root without consulting the spec's device list. A device arrives only if the guest kernel already exposed it in devtmpfs, carrying the kernel's permissions. Those are stricter than what a machine running udev shows: /dev/net/tun and /dev/fuse are 0600 root here and 0666 on any systemd host, which its udev rules set. A container process that is not root therefore cannot open them.
4. A container in a pod has nowhere to name devices, so a device that a machine running udev would show relaxed arrives with the kernel's permissions instead.
Why it matters, end to end
Rootless docker runs under RootlessKit as an ordinary user and needs /dev/net/tun for slirp4netns. Without the spec asking for the permissions a udev host applies, rootlesskit fails to create its tap and the daemon never starts. Rootful docker reaches cgroups directly and holds its containers to the limits they were given, which the rootless daemon drops, since moby takes the none cgroup driver when no systemd is there to delegate to it.
I have an integration case covering both: a docker daemon inside the guest, pulling an image over the network and running a container of its own.
Question for the maintainers
Creating a device the spec names, and setting the permissions it asks for on one already present, is what makes the spec authoritative about what the container sees. That is a behaviour change for any container whose spec lists a device today, so I would rather confirm it is the behaviour you want than assume it.
Related: this builds on the pod fixes in #879 and the storage work in #872.
A container's runtime specification carries hooks and a device list, and neither reaches the guest. The consequence is concrete: a nested container runtime cannot start inside a container here, rootless or rootful. I have this implemented and would like to check the shape before opening the pull request.
What is missing, in four parts
1. The spec's hooks have a field and no way to reach it.
Spec.hooksexists on the type, the configuration has nowhere to put them, and the spec a container is built from never carries any. Whether anything runs them is the launching runtime's business, which the field's own documentation says; the gap here is that a caller cannot set them at all.2. A container in a pod cannot reach an OCI runtime. It is spawned with the runtime path hardcoded to
nil, so the runtime a standalone container reaches throughociRuntimePathis out of reach for the same container placed in a pod, and the spec it is built from carries no hooks for that runtime to run. Nothing about a pod prevents either: the runtime path is passed per process, and the pause process is the only one with a reason to stay on the default.3.
vmexecbuilds the container's root without consulting the spec's device list. A device arrives only if the guest kernel already exposed it in devtmpfs, carrying the kernel's permissions. Those are stricter than what a machine running udev shows:/dev/net/tunand/dev/fuseare0600 roothere and0666on any systemd host, which its udev rules set. A container process that is not root therefore cannot open them.4. A container in a pod has nowhere to name devices, so a device that a machine running udev would show relaxed arrives with the kernel's permissions instead.
Why it matters, end to end
Rootless docker runs under RootlessKit as an ordinary user and needs
/dev/net/tunfor slirp4netns. Without the spec asking for the permissions a udev host applies,rootlesskitfails to create its tap and the daemon never starts. Rootful docker reaches cgroups directly and holds its containers to the limits they were given, which the rootless daemon drops, since moby takes thenonecgroup driver when no systemd is there to delegate to it.I have an integration case covering both: a docker daemon inside the guest, pulling an image over the network and running a container of its own.
Question for the maintainers
Creating a device the spec names, and setting the permissions it asks for on one already present, is what makes the spec authoritative about what the container sees. That is a behaviour change for any container whose spec lists a device today, so I would rather confirm it is the behaviour you want than assume it.
Related: this builds on the pod fixes in #879 and the storage work in #872.