sec(vm): confine Linux VM sidecar job containers with AppArmor - #145
Draft
luthermonson wants to merge 1 commit into
Draft
sec(vm): confine Linux VM sidecar job containers with AppArmor#145luthermonson wants to merge 1 commit into
luthermonson wants to merge 1 commit into
Conversation
Native Linux hosts confine job containers with the ephemerd-default AppArmor profile (#124), but the Linux VM sidecar that runs Linux jobs on Windows and macOS hosts did not: apparmorOpts fail-opened inside the VM, so Linux jobs ran under seccomp + the reduced capability set + the read-only /proc,/sys mounts, but with no AppArmor — a containment-parity gap versus the native Linux pools. Three things were missing in the guest; none needs a kernel rebuild (the Alpine linux-virt kernel has AppArmor compiled in, only disabled at boot): - Kernel cmdline: add `apparmor=1 security=apparmor` on both the Hyper-V (linuxvm_windows.go) and Apple Vz (linuxvm_darwin.go) boots. The kernel's builtin CONFIG_LSM is landlock,lockdown,yama,loadpin,safesetid,integrity (no apparmor) and neither cmdline sets lsm=, so security=apparmor is honored and appends apparmor to the LSM order rather than being superseded by it. - Init script: mount securityfs at /sys/kernel/security before the daemon starts, in both initrd init scripts, so the loaded-profile list can be read and the profile loaded. - Rootfs: add the Alpine apparmor (apparmor_parser) + libintl packages and ship /etc/apparmor.d/abi/3.0, the feature ABI that the containerd- generated profile's `abi <abi/3.0>,` include resolves against. The full apparmor-profiles package is deliberately NOT added: it would also install tunables/global + abstractions/base, flipping containerd's macroExists and pulling glibc-oriented includes into the generated profile. Keeping /etc/apparmor.d minimal leaves the profile self-contained (apparmor_parser links only libintl, not libapparmor). Validated on a Windows Hyper-V sidecar: the kernel now boots lsm=lockdown,capability,landlock,apparmor with "AppArmor initialized"; the in-VM resolver logs "job containers are AppArmor confined profile=ephemerd-default mode=enforce" (previously "running WITHOUT AppArmor confinement"); and a container dispatched through the real runtime.Create path gets an OCI spec with Process.ApparmorProfile=ephemerd-default while seccomp, the dropped CAP_MKNOD, the device-cgroup deny-all, and the masked/RO proc,sys mounts all remain in place.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Native Linux hosts (coyotes/sharks) confine job containers with the
ephemerd-defaultAppArmor profile (#124). The Linux VM sidecar that runs Linux jobs on Windows and macOS hosts did not:apparmorOptsfail-opened inside the VM, so Linux jobs ran under seccomp + the reduced capability set + read-only/proc,/sys— but with no AppArmor. Everything else (seccomp, cap-drop incl. CAP_MKNOD, device-cgroup deny, masked/RO proc+sys, mount/PID isolation, egress) already matched; AppArmor was the sole containment-parity gap.Confirmed on a metal test: inside a real sidecar container
Process.ApparmorProfile="",/proc/1/attr/current→EINVAL,/sys/module/apparmor/parameters/enabled=N.Root cause
The Alpine
linux-virt6.12 kernel ships AppArmor compiled in but disabled at boot — its builtinCONFIG_LSMislandlock,lockdown,yama,loadpin,safesetid,integrity(no apparmor),securityfsis not mounted, and the cmdline lacked the enable flags. The sidecar rootfs also had noapparmor_parser. No kernel rebuild is required (verified via the AppArmor symbols/strings and module params in the embeddedvmlinuz).Fix
linuxvm_windows.go+linuxvm_darwin.go): addapparmor=1 security=apparmor. Neither cmdline setslsm=, so on 6.12security=apparmoris honored (not superseded) and appends apparmor to the LSM order.mage/download/download.go): mountsecurityfsat/sys/kernel/securitybefore the daemon starts.download.go): add Alpineapparmor(→/sbin/apparmor_parser) +libintl(the only lib the parser links), and ship/etc/apparmor.d/abi/3.0(the ABI the containerd-generated profile'sabi <abi/3.0>,include resolves against). The fullapparmor-profilespackage is intentionally avoided — it would also droptunables/global+abstractions/baseinto/etc/apparmor.dand flip containerd'smacroExists, pulling glibc-oriented includes into the generated profile. Keeping/etc/apparmor.dminimal keeps the profile self-contained.ephemerd-defaultprofile itself is generated + loaded in-VM by the existingpkg/runtime/apparmor.go(containerdcontrib/apparmor) — no static profile file needed.Validation (Windows Hyper-V sidecar, real dispatch path)
Booted the sidecar via
vm.StartLinuxVMand dispatched a container throughruntime.Create:LSM: initializing lsm=lockdown,capability,landlock,apparmor,AppArmor initialized,AppArmor Filesystem Enabled.job containers are AppArmor confined profile=ephemerd-default mode=enforce(wasrunning WITHOUT AppArmor confinement).Process.ApparmorProfile=ephemerd-default.SCMP_ACT_ERRNO(12 rules), 9 caps with CAP_MKNOD dropped, device-cgroup deny-all, MaskedPaths=11, ReadonlyPaths=5.Follow-ups
linuxvm_darwin.go) gets the same cmdline + shares the init/rootfs changes, but could not be validated on this Windows box — needs a smoke test on a Mac host.ephemerd-linux,vmlinuz,initrd, rootfs) are gitignored/built bymage build, so this PR is source-only; CI must rebuild them.