Repository navigation
EN_Virtualization
A summary of virtualization concepts for DevOps engineers. Click each link for full details.
- Virtualization: Technology that creates virtual versions of physical devices or resources in software
- Type 1 Hypervisor: Runs directly on bare-metal hardware (VMware ESXi, Microsoft Hyper-V)
- Type 2 Hypervisor: Runs on top of a host OS — used for dev/test (VirtualBox, VMware Workstation)
- Full Virtualization: Complete hardware emulation — guest OS requires no modification
- Para-Virtualization: Guest OS is aware of the hypervisor and communicates directly — better I/O performance
- Container Virtualization: Shares host OS kernel, isolates only the app and its dependencies — lighter than VMs (Docker, Kubernetes)
- RAID: Combines multiple physical disks into one logical unit — improves redundancy and performance
- LVM (Logical Volume Manager): Abstraction layer over physical storage — flexible resizing without physical constraints
- Docker: Container platform leveraging Linux cgroups, namespaces, and OverlayFS
- Dockerfile: Text file defining image build instructions (FROM, RUN, COPY, CMD, EXPOSE, ENV)
-
Docker Compose: Tool to define and run multi-container apps using
docker-compose.yml - Docker Hub: Public registry for Docker images
- cgroups: Limits CPU, memory, and I/O resources per process group
- Namespaces: Isolates PID, network, mount, and other resources per container
-
Docker-in-Docker (DinD): Running a separate Docker daemon (dockerd) inside a CI runner container to build images — the
docker:dindsidecar must run privileged (dockerd needs host-level access such as mounting filesystems, configuring cgroups and reaching devices) -
Kaniko: Daemonless tool that builds images by executing a Dockerfile in userspace without a Docker daemon — runs non-privileged, making it CI/Kubernetes-friendly, with layer cache support (
--cache). The upstream project was archived on 2025-06-03; a Chainguard fork continues with security patches only and publishes no public images -
BuildKit: Docker's default build engine on Linux since Engine 23.0 (parallel builds, cache mounts, multi-arch) — drivers:
docker(built-in),docker-container(on a local container / dind),kubernetes,remote -
buildx: The Docker CLI frontend that drives BuildKit (
docker buildx) — builds amd64+arm64 multi-arch manifest lists viabinfmt/QEMU
Covers the concept of abstracting physical resources, comparison with RAID and LVM, full vs para-virtualization, and the differences between host, container, and hypervisor virtualization with their use cases.
→ Details
Covers how Docker works (cgroups, namespaces, OverlayFS), Dockerfile syntax and the image build process, and how to define multi-container services with Docker Compose.
→ Details
Core problem: The CI runner is itself a container, but running docker build inside it needs a Docker daemon — three approaches solve this.
-
DinD: Run a
docker:dindsidecar in privileged mode. Because a real dockerd runs, you accept the privileged cost in exchange for full Docker parity — which is why it stays popular despite the security trade-off. Advantages:-
Full Docker compatibility: every Docker feature and command works as-is — buildx multi-arch,
docker compose, advanced BuildKit features, special builds. None of Kaniko's Dockerfile-instruction / feature gaps. -
Build-and-run in the same job: the image you just built can be launched right away with
docker runin the same job for integration / smoke tests (a daemonless builder cannot run images). -
Familiar workflow: the same
docker buildcommands and Dockerfile you use locally run unchanged in CI — low learning curve and local↔CI behavior parity. - Isolated daemon per job: each CI job gets its own dockerd sidecar, so the build environment is isolated and there is no daemon-state / cache contamination from other jobs.
Cons: privileged means host-kernel access (larger security blast radius), nested overhead, and no layer cache between jobs, since each job starts a fresh daemon — by default every layer of the image is downloaded on every build, so a registry image has to be named as the cache with
--cache-from, and sharing one/var/lib/dockervolume between several daemons is unsafe (the daemon assumes exclusive access to that directory). -
Full Docker compatibility: every Docker feature and command works as-is — buildx multi-arch,
-
Kaniko: Daemonless and non-privileged. Pros: security (no privileged needed), Kubernetes-friendly, layer cache. Cons: limited multi-arch and advanced BuildKit features, plus compatibility gaps on some Dockerfile instructions. The upstream project was also archived on 2025-06-03 and is no longer developed (a Chainguard fork carries security patches only, and the
gcr.io/kaniko-project/executorimage is no longer updated). -
BuildKit + buildx: Docker's default builder. The
docker-containerdriver runs on a local container (e.g. dind); theremote/kubernetesdrivers connect to a separate BuildKit pod. Strong at multi-arch (binfmt), parallel builds, and cache mounts.
| Builder | Docker daemon | Privileged | Multi-arch | Cache | CI / K8s fit |
|---|---|---|---|---|---|
| DinD | Required (dockerd) | Required | Yes (via buildx) | Registry cache (--cache-from) |
Works, but privileged |
| Kaniko | Not needed | Not needed | Limited | Layer cache (--cache) |
Strong (but upstream archived 2025-06) |
| BuildKit (buildx) | Depends on driver | Depends on how BuildKit runs (rootless mode needs no privileged, but in a container it still needs seccomp/AppArmor relaxed plus systempaths=unconfined or --oci-worker-no-process-sandbox) |
Yes (binfmt) | Cache mount | Strong (remote/k8s driver) |
Practical pattern: It is common to split by purpose — build application images with Kaniko (non-privileged), and build amd64+arm64 multi-arch shared images with dind + buildx (docker-container driver). Since the upstream Kaniko project was archived on 2025-06-03, a new setup is better served by BuildKit rootless or Buildah on the non-privileged side — GitLab's docs now mark their Kaniko page as removed and list Buildah among the alternatives, while their BuildKit page presents BuildKit rootless as a replacement for Kaniko (checked 2026-10).