feat(builder): Add rootfs implementation for reading OCI images

[craciunoiuc]

Tested work and it seemed fine. Pasting again uses:
Case 1:

rootfs:
  format: erofs
  type: oci
  source: index.unikraft.io/test/check-pr:latest

Case 2:

rootfs:
  format: erofs
  type: oci
  source: index.docker.io/hello-world:latest

You can use these prefixes: oci://, oci-layout://, oci-archive:// taken from image-spec:

case imagespec.URISchemeOCI, imagespec.URISchemeOCILayout, imagespec.URISchemeOCIArchive:

We decided to use only the oci type variable.

Also needs another bump to the gomod.

Closes: TOOL-1073
Depends-on: unikraft-cloud/x#400

[craciunoiuc]

let me know if these single use functions you want to inline

I found them significant enough to keep separate for now

[Copilot]

Pull request overview

Adds support in the builder to treat OCI image references (registry, OCI layout, OCI archive) as rootfs sources, including logic to unpack regular OCI layers into a rootfs and to use a Unikraft-specific initrd component when present.

Changes:

• Detect OCI rootfs sources via URI schemes and route BuildRootfs to a new OCI implementation.
• Implement buildRootfsOCI to load OCI images per requested platform, flatten layers into a directory, and package into CPIO/EROFS.
• Update kraftfile-to-buildopts translation and expand unit/integration-style tests for OCI sources.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
internal/builder/rootfs.go	Adds OCI source detection + buildRootfsOCI (load, platform match, layer flatten, repack).
internal/builder/rootfs_test.go	Adds DetectSourceType tests for OCI schemes and integration-style rootfs OCI tests.
internal/builder/rootfs_oci_test.go	Adds helpers to generate local OCI archives used by tests.
internal/builder/kraftfile.go	Avoids filepath-joining when rootfs/ROM source is an OCI reference or explicitly typed OCI.
internal/builder/kraftfile_test.go	Adds coverage for OCI paths/types in kraftfile-to-buildopts conversion.
go.mod	Pulls in new (indirect) deps required by containerd/v2 usage.
go.sum	Updates sums for newly introduced transitive dependencies.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

internal/builder/rootfs.go:57

• cfg.Labels = opts.Labels overwrites any labels coming from the base image config even when opts.Labels is nil/unset, which loses inherited labels for OCI-rootfs builds (and any other path that uses applyConfigOverrides). Consider only overriding labels when the caller actually provided them (e.g., if opts.Labels != nil { ... }), or merge maps so explicitly-provided keys override while preserving the rest.

func applyConfigOverrides(base ocispec.ImageConfig, opts BuildOpts) ocispec.ImageConfig {
	cfg := base
	if opts.Cmd != nil {
		cfg.Cmd = opts.Cmd
	}
	if opts.Env != nil {
		env := make([]string, 0, len(opts.Env)+len(cfg.Env))
		for _, kv := range opts.Env {
			env = append(env, fmt.Sprintf("%s=%s", kv.Key, kv.Value))
		}
		cfg.Env = append(env, cfg.Env...)
	}
	cfg.Labels = opts.Labels
	return cfg
}

internal/builder/rootfs_oci_test.go:90

• In OCI image config, RootFS.DiffIDs are expected to be the digests of the uncompressed layer tar streams (diffIDs), while the manifest layer descriptors typically use the digest of the (often compressed) blob. Here diffIDs are derived from desc.Digest (from tarGzipLayer), which makes the fixture OCI config non-spec-compliant and could break consumers that validate diffIDs. Consider computing diffIDs from the uncompressed tar bytes (or streaming hasher before gzip) while keeping desc.Digest for the compressed blob.

	diffIDs := make([]digest.Digest, 0, len(layerDescs))
	for _, desc := range layerDescs {
		diffIDs = append(diffIDs, desc.Digest)
	}

	config := ocispec.Image{
		Platform: ocispec.Platform{
			Architecture: "amd64",
			OS:           "linux",
		},
		Config: ocispec.ImageConfig{
			Cmd: []string{"/bin/sh"},
		},
		RootFS: ocispec.RootFS{
			Type:    "layers",
			DiffIDs: diffIDs,
		},
	}

[craciunoiuc]

Code changed quite a bit cause there were 80 commits merged since I last touched this.

[Copilot]

🟡 Changes recommended

There are a couple of concrete correctness/UX issues in the new shared helper logic (label override semantics and OCI source conflict validation) that should be addressed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 7/8 changed files
• Comments generated: 2
• Review effort level: Lite

[craciunoiuc]

Comment removal comments

[jedevc]

Four integration tests now pull hello-world from Docker Hub anonymously, which is rate limited per-IP and will flake in CI eventually.

suggestion: Maybe we should build an image using SharedImage for this? Then it's on index.unikraft.io with no rate limits.

[craciunoiuc]

Actually, it's a bit weird, we would need to use an image which is standard, so not our usual projects, and it needs to be public also

Maybe something in kraftkit.sh? 🤔

There's this, only 3MB big, maybe ok? kraftkit.sh/xen:4.19

It's stable, not changing any time soon

[craciunoiuc]

alternatively 4.18 which is unchanged since 2024

[jedevc]

Why can't we mirror it temporarily? And just delete it? Why does it need to be standard?

I ideally don't want dependencies on ye olde kraftkit stuff.

[craciunoiuc]

(It's actually a build of the go bindings for using the xen xl library 🤓 )

Ok, but then where would that sit? A new project? I currently know only of the kraftkit.sh where we put regular docker images (note: there are not the user/ukc projects)

Should we create a new one cli? or?