Skip to content

feat: add cross-compilation support and Docker containerized builds for kernel packaging#33

Open
guanquan (GuanquanTian) wants to merge 4 commits into
qualcomm-linux:mainfrom
GuanquanTian:build-kernel-deb-cross-improvements
Open

feat: add cross-compilation support and Docker containerized builds for kernel packaging#33
guanquan (GuanquanTian) wants to merge 4 commits into
qualcomm-linux:mainfrom
GuanquanTian:build-kernel-deb-cross-improvements

Conversation

@GuanquanTian

@GuanquanTian guanquan (GuanquanTian) commented Jul 8, 2026

Copy link
Copy Markdown

Summary

This PR enhances the kernel build infrastructure with cross-compilation support and Docker containerization:

  1. build-kernel-deb.sh improvements

    • Add cross-compilation support for building arm64 kernels on non-arm64 hosts via dpkg foreign architecture and gcc cross-toolchain
    • Support VERSION_SUFFIX parameter for traceable local builds with git commit hash (including special auto value)
    • Add SKIP_BUILD_DEP environment variable to skip dependency installation (useful for pre-built Docker images)
    • Improve artifact collection with version-based matching to avoid stale packages
    • Add do_skip_checks flag to bypass config-policy check for optional toolchains
  2. docker-build-kernel.sh wrapper script

    • New Docker wrapper for containerized kernel builds
    • Auto-detect host architecture and select appropriate Docker image
    • Support SKIP_BUILD_DEP flag for bare Ubuntu images vs pre-built images
  3. Documentation improvements

    • Add scripts/README.md with comprehensive setup instructions
    • Provide multiple usage examples showing partial parameters and environment variable usage

Features

  • Native builds: arm64 host building arm64 kernels
  • Cross-compilation: e.g., amd64 host building arm64 kernels
  • Automatic changelog restoration: prevents dirty tree state
  • Docker containerization: isolated build environment with preinstalled dependencies

@GuanquanTian
guanquan (GuanquanTian) force-pushed the build-kernel-deb-cross-improvements branch from f8af24b to cbbcfea Compare July 8, 2026 13:13

@bjordiscollaku Bjordis Collaku (bjordiscollaku) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the substantial effort here. I want to be direct about the architecture first, because it is the thing that decides this PR.

This repository already has a reproducible, org-owned, containerized build: build-kernel.yml drives qualcomm-linux/docker-pkg-build and builds inside ghcr.io/qualcomm-linux/pkg-builder:<suite>. scripts/build-kernel-deb.sh is, by our own docs, a legacy local helper that no workflow references. This PR grows that legacy script into a second, parallel build system with its own image, its own registry, and its own build logic. Two build systems drift, and the one added here is anchored to a personal Docker image the rest of us cannot reproduce or reach.

So the ask at the top is a redesign, not a patch. I am supportive of improving how we build, but I would strictly propose we standardize on native arm64 builds rather than cross-compilation. Our official CI build, the one that produces the packages we generate the nightly images from, already runs natively on an arm64 runner, so native arm64 is the path our shipping artifacts come from. I would rather invest in that single path than take on a cross toolchain and a second build system to maintain alongside it, each with its own failure modes. If we keep a local convenience wrapper at all, it should target the same official pkg-builder image.

Blocking items are marked inline. Requesting changes.

Comment thread scripts/docker-build-kernel.sh Outdated
if [ -z "${IMAGE:-}" ]; then
case "$CURRENT_ARCH" in
arm64)
IMAGE="${DOCKER_REGISTRY}/guanquan/kernel-build-docker:resolute-arm64-deps"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking. This hard-couples a public repository to one person's image: docker-registry.qualcomm.com/guanquan/kernel-build-docker. Three independent problems, each disqualifying on its own:

  1. It is a personal namespace on an internal registry. This is an external, public PR. A contributor following scripts/README.md cannot pull this, and the script then hard-exits at the docker pull. The documented happy path is unreachable for the audience it is written for.
  2. No Dockerfile ships with the PR. The image is opaque: we cannot audit its toolchain, rebuild it, or pin it. When this namespace is renamed or the tag is deleted, every documented build breaks with no recovery path. That is a bus factor of one.
  3. It duplicates infrastructure we already own. CI builds in ghcr.io/qualcomm-linux/pkg-builder:<suite> via qualcomm-linux/docker-pkg-build.

Fix: do not introduce a second image. Default to the official pkg-builder image the CI already standardizes on. If a distinct toolchain is genuinely needed for cross builds, extend that image and check its Dockerfile into this repo under the org namespace, built in CI. A build image referenced by committed tooling must itself be defined by committed, reviewable source.

Comment thread scripts/docker-build-kernel.sh Outdated
# Get the workspace root (two levels up from scripts/)
# First resolve the script's absolute directory, then go up two levels
SCRIPT_ABS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
WORKSPACE_DIR="$(cd "${SCRIPT_ABS}/../.." && pwd)"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WORKSPACE_DIR resolves to the grandparent of the repo (scripts/../..), and line 132 bind-mounts that entire directory read-write into a container that runs as root. The mount scope is inferred from filesystem layout, not chosen by the user. Clone this repo directly into $HOME and you have handed a root process in an opaque image read-write access to your whole home directory: SSH keys, unrelated repos, all of it. It also only works inside the exact sibling-checkout layout the new README prescribes; a normal clone breaks SOURCE_DIR resolution.

Fix: mount only what the build needs. Bind the source tree and a dedicated output directory, nothing else, and run with --user "$(id -u):$(id -g)" so artifacts are not left root-owned on the host. Derive the source root from git -C "$SOURCE_DIR" rev-parse --show-toplevel, never from a fixed two-levels-up heuristic.

Comment thread scripts/build-kernel-deb.sh Outdated
fi

log "Running debian/rules clean (generates debian/control and debian/changelog)..."
( cd "${SOURCE_DIR}" && env ${CROSS_ENV} fakeroot debian/rules clean ) \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ordering bug. debian/rules clean runs here, before apt-get build-dep below. For this tree, clean regenerates debian/control through the kernel packaging machinery, which needs fakeroot, dpkg-dev, kernel-wedge, rsync, and friends. Those are exactly the build dependencies that are not installed yet on the documented native path (SKIP_BUILD_DEP=0). The first thing this line does is call fakeroot, which does not exist until build-dep installs it, so the standalone path dies right here before it can install anything. The Docker path only survives because the prebaked image already contains these tools.

Fix: install the build prerequisites before clean, or gate the standalone path with a clear precondition. Better, drive the whole thing through dpkg-buildpackage, which orders this correctly and also emits the .changes and .buildinfo files that the collection step below globs for but never produces.

Comment thread scripts/build-kernel-deb.sh Outdated
if [ "${CROSS_BUILD}" = true ]; then
# For cross-compilation, llvm-21-dev needs :native qualifier.
log "Patching debian/control: mark llvm-21-dev as a native (build-arch) dependency..."
sed -i 's/^ llvm-21-dev,$/ llvm-21-dev:native,/' "${SOURCE_DIR}/debian/control"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two defects in one line. First, this hardcodes llvm-21-dev and anchors on exact spacing. sed returns success on a non-match, so the day this kernel bumps to llvm-22-dev, or debian/control is reindented, this silently no-ops and the cross build fails later with a message that never names the root cause. Second, this edit to debian/control is never reverted. The changelog edit above is carefully restored via the EXIT trap; control is not, and that trap is not even registered unless VERSION_SUFFIX is set. So the script's own promise that it "never leaves the tree dirty" is false for cross builds.

Fix: do not mutate a generated file with a version-pinned regex. Express the :native build-tool dependency in the packaging itself via build profiles, or derive the package name dynamically and hard-fail if the substitution changed nothing. And restore control on exit the same way changelog is, with the trap registered unconditionally.

Comment thread README.md Outdated

| Doc | For |
|-----|-----|
| **[scripts/README.md](scripts/README.md)** | Users - building kernel .deb packages locally |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This promotes a legacy, non-CI local script to a first-class, user-facing entry in the primary navigation, above INTEGRATION and PIPELINE. That misrepresents how this repository actually works. The model is mirror plus CI plus S3: the kernel is built by the pipeline and published to the bucket. Sending newcomers to a local build path that requires an unreachable personal image, and that (per scripts/README.md) then walks them into qdl hardware flashing, points people at the least reproducible path we have and frames it as the supported one.

Fix: do not advertise a local build as the user path. If this doc survives the architecture discussion at all, scope it to maintainer and debug use, and keep the index pointing users at the released packages and the pipeline.

@GuanquanTian
guanquan (GuanquanTian) force-pushed the build-kernel-deb-cross-improvements branch from cbbcfea to d04703c Compare July 13, 2026 17:50
…uisites before clean

- Detect a build/host arch mismatch and enable cross builds via dpkg
  foreign-architecture, the "cross" build profile, and
  DEB_HOST_ARCH/GNU_TYPE/MULTIARCH propagation.
- Factor debian/control generation and native-host-tool-dep patching into
  scripts/lib/gen-real-control.sh, shared with build-docker-image.sh.
- Install fakeroot/debhelper before running `debian/rules clean`, which
  requires both to run.
- Mark native-host build tools (e.g. llvm-*-dev) `:native` via a
  pattern-keyed NATIVE_HOST_TOOL_DEPS table, and restore debian/control
  from a backup on exit.
- Support OUTPUT_DIR override, and exit with an explicit error message
  when apt-get build-dep or dch need root but no sudo binary is available.
- Add binary-indep to the build targets so the arch:all
  linux-qcom-headers-* package is produced alongside the flavour packages.

Signed-off-by: Guanquan Tian <guanquan@qti.qualcomm.com>
…rfile

- Add scripts/Dockerfile.kernel-build, built FROM
  docker.io/library/ubuntu:resolute (falling back to
  public.ecr.aws/ubuntu/ubuntu:resolute), with build deps resolved
  dynamically from the kernel tree's own debian/control via
  `debian/rules clean`.
- Add scripts/build-docker-image.sh to drive it: runs `debian/rules clean`
  on the host, copies the generated debian/control into a scratch build
  context, and passes TARGET_ARCH/CROSS build args.
- The image is built from a Dockerfile checked into this repo.

Signed-off-by: Guanquan Tian <guanquan@qti.qualcomm.com>
…t uid/gid

- Build the local image on demand via build-docker-image.sh if not
  already present.
- Bind-mount the current directory, OUTPUT_DIR, SOURCE_DIR's parent
  (where debian/rules writes .deb/.changes/.buildinfo), and the script
  directory (read-only).
- Run the container as the invoking host uid/gid.
- Resolve SOURCE_DIR from the path passed in.
- Support OUTPUT_DIR override, created on the host before the container
  starts.

Signed-off-by: Guanquan Tian <guanquan@qti.qualcomm.com>
Add scripts/README.md covering build-kernel-deb.sh,
docker-build-kernel.sh, and build-docker-image.sh usage, arguments,
environment variables, and cross-compilation notes.

Signed-off-by: Guanquan Tian <guanquan@qti.qualcomm.com>
@GuanquanTian
guanquan (GuanquanTian) force-pushed the build-kernel-deb-cross-improvements branch from d04703c to b2dbc56 Compare July 13, 2026 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants