feat: add cross-compilation support and Docker containerized builds for kernel packaging#33
Conversation
f8af24b to
cbbcfea
Compare
There was a problem hiding this comment.
Thanks for the substantial effort here. I want to be direct about the architecture first, because it is the thing that decides this PR.
This repository already has a reproducible, org-owned, containerized build: build-kernel.yml drives qualcomm-linux/docker-pkg-build and builds inside ghcr.io/qualcomm-linux/pkg-builder:<suite>. scripts/build-kernel-deb.sh is, by our own docs, a legacy local helper that no workflow references. This PR grows that legacy script into a second, parallel build system with its own image, its own registry, and its own build logic. Two build systems drift, and the one added here is anchored to a personal Docker image the rest of us cannot reproduce or reach.
So the ask at the top is a redesign, not a patch. I am supportive of improving how we build, but I would strictly propose we standardize on native arm64 builds rather than cross-compilation. Our official CI build, the one that produces the packages we generate the nightly images from, already runs natively on an arm64 runner, so native arm64 is the path our shipping artifacts come from. I would rather invest in that single path than take on a cross toolchain and a second build system to maintain alongside it, each with its own failure modes. If we keep a local convenience wrapper at all, it should target the same official pkg-builder image.
Blocking items are marked inline. Requesting changes.
| if [ -z "${IMAGE:-}" ]; then | ||
| case "$CURRENT_ARCH" in | ||
| arm64) | ||
| IMAGE="${DOCKER_REGISTRY}/guanquan/kernel-build-docker:resolute-arm64-deps" |
There was a problem hiding this comment.
Blocking. This hard-couples a public repository to one person's image: docker-registry.qualcomm.com/guanquan/kernel-build-docker. Three independent problems, each disqualifying on its own:
- It is a personal namespace on an internal registry. This is an external, public PR. A contributor following
scripts/README.mdcannot pull this, and the script then hard-exits at thedocker pull. The documented happy path is unreachable for the audience it is written for. - No Dockerfile ships with the PR. The image is opaque: we cannot audit its toolchain, rebuild it, or pin it. When this namespace is renamed or the tag is deleted, every documented build breaks with no recovery path. That is a bus factor of one.
- It duplicates infrastructure we already own. CI builds in
ghcr.io/qualcomm-linux/pkg-builder:<suite>viaqualcomm-linux/docker-pkg-build.
Fix: do not introduce a second image. Default to the official pkg-builder image the CI already standardizes on. If a distinct toolchain is genuinely needed for cross builds, extend that image and check its Dockerfile into this repo under the org namespace, built in CI. A build image referenced by committed tooling must itself be defined by committed, reviewable source.
| # Get the workspace root (two levels up from scripts/) | ||
| # First resolve the script's absolute directory, then go up two levels | ||
| SCRIPT_ABS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
| WORKSPACE_DIR="$(cd "${SCRIPT_ABS}/../.." && pwd)" |
There was a problem hiding this comment.
WORKSPACE_DIR resolves to the grandparent of the repo (scripts/../..), and line 132 bind-mounts that entire directory read-write into a container that runs as root. The mount scope is inferred from filesystem layout, not chosen by the user. Clone this repo directly into $HOME and you have handed a root process in an opaque image read-write access to your whole home directory: SSH keys, unrelated repos, all of it. It also only works inside the exact sibling-checkout layout the new README prescribes; a normal clone breaks SOURCE_DIR resolution.
Fix: mount only what the build needs. Bind the source tree and a dedicated output directory, nothing else, and run with --user "$(id -u):$(id -g)" so artifacts are not left root-owned on the host. Derive the source root from git -C "$SOURCE_DIR" rev-parse --show-toplevel, never from a fixed two-levels-up heuristic.
| fi | ||
|
|
||
| log "Running debian/rules clean (generates debian/control and debian/changelog)..." | ||
| ( cd "${SOURCE_DIR}" && env ${CROSS_ENV} fakeroot debian/rules clean ) \ |
There was a problem hiding this comment.
Ordering bug. debian/rules clean runs here, before apt-get build-dep below. For this tree, clean regenerates debian/control through the kernel packaging machinery, which needs fakeroot, dpkg-dev, kernel-wedge, rsync, and friends. Those are exactly the build dependencies that are not installed yet on the documented native path (SKIP_BUILD_DEP=0). The first thing this line does is call fakeroot, which does not exist until build-dep installs it, so the standalone path dies right here before it can install anything. The Docker path only survives because the prebaked image already contains these tools.
Fix: install the build prerequisites before clean, or gate the standalone path with a clear precondition. Better, drive the whole thing through dpkg-buildpackage, which orders this correctly and also emits the .changes and .buildinfo files that the collection step below globs for but never produces.
| if [ "${CROSS_BUILD}" = true ]; then | ||
| # For cross-compilation, llvm-21-dev needs :native qualifier. | ||
| log "Patching debian/control: mark llvm-21-dev as a native (build-arch) dependency..." | ||
| sed -i 's/^ llvm-21-dev,$/ llvm-21-dev:native,/' "${SOURCE_DIR}/debian/control" |
There was a problem hiding this comment.
Two defects in one line. First, this hardcodes llvm-21-dev and anchors on exact spacing. sed returns success on a non-match, so the day this kernel bumps to llvm-22-dev, or debian/control is reindented, this silently no-ops and the cross build fails later with a message that never names the root cause. Second, this edit to debian/control is never reverted. The changelog edit above is carefully restored via the EXIT trap; control is not, and that trap is not even registered unless VERSION_SUFFIX is set. So the script's own promise that it "never leaves the tree dirty" is false for cross builds.
Fix: do not mutate a generated file with a version-pinned regex. Express the :native build-tool dependency in the packaging itself via build profiles, or derive the package name dynamically and hard-fail if the substitution changed nothing. And restore control on exit the same way changelog is, with the trap registered unconditionally.
|
|
||
| | Doc | For | | ||
| |-----|-----| | ||
| | **[scripts/README.md](scripts/README.md)** | Users - building kernel .deb packages locally | |
There was a problem hiding this comment.
This promotes a legacy, non-CI local script to a first-class, user-facing entry in the primary navigation, above INTEGRATION and PIPELINE. That misrepresents how this repository actually works. The model is mirror plus CI plus S3: the kernel is built by the pipeline and published to the bucket. Sending newcomers to a local build path that requires an unreachable personal image, and that (per scripts/README.md) then walks them into qdl hardware flashing, points people at the least reproducible path we have and frames it as the supported one.
Fix: do not advertise a local build as the user path. If this doc survives the architecture discussion at all, scope it to maintainer and debug use, and keep the index pointing users at the released packages and the pipeline.
cbbcfea to
d04703c
Compare
…uisites before clean - Detect a build/host arch mismatch and enable cross builds via dpkg foreign-architecture, the "cross" build profile, and DEB_HOST_ARCH/GNU_TYPE/MULTIARCH propagation. - Factor debian/control generation and native-host-tool-dep patching into scripts/lib/gen-real-control.sh, shared with build-docker-image.sh. - Install fakeroot/debhelper before running `debian/rules clean`, which requires both to run. - Mark native-host build tools (e.g. llvm-*-dev) `:native` via a pattern-keyed NATIVE_HOST_TOOL_DEPS table, and restore debian/control from a backup on exit. - Support OUTPUT_DIR override, and exit with an explicit error message when apt-get build-dep or dch need root but no sudo binary is available. - Add binary-indep to the build targets so the arch:all linux-qcom-headers-* package is produced alongside the flavour packages. Signed-off-by: Guanquan Tian <guanquan@qti.qualcomm.com>
…rfile - Add scripts/Dockerfile.kernel-build, built FROM docker.io/library/ubuntu:resolute (falling back to public.ecr.aws/ubuntu/ubuntu:resolute), with build deps resolved dynamically from the kernel tree's own debian/control via `debian/rules clean`. - Add scripts/build-docker-image.sh to drive it: runs `debian/rules clean` on the host, copies the generated debian/control into a scratch build context, and passes TARGET_ARCH/CROSS build args. - The image is built from a Dockerfile checked into this repo. Signed-off-by: Guanquan Tian <guanquan@qti.qualcomm.com>
…t uid/gid - Build the local image on demand via build-docker-image.sh if not already present. - Bind-mount the current directory, OUTPUT_DIR, SOURCE_DIR's parent (where debian/rules writes .deb/.changes/.buildinfo), and the script directory (read-only). - Run the container as the invoking host uid/gid. - Resolve SOURCE_DIR from the path passed in. - Support OUTPUT_DIR override, created on the host before the container starts. Signed-off-by: Guanquan Tian <guanquan@qti.qualcomm.com>
Add scripts/README.md covering build-kernel-deb.sh, docker-build-kernel.sh, and build-docker-image.sh usage, arguments, environment variables, and cross-compilation notes. Signed-off-by: Guanquan Tian <guanquan@qti.qualcomm.com>
d04703c to
b2dbc56
Compare
Summary
This PR enhances the kernel build infrastructure with cross-compilation support and Docker containerization:
build-kernel-deb.sh improvements
autovalue)docker-build-kernel.sh wrapper script
Documentation improvements
Features