diff --git a/docs/android-redroid-runbook.md b/docs/android-redroid-runbook.md new file mode 100644 index 0000000..0583484 --- /dev/null +++ b/docs/android-redroid-runbook.md @@ -0,0 +1,172 @@ +# Running Android (redroid) inside a matchlock sandbox VM + +Proven end-to-end 2026-08-04: Android 15 arm64 boots inside an rsv2 sandbox VM, +the RSHomebuyerApp debug APK builds *inside the same VM*, and the app signs into +that instance's Rails — no emulator binary, no KVM, no host-side Android +tooling. This runbook records the full recipe and every trap we hit. + +## Why redroid and not the emulator + +Google ships **no linux-aarch64 Android emulator** in the SDK channel +(repository2-3.xml has linux_x64 / darwin_x64 / darwin_aarch64 / windows_x64 +only). A CI build (`emulator-linux_aarch64_gfxstream`) exists on ci.android.com +but is unfetchable without authenticated `fetch_artifact`. Combined with no +`/dev/kvm` in the guest, emulator-based Android is a dead end. redroid runs +Android as a container on the guest kernel at native ARM speed. + +## 1. Kernel (this PR) + +redroid needs guest-kernel features the matchlock kernel didn't have. All are +`=y` (CONFIG_MODULES=n): + +- `CONFIG_ANDROID_BINDER_IPC` / `CONFIG_ANDROID_BINDERFS` / + `CONFIG_ANDROID_BINDER_DEVICES="binder,hwbinder,vndbinder"` — Android IPC. + binderfs allows multiple Android instances per kernel. ashmem is gone since + 5.18; modern redroid needs binder only. +- `CONFIG_BINFMT_MISC` — lets qemu-user run the x86_64-only Android SDK + tooling (aapt2, NDK clang) in the arm64 guest. +- `CONFIG_SECURITY` + `SECURITY_NETWORK` + `AUDIT` + `SECURITY_SELINUX` + + `SELINUX_DEVELOP` — Android init expects /sys/fs/selinux; DEVELOP boots + permissive. SECURITY_SELINUX's Kconfig deps drag the rest in. +- `CONFIG_MEMCG_V1` / `CPUSETS_V1` — Android's libprocessgroup sets task + profiles against cgroup v1 (/dev/cpuctl, /dev/cpuset). Note 6.19 has **no** + `CONFIG_CGROUP_V1`; that symbol was sitting in the config doing nothing. + +`olddefconfig` drops unknown/unmet options **silently** — we found two phantom +symbols (`CONFIG_CGROUP_V1`, `CONFIG_SELINUX`) that read as authoritative but +did nothing. `guest/kernel/verify-config.sh` now fails the Docker kernel build +on any *new* drift; `known-drift.txt` holds the untriaged pre-existing drops. + +Build: `KERNEL_VERSION=6.19.8-dmcrypt-vfs-binder2 KERNEL_SOURCE_VERSION=6.19.8 +ARCH=arm64 ./scripts/build-kernel.sh` → lands in +`~/.cache/matchlock/kernels//kernel-arm64`. + +## 2. Boot an instance on the new kernel (no fleet impact) + +rsv2 pins the kernel in `qa/rsv2-cli/cmd/setup.go` (`sandboxKernelVersion`); +per-launch overrides avoid touching the pin: + +```sh +RSV2_SANDBOX_KERNEL=$HOME/.cache/matchlock/kernels/6.19.8-dmcrypt-vfs-binder2/kernel-arm64 \ +RSV2_SANDBOX_MEMORY_MB=9216 \ +rsv2 sandbox 1 --detach +``` + +Restarting a VM destroys `/opt` and `/root` (ephemeral overlay). Salvage +first, over virtiofs, from a detached container so the exec channel stays free: + +```sh +matchlock exec -- sh -c 'docker run -d --name salvage \ + -v /workspace/app/tmp:/out -v /opt:/s/opt:ro -v /root/.gradle:/s/root/.gradle:ro \ + alpine tar -C /s -cf /out/toolchain.tar opt/android-sdk opt/node20 opt/work root/.gradle' +``` + +**Per-VM CA trap:** every VM boot mints a fresh matchlock MITM CA. A salvaged +Java cacerts carries the old CA and Gradle fails with PKIX "signature check +failed" (while curl works — it uses the system store guest-init refreshes). +After any VM swap, re-import: + +```sh +keytool -delete -cacerts -storepass changeit -alias matchlock +keytool -importcert -noprompt -cacerts -storepass changeit -alias matchlock \ + -file /etc/ssl/certs/matchlock-ca.crt +``` + +## 3. redroid + +```sh +docker run -d --privileged --name redroid -v /data/redroid:/data -p 5555:5555 \ + redroid/redroid:15.0.0_64only-latest androidboot.redroid_gpu_mode=guest +``` + +- Boots to `sys.boot_completed=1` in ~2 min with ≥4 CPU / ≥6 GB (it failed + mysteriously at 1 CPU / 512 MB — check resources before debugging Android). +- **Black screencap ≠ GPU bug.** redroid starts no Home app, so nothing draws. + `am start -a android.intent.action.MAIN -c android.intent.category.HOME` + starts the launcher. Read the GPU mode back as `ro.boot.redroid_gpu_mode` + (NOT `redroid.gpu.mode`). Byte-size check: black 720x1280 PNG ≈ 6 KB, real + content ≥ 50 KB. +- **redroid's netd programs no kernel routes** — policy-routing tables 1003/99 + are empty, so every app socket dies ENETUNREACH. Fix inside the container: + + ```sh + ip rule add from all lookup main pref 20000 + ip route add default via 172.17.0.1 dev eth0 onlink # only for egress + ``` + +- No adb needed: `docker cp` + `docker exec redroid pm install`, `am start`, + `input tap/text`, `screencap -p | base64`. +- No Google Play services (AOSP image): the app's map view is dead. Everything + else works. A GApps/microG redroid variant exists if maps ever matter. + +## 4. In-VM networking (the shield changes the rules) + +- Containers **cannot** reach VM host-netns ports (shield INPUT chain) and + **cannot** hairpin to published ports via the gateway. Same-bridge + container-to-container traffic is the only reliable path. +- Put everything the app needs on redroid's bridge: + `docker network connect bridge rsv2-qa-1-web-1`, run Metro as a bridge + container. Talk container-IP to container-IP. +- Rails host authorization only admits `.rslocal.test` — point the name at the + web container's bridge IP inside redroid (`/system/etc/hosts` works; bionic + reads it) and give the app `http://www.rslocal.test:3000`. + +## 5. Building the APK inside the VM + +- **binfmt_misc is per-user-namespace, and every matchlock exec AND every + docker container here gets its own instance.** A qemu registration made + anywhere else is invisible to the build. Register inside the same exec that + runs Gradle (F flag pins the interpreter at registration): + + ```sh + mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc + printf ':qemu-x86_64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x3e\x00:\xff\xff\xff\xff\xff\xfe\xfe\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-x86_64-static:F\n' \ + > /proc/sys/fs/binfmt_misc/register + ``` + +- Dynamic x86_64 binaries need amd64 multiarch libs: + `dpkg --add-architecture amd64`, add an archive.ubuntu.com amd64 source + (arm64 stays on ports.ubuntu.com), then + `apt-get install libc6:amd64 libstdc++6:amd64 zlib1g:amd64`. dpkg's + configure step fails without VM-wide binfmt — harmless; the unpacked files + are what matters. +- AGP demands the aapt2 override be a file named exactly `aapt2` + (`android.aapt2FromMavenOverride=/opt/aapt2-shim/aapt2`, a one-line + qemu-exec wrapper). +- `reactNativeArchitectures=arm64-v8a` in gradle.properties — the default + builds 4 ABIs; under emulation that's 4x the C++ for nothing. +- Shield allowlist needed: `dl.google.com maven.google.com + repo.maven.apache.org plugins.gradle.org plugins-artifacts.gradle.org + services.gradle.org archive.ubuntu.com ports.ubuntu.com` + (`matchlock allow-list add ...`). +- Run the build as ONE long exec logging to virtiofs + (`> /workspace/app/tmp/build.log`) and read the log from the host — execs + serialize, and polling from a second exec collides with the build. +- Expo debug builds bundle no JS: run Metro + (`npx expo start --offline --port 8081`) as a bridge-network container and + aim the dev client with + `am start -a android.intent.action.VIEW -d + "rshomebuyerapp://expo-development-client/?url=http%3A%2F%2F%3A8081"`. + +## 6. Demo-data gotchas (rsv2 seed wizard) + +- Seeded listings live in one MLS (RECSFAR here, all "san francisco") — a user + whose saved searches target another MLS sees zero matches. jordan.reyes's QA + searches target RECSFAR; harry.homebuyer's don't. +- Favorites are `Verdict` rows (`prose: 'saved'`); create them via Rails + console to pre-populate the interested tab. +- The seed generator writes **relative** `/stub_photos/...` photo URLs — fine + in a browser, opaque to React Native's `Image`. Absolutize in ES + (`_update_by_query` prepending `http://www.rslocal.test:3000`) and photos + render, served by the in-VM web container. + +## Next direction: Rosetta instead of qemu + +The one real tax left is qemu TCG emulation of the x86_64 NDK toolchain. +Virtualization.framework can inject Apple's Rosetta into a Linux guest as a +read-only directory share (`VZLinuxRosettaDirectoryShare`); registering +`/media/rosetta/rosetta` as the x86_64 binfmt handler gives near-native speed — +the mechanism Docker Desktop uses for fast amd64 containers. Security-neutral +relative to qemu: translation runs entirely in-guest, no host channel, one more +read-only virtiofs device. Requires host-side plumbing in the VZ config, which +we own in this fork. diff --git a/docs/android/build-apk.sh b/docs/android/build-apk.sh new file mode 100644 index 0000000..69aa68d --- /dev/null +++ b/docs/android/build-apk.sh @@ -0,0 +1,20 @@ +#!/bin/sh +# Runs inside one matchlock exec. binfmt_misc instances are per-userns and +# every exec gets a fresh userns, so the qemu registration must happen in +# the same exec that runs the build — it cannot be done once VM-wide. +LOG=/workspace/app/tmp/apk-build.log +{ + set -ex + mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc 2>/dev/null || true + printf ':qemu-x86_64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x3e\x00:\xff\xff\xff\xff\xff\xfe\xfe\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-x86_64-static:F\n' > /proc/sys/fs/binfmt_misc/register + /opt/android-sdk/build-tools/36.0.0/aapt2 version + export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-arm64 + export ANDROID_HOME=/opt/android-sdk + export ANDROID_SDK_ROOT=/opt/android-sdk + export PATH="/opt/node20/bin:$JAVA_HOME/bin:$PATH" + node --version + cd /opt/work/RSHomebuyerApp/android + ./gradlew assembleDebug + cp app/build/outputs/apk/debug/app-debug.apk /workspace/app/tmp/RSHomebuyerApp-debug.apk + echo BUILD_SCRIPT_DONE +} > "$LOG" 2>&1 diff --git a/docs/android/register-binfmt.sh b/docs/android/register-binfmt.sh new file mode 100644 index 0000000..36e0703 --- /dev/null +++ b/docs/android/register-binfmt.sh @@ -0,0 +1,16 @@ +#!/bin/sh +# Register qemu-x86_64 as the binfmt_misc handler for x86_64 ELF binaries. +# Must run in the VM's init user namespace (e.g. a privileged docker +# container) — matchlock execs get their own userns, whose binfmt instance +# evaporates with the exec. The F flag pins the interpreter at registration +# so callers never need /usr/bin/qemu-x86_64-static in their own mount ns. +set -e +mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc 2>/dev/null || true +if [ -e /proc/sys/fs/binfmt_misc/qemu-x86_64 ]; then + echo "already registered:" + head -3 /proc/sys/fs/binfmt_misc/qemu-x86_64 + exit 0 +fi +printf ':qemu-x86_64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x3e\x00:\xff\xff\xff\xff\xff\xfe\xfe\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-x86_64-static:F\n' > /proc/sys/fs/binfmt_misc/register +echo "registered:" +head -3 /proc/sys/fs/binfmt_misc/qemu-x86_64 diff --git a/guest/kernel/Dockerfile b/guest/kernel/Dockerfile index 1580095..17ddf5a 100644 --- a/guest/kernel/Dockerfile +++ b/guest/kernel/Dockerfile @@ -44,10 +44,14 @@ RUN set -eux; \ # --------------------------------------------------------------------------- FROM base AS build-x86_64 -COPY x86_64.config /tmp/linux-src/.config +COPY x86_64.config /tmp/requested.config +COPY verify-config.sh /usr/local/bin/verify-config.sh +COPY known-drift.txt /tmp/known-drift.txt RUN cd /tmp/linux-src \ + && cp /tmp/requested.config .config \ && make olddefconfig \ + && sh /usr/local/bin/verify-config.sh /tmp/requested.config .config /tmp/known-drift.txt \ && make -j"$(nproc)" vmlinux # --------------------------------------------------------------------------- @@ -55,10 +59,14 @@ RUN cd /tmp/linux-src \ # --------------------------------------------------------------------------- FROM base AS build-arm64 -COPY arm64.config /tmp/linux-src/.config +COPY arm64.config /tmp/requested.config +COPY verify-config.sh /usr/local/bin/verify-config.sh +COPY known-drift.txt /tmp/known-drift.txt RUN cd /tmp/linux-src \ + && cp /tmp/requested.config .config \ && make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- olddefconfig \ + && sh /usr/local/bin/verify-config.sh /tmp/requested.config .config /tmp/known-drift.txt \ && make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j"$(nproc)" Image # --------------------------------------------------------------------------- diff --git a/guest/kernel/arm64.config b/guest/kernel/arm64.config index 92afae3..1a4d1aa 100644 --- a/guest/kernel/arm64.config +++ b/guest/kernel/arm64.config @@ -142,6 +142,26 @@ CONFIG_BLK_DEV_INITRD=y CONFIG_RD_GZIP=y CONFIG_BINFMT_ELF=y CONFIG_BINFMT_SCRIPT=y +# Lets qemu-user register foreign-arch binaries, so x86_64-only vendor tooling +# (Android SDK aapt2/adb ship no linux-aarch64 build) runs in an arm64 guest. +CONFIG_BINFMT_MISC=y + +# Android binder — runs containerised Android (redroid) in the guest, which +# needs no KVM and no vendor emulator binary. Both are bool, so =y is required +# under CONFIG_MODULES=n. +CONFIG_ANDROID_BINDER_IPC=y +CONFIG_ANDROID_BINDERFS=y +CONFIG_ANDROID_BINDER_DEVICES="binder,hwbinder,vndbinder" + +# SELinux — Android's init and system_server expect /sys/fs/selinux to exist. +# SECURITY_SELINUX depends on SECURITY_NETWORK && AUDIT && NET && INET, so all +# of these have to come along; SELINUX_DEVELOP boots permissive, which is what +# we want (Android supplies its own policy, we are not enforcing a host one). +CONFIG_SECURITY=y +CONFIG_SECURITY_NETWORK=y +CONFIG_AUDIT=y +CONFIG_SECURITY_SELINUX=y +CONFIG_SECURITY_SELINUX_DEVELOP=y # Kernel options CONFIG_PREEMPT_NONE=y @@ -178,7 +198,11 @@ CONFIG_BLK_CGROUP=y CONFIG_CGROUP_WRITEBACK=y CONFIG_CGROUP_BPF=y CONFIG_CGROUP_CPUACCT=y -CONFIG_CGROUP_V1=y +# 6.19 has no CONFIG_CGROUP_V1 — the legacy hierarchies split per controller. +# Android's libprocessgroup sets task profiles against cgroup v1 (/dev/cpuctl, +# /dev/cpuset), so redroid needs these. +CONFIG_MEMCG_V1=y +CONFIG_CPUSETS_V1=y # Namespaces (all types required for rootless BuildKit / runc) CONFIG_NAMESPACES=y @@ -235,9 +259,6 @@ CONFIG_DEBUG_INFO=n CONFIG_DEBUG_KERNEL=n CONFIG_WATCHDOG=n CONFIG_INPUT=n -CONFIG_SELINUX=n -CONFIG_SECURITY=n -CONFIG_AUDIT=n CONFIG_PROFILING=n CONFIG_WEXT_CORE=n CONFIG_WEXT_PROC=n diff --git a/guest/kernel/known-drift.txt b/guest/kernel/known-drift.txt new file mode 100644 index 0000000..c1b1661 --- /dev/null +++ b/guest/kernel/known-drift.txt @@ -0,0 +1,34 @@ +# Options that olddefconfig already drops from the configs on 6.19.8, from +# before verify-config.sh existed — so the kernels we ship today are already +# built without them. Listed here to keep the build moving while still failing +# on any NEW drift. +# +# These have NOT been triaged. Two different things are mixed together: +# - Options that no longer exist on this arch/version (EXT4_FS_XATTR was +# folded into ext4; EARLY_PRINTK is largely x86) — harmless. +# - Options with genuinely unmet dependencies. The netfilter entries are +# this kind, and Docker NAT depends on them. +# Shrinking this list is the actual fix; every line is a known gap. +# +# One entry has already been triaged off this list: CONFIG_CGROUP_V1 was a +# symbol that does not exist in 6.19 (the legacy hierarchies split into +# per-controller MEMCG_V1/CPUSETS_V1), so it read as authoritative while +# silently doing nothing. Expect others here to be the same kind of phantom — +# check the symbol exists for this version before assuming it is a real gap. + +CONFIG_IP_NF_FILTER +CONFIG_IP_NF_MANGLE +CONFIG_IP_NF_RAW +CONFIG_IP_NF_NAT +CONFIG_IP_NF_TARGET_MASQUERADE +CONFIG_IP_NF_TARGET_REDIRECT +CONFIG_IP6_NF_FILTER +CONFIG_IP6_NF_MANGLE +CONFIG_IP6_NF_RAW +CONFIG_IP6_NF_NAT +CONFIG_IP6_NF_TARGET_MASQUERADE +CONFIG_EXT4_FS_XATTR +CONFIG_EARLY_PRINTK +CONFIG_ARCH_VIRT +CONFIG_HYPERVISOR_GUEST +CONFIG_BLK_MQ_VIRTIO diff --git a/guest/kernel/verify-config.sh b/guest/kernel/verify-config.sh new file mode 100755 index 0000000..b0f5cc1 --- /dev/null +++ b/guest/kernel/verify-config.sh @@ -0,0 +1,53 @@ +#!/bin/sh +# Fail the kernel build when olddefconfig silently drops a requested option. +# +# olddefconfig resolves unmet dependencies by dropping the option rather than +# erroring, so a config can build "successfully" without features the guest +# needs — this bit us with netfilter (Docker NAT) on 6.19.8. Comparing the +# requested config against the resolved .config is the only way to catch it. +# +# Usage: verify-config.sh [known-drift.txt] +# +# known-drift.txt lists options already being dropped before this check +# existed. They are reported but do not fail the build, so pre-existing gaps +# stay visible without blocking every kernel build on a backlog. + +set -eu + +requested="$1" +resolved="$2" +drift="${3:-}" + +known() { + [ -n "$drift" ] && [ -f "$drift" ] || return 1 + grep -v '^#' "$drift" | grep -qxF "$1" +} + +missing="" +tolerated="" +while IFS= read -r line; do + grep -qxF "$line" "$resolved" && continue + if known "${line%=y}"; then + tolerated="${tolerated} ${line} +" + else + missing="${missing}${line} +" + fi +done <&2 + printf '%s' "$missing" >&2 + echo "Check each option's Kconfig dependencies against this kernel version." >&2 + exit 1 +fi + +echo "verify-config: all requested =y options survived olddefconfig." diff --git a/guest/kernel/x86_64.config b/guest/kernel/x86_64.config index 11ea9d8..abddc02 100644 --- a/guest/kernel/x86_64.config +++ b/guest/kernel/x86_64.config @@ -148,6 +148,26 @@ CONFIG_BLK_DEV_INITRD=y CONFIG_RD_GZIP=y CONFIG_BINFMT_ELF=y CONFIG_BINFMT_SCRIPT=y +# Lets qemu-user register foreign-arch binaries, so aarch64-only tooling runs +# in an x86_64 guest. +CONFIG_BINFMT_MISC=y + +# Android binder — runs containerised Android (redroid) in the guest, which +# needs no KVM and no vendor emulator binary. Both are bool, so =y is required +# under CONFIG_MODULES=n. +CONFIG_ANDROID_BINDER_IPC=y +CONFIG_ANDROID_BINDERFS=y +CONFIG_ANDROID_BINDER_DEVICES="binder,hwbinder,vndbinder" + +# SELinux — Android's init and system_server expect /sys/fs/selinux to exist. +# SECURITY_SELINUX depends on SECURITY_NETWORK && AUDIT && NET && INET, so all +# of these have to come along; SELINUX_DEVELOP boots permissive, which is what +# we want (Android supplies its own policy, we are not enforcing a host one). +CONFIG_SECURITY=y +CONFIG_SECURITY_NETWORK=y +CONFIG_AUDIT=y +CONFIG_SECURITY_SELINUX=y +CONFIG_SECURITY_SELINUX_DEVELOP=y # Kernel options CONFIG_PREEMPT_NONE=y @@ -184,7 +204,11 @@ CONFIG_BLK_CGROUP=y CONFIG_CGROUP_WRITEBACK=y CONFIG_CGROUP_BPF=y CONFIG_CGROUP_CPUACCT=y -CONFIG_CGROUP_V1=y +# 6.19 has no CONFIG_CGROUP_V1 — the legacy hierarchies split per controller. +# Android's libprocessgroup sets task profiles against cgroup v1 (/dev/cpuctl, +# /dev/cpuset), so redroid needs these. +CONFIG_MEMCG_V1=y +CONFIG_CPUSETS_V1=y # Namespaces (all types required for rootless BuildKit / runc) CONFIG_NAMESPACES=y @@ -232,9 +256,6 @@ CONFIG_DEBUG_INFO=n CONFIG_DEBUG_KERNEL=n CONFIG_WATCHDOG=n CONFIG_INPUT=n -CONFIG_SELINUX=n -CONFIG_SECURITY=n -CONFIG_AUDIT=n CONFIG_PROFILING=n CONFIG_WEXT_CORE=n CONFIG_WEXT_PROC=n diff --git a/repro-kernel-config-drift.sh b/repro-kernel-config-drift.sh new file mode 100755 index 0000000..8556ec9 --- /dev/null +++ b/repro-kernel-config-drift.sh @@ -0,0 +1,104 @@ +#!/usr/bin/env bash +# Reproduction: kernel 6.19.8 olddefconfig silently drops netfilter options +# required for Docker bridge/NAT networking. +# +# This runs inside Docker — no host toolchain needed. +# Requires: Docker with BuildKit. +# +# Expected result: +# 6.1.137: all 6 iptables options preserved after olddefconfig +# 6.19.8: all 6 iptables options silently dropped after olddefconfig +# +# See: https://github.com/jingkaihe/matchlock/issues/XXX + +set -euo pipefail + +CONFIG_DIR="$(cd "$(dirname "$0")/guest/kernel" && pwd)" +ARM64_CONFIG="$CONFIG_DIR/arm64.config" + +if [ ! -f "$ARM64_CONFIG" ]; then + echo "ERROR: arm64.config not found at $ARM64_CONFIG" >&2 + exit 1 +fi + +# Explicitly set in arm64.config, and all `default n` with nothing selecting +# them — so olddefconfig will drop them silently rather than erroring. +# Netfilter: Docker iptables-legacy NAT/filter/mangle support. +# Binder: containerised Android (redroid) in the guest. +# BINFMT_MISC: qemu-user registration for foreign-arch binaries. +# CONFIG_ANDROID_BINDER_DEVICES is omitted deliberately — it is a string +# option, not =y, and its upstream default is already the value we want. +CRITICAL_OPTIONS=( + CONFIG_IP_NF_FILTER + CONFIG_IP_NF_MANGLE + CONFIG_IP_NF_NAT + CONFIG_IP6_NF_FILTER + CONFIG_IP6_NF_MANGLE + CONFIG_IP6_NF_NAT + CONFIG_BINFMT_MISC + CONFIG_ANDROID_BINDER_IPC + CONFIG_ANDROID_BINDERFS +) + +# Derived so the grep below cannot drift from the list above. +CRITICAL_REGEX=$(IFS='|'; echo "${CRITICAL_OPTIONS[*]}") + +check_kernel() { + local version="$1" + local source_version="$version" + + # kernel.org strips trailing .0 from URLs + case "$source_version" in + *.*.0) source_version="${source_version%.0}" ;; + esac + + echo "" + echo "=== Kernel $version ===" + + local result + result=$(docker run --rm \ + -v "$ARM64_CONFIG:/config/arm64.config:ro" \ + ubuntu:22.04 bash -c " + apt-get update -qq >/dev/null 2>&1 + apt-get install -y -qq --no-install-recommends \ + build-essential bc flex bison libelf-dev libssl-dev wget ca-certificates \ + gcc-aarch64-linux-gnu >/dev/null 2>&1 + + wget -qO /tmp/linux.tar.xz \ + https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-${source_version}.tar.xz + mkdir /tmp/linux-src + tar xf /tmp/linux.tar.xz -C /tmp/linux-src --strip-components=1 + + cp /config/arm64.config /tmp/linux-src/.config + cd /tmp/linux-src + make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- olddefconfig >/dev/null 2>&1 + + grep -E '^(${CRITICAL_REGEX})=' .config || true + " 2>&1) + + echo "$result" + + local dropped=0 + for opt in "${CRITICAL_OPTIONS[@]}"; do + if ! echo "$result" | grep -q "${opt}=y"; then + echo " DROPPED: ${opt}" + dropped=$((dropped + 1)) + fi + done + + if [ $dropped -eq 0 ]; then + echo " All ${#CRITICAL_OPTIONS[@]} critical options preserved." + else + echo " $dropped/${#CRITICAL_OPTIONS[@]} critical options silently dropped." + fi +} + +echo "Reproduction: arm64.config default-n options dropped by olddefconfig on 6.19.8" +echo "Config: $ARM64_CONFIG" + +check_kernel "6.1.137" +check_kernel "6.19.8" + +echo "" +echo "If 6.19.8 shows dropped options, the bug is confirmed." +echo "Docker fails with: iptables ... can't initialize iptables table 'nat'"