fw-download fails on 16 KiB-page ARM64 kernel without HugeTLB/THP

[m4r1k]

Problem

nvme fw-download exits with status 1 when given a firmware image of 512 KiB or larger on my Raspberry Pi 5. This kernel uses 16 KiB base pages and has no HugeTLB or THP support. The failure happens before nvme-cli sends a Firmware Image Download command to the controller.

This differs from #2848 because I cannot reserve huge pages as suggested in #2853. /sys/kernel/mm/hugepages does not exist, and /proc/meminfo has no HugePages or Hugepagesize entries.

Environment

$ nvme version
nvme version 2.13 (git 2.13)
libnvme version 1.13 (git 1.13)

$ uname -a
Linux homeassistant.lan 6.18.39+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.18.39-1+rpt1 (2026-07-29) aarch64 GNU/Linux

$ getconf PAGE_SIZE
16384

$ ls /sys/kernel/mm/hugepages
ls: cannot access '/sys/kernel/mm/hugepages': No such file or directory

The firmware image is 2 MiB. I checked tags v2.14, v2.15, and v2.16. The same allocation code is present in all three. It is also present in current 3.0 development under libnvme/src/nvme/mem-linux.c.

Reproduction

# nvme fw-download /dev/nvme0 -f firmware.bin -x 16384
# echo $?
1

Version 2.13 prints no error message. strace shows the failed allocations:

mmap(NULL, 2097152, PROT_READ|PROT_WRITE,
     MAP_PRIVATE|MAP_ANONYMOUS|MAP_HUGETLB, -1, 0) = -1 EINVAL (Invalid argument)
mmap(NULL, 4210688, PROT_READ|PROT_WRITE,
     MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x...
madvise(0x..., 2097152, MADV_HUGEPAGE) = -1 EINVAL (Invalid argument)
+++ exited with 1 +++

nvme_alloc_huge() treats the failed advisory madvise() call as fatal and frees the ordinary allocation. The function is now named libnvme_alloc_huge() in the 3.0 branch. The discussion in #2128 says that MADV_HUGEPAGE should not fail, but it returns EINVAL on this configuration.

Validated workaround

I split the image into eight 256 KiB files so that each invocation stayed below HUGE_MIN. I then downloaded the pieces in order, passing the offset in DWORDs:

nvme fw-download /dev/nvme0 -f part-00 -x 16384 -O 0
nvme fw-download /dev/nvme0 -f part-01 -x 16384 -O 65536
nvme fw-download /dev/nvme0 -f part-02 -x 16384 -O 131072
# ... through part-07 at offset 458752

All eight commands returned Firmware download success. The controller accepted the assembled image with nvme fw-commit -s 2 -a 0, and the firmware activated successfully after a cold reset.

For this controller and firmware, nvme-cli did not need to map the whole file or use physically contiguous huge pages. Sending smaller sections with the correct offsets worked.

Suggested behavior

I see two possible fixes:

1. Keep the aligned ordinary allocation when madvise(..., MADV_HUGEPAGE) fails, then let the kernel's DMA mapping determine whether it can use the buffer.
2. Read and send the firmware in transfer-sized sections instead of allocating the whole image at once.

If the allocator cannot be changed, the documentation should mention that some kernels do not expose nr_hugepages and show how to perform a segmented download.

Related: #2848, #2850, #2853, #2128.

[igaw]

I'd say we should make the madvise fail not fatal and continue trying to the kernel to handle the mapping. If this also fails, we could do try the chunked transfer as well. I think the fallback to chucked transfer should be a opt-in thing or at least it should be possible for the user to enable/disable it, this follows the existing patterns of nvme-cli.