hi3516cv6xx: add the lite variant, and drop the digital TV stack to f… - #2447
Conversation
PR Summary by QodoAdd 8 MiB hi3516cv6xx lite image and trim unused kernel features
AI Description
Diagram
High-Level Assessment
Files changed (3)
|
Code Review by Qodo
1. Lite boards get no full flash image
|
![qodo-free-for-open-source-projects[bot]](https://avatars.githubusercontent.com/in/1068000?s=60&v=4){kind=small}
135a055 to
5c9bf61
Compare
5c9bf61 to
9e748ff
Compare
|
Code review by qodo was updated up to the latest commit 9e748ff |
openipc-ai
left a comment
openipc-ai
left a comment
There was a problem hiding this comment.
The media diagnosis is the good part of this and I could reproduce the reasoning: on stock 5.10 the five MEDIA_*_SUPPORT classes carry a prompt only under MEDIA_SUPPORT_FILTER and otherwise default y, so the BSP expansion really did turn 146 TV drivers on by itself. REGMAP_I2C falling out with them checks out too -- olddefconfig would have kept it if anything else selected it.
Gates pass here: ci-matrix.py --self-test -> ok (100 boards, 136 packages, 56 cases), and the kernel config path narrows to both cv6xx boards, so ultimate does get built. EXTREME_COMP -> -Xbcj arm,armthumb is confirmed in fs/squashfs/Config.in and the kernel has XZ_DEC_ARM/XZ_DEC_ARMTHUMB/XZ_DEC_BCJ, so that 52 KiB is free as claimed.
The thing the hardware test cannot see is the bootloader, which the PR is honest about. I decompressed the cv610 u-boot this firmware ships with (boot-hi3516cv610-*-nor.bin, a HiSilicon boot table plus a gzip member at offset 40128) and its compiled-in env is:
kernaddr=0x50000 kernsize=0x200000 rootaddr=0x250000 rootsize=0x500000
mtdparts=sfc:256k(boot),64k(env),2048k(kernel),5120k(rootfs),7168k@0x50000(firmware),-(rootfs_data)
bootnor=... sf read ${baseaddr} ${kernaddr} ${kernsize}; bootm ${baseaddr}
bootargs=... root=/dev/mtdblock3 ...
Nothing rewrites it later -- set_allocator only edits the mmz half of bootargs. Against this build's own numbers:
| this PR | partition | |
|---|---|---|
| fitImage | 2496 KiB | 2048 KiB ❌ |
| rootfs.squashfs | 5184 KiB | 5120 KiB ❌ |
| firmware.bin | 7680 KiB | 7168 KiB ❌ |
The tested camera does not exercise any of this: its vendor u-boot reports 192K(uboot),2496K(kernel),5184K(rootfs),320K(rootfs_data), which is sized to this exact build, so every bound happens to fit there and nowhere else.
Two consequences worth knowing. A camera running OpenIPC's own u-boot cannot sysupgrade to this build -- check_combined_fits measures the FIT against the kernel partition and dies with "the FIT ... is 2496 KB, /dev/mtd2 holds 2048 KB. Nothing was written." That is a safe failure, no half-written flash. And a fresh whole-chip flash of the assembled .bin would boot a FIT truncated at 2048 KiB by sf read.
This is not something this PR introduced. The published cv6xx ultimate blob has the same mismatch and worse (FIT 2709 KiB, rootfs at +2752 KiB where u-boot expects +2048, blob 10496 KiB against a 7168 KiB partition), and hi3519dv500 has the offset half of it too. It looks like the cv610 env needs fixing rather than this branch needing another 448 KiB of trim -- but since this PR is what turns that into a published 8 MiB image, it seems worth settling first. I have put the detail on #2448, whose create_hisi is where the layout is actually decided.
On the bot review: finding 3 (curl parent + TLS backend) is genuinely addressed, and the defconfig comment explaining why lite has to name zlib/mbedtls/libcurl by hand where ultimate gets them through quirc and uacme is exactly the kind of comment this tree wants. Finding 2 reads stale against the current description, which claims byte-identical md5s from a clean build with the device-specific files on the JFFS2 overlay rather than in the squashfs.
Smaller points inline.
openipc-ai
left a comment
openipc-ai
left a comment
There was a problem hiding this comment.
Correcting my own verdict: I asked for two things before merge in #2447 (review) and then submitted it as a comment. Restating them as the blocking pair, so the state matches the ask.
1. The shared kernel config reaches hi3516cv6xx_ultimate, and every piece of evidence here is lite. br-ext-chip-hisilicon/board/hi3516cv6xx/hi3516cv6xx.generic.config is the family's config, so the 16 MiB cameras lose the same 146 drivers plus BPF_SYSCALL, IO_URING and zstd. I looked for a reason that would not boot and did not find one, and ci-matrix.py --stdin does narrow to both boards so it gets compiled -- but compiling is not booting, and ultimate is the half of the family with cameras in the field. One boot of an ultimate image, with dmesg and a stream, closes it.
2. The lite artifact this PR adds cannot be installed on a cv6xx camera running OpenIPC's own u-boot. Its mtdparts give 2048k(kernel),5120k(rootfs); this build is a 2496 KiB FIT and a 5184 KiB rootfs, so sysupgrade's check_combined_fits dies with "the FIT ... is 2496 KB, /dev/mtd2 holds 2048 KB. Nothing was written," and a whole-chip flash of the assembled .bin boots a FIT truncated by sf read ${kernaddr} ${kernsize}.
I flagged in the first review that this mismatch is pre-existing -- ultimate has it and worse -- and I let that soften the verdict, which was the wrong call. That the cv610 env needs fixing is an argument for fixing the env, not for publishing a second image into a layout nothing fits. Either the env moves or the build fits 2048/5120; either way it wants settling before an 8 MiB image is offered to people whose recovery path is a clip.
Everything else in that review stays advisory: the block-size RAM trade, the CHECK_SIZE offset, and the EXTREME_COMP observation are all worth doing and none of them should hold this up.
The media work itself is good and I would like to see it land.
9e748ff to
946afa4
Compare
create_hisi laid u-boot at 0 and the combined firmware.bin verbatim at the
firmware offset, on a canvas hardcoded to 16384 KiB. Two things were wrong with
that, one new and one old.
The new one: a lite variant is flashed to an 8 MiB part, and an image whose
tail runs past the end of the chip cannot be written to one. The part size is
now the sixth argument, defaulting to 16384 so every existing caller keeps its
behaviour, and the cv6xx loop emits both variants at the size each is built
for.
The old one: the boot images this job pairs with the firmware carry their
partition table in a compiled-in env, and nothing rewrites it later.
Decompressed from boot-hi3516cv610-*-nor.bin (a HiSilicon boot table with a
gzip'd u-boot behind it):
cv6xx 256k(boot),64k(env),2048k(kernel),5120k(rootfs),7168k@0x50000(firmware),-(rootfs_data)
bootnor: sf read ${kernaddr}=0x50000 ${kernsize}=0x200000; root=/dev/mtdblock3
dv500 512k(boot),256k(env),6144k(kernel),8192k(rootfs),14336k@0xC0000(firmware),-(rootfs_data)
So u-boot reads the kernel as one fixed slot and mounts root at a fixed offset
behind it. firmware.bin is the FIT with the squashfs packed at the next 64 KiB
boundary (board post-image.sh), which only agrees with that table when the FIT
happens to end exactly at the slot. On every published cv6xx and dv500 image it
does not: root=/dev/mtdblockN points 704 KiB before the squashfs on cv6xx
ultimate and 512 KiB past it on dv500, and the cv6xx FIT is larger than its
2048 KiB slot, so `sf read` hands bootm a truncated image. sysupgrade already
gets this right (do_update_firmware splits the blob at the FIT boundary and
writes each half to its partition); a fresh whole-chip flash of the release
.bin did not.
Do what sysupgrade does: split at the FIT boundary (the FIT's total size is the
big-endian word at byte 4; the squashfs length is bytes_used in its
superblock), put the FIT at the firmware offset and the rootfs at the rootfs
partition, and measure each piece against the partition it goes into -- u-boot
against the boot partition (the env sits behind it, so the firmware offset is
the wrong bound), the FIT against the kernel slot, the rootfs against the
rootfs slot, the firmware partition against the part. Each failure prints a
::error:: and fails the step rather than uploading something that boots a
truncated kernel or overwrites rootfs_data. The exact-size check on the
assembled canvas stays as the backstop.
The layout table is a case on $soc inside the job; adding a family means
adding its u-boot's env line there. With this in place the cv6xx ultimate image
on master fails the FIT check (2709 KiB in 2048) and dv500 ultimate fails the
rootfs check (8832 KiB in 8192) -- both are the same table disagreement, now
reported instead of published. OpenIPC#2447 fits both cv6xx slots.
|
Both blocking points addressed in Fits the cv610 u-boot's slots. FIT 2014 KB of 2048, squashfs 5076 KB of 5120, Booted as the release image. The camera now runs exactly what #2448's assembler produces for this build: the published Ultimate. Cannot be booted here (8 MiB part). What I can offer: ultimate's kernel built from this branch differs from the booted lite kernel by exactly the three iptables symbols its package fixup enables ( On the block-size note: 1 MiB stays. Your arithmetic is right about the ~2.5 MiB; measured |
openipc-ai
left a comment
openipc-ai
left a comment
There was a problem hiding this comment.
Both blocking points are answered, and the second one is answered in the way I most wanted and least expected: the camera is now running the artifact the assembler produces, not a hand-laid approximation of it. Published 20s u-boot at 0, erased env, FIT at 0x50000, squashfs at 0x250000, u-boot's own table showing up in /proc/mtd. That was the whole gap, and it is closed.
FIT 2014 KB of 2048 and squashfs 5076 KB of 5120 both fit the slots the cv610 env declares, and the Makefile now says so on the 8 MiB arm, so the next overrun fails in the board matrix instead of in the release job. Leaving the 16 MiB arm on the whole-blob figure with the reason written down is the right call -- that one is a u-boot table question and a size check here could not settle it.
I checked the wider trim for anything it could strand rather than take the list on trust. No cv6xx defconfig selects usb-dual-role or ffmpeg-openipc, so the USB gadget and ALSA removals have no consumer on this family; there is no cv610 or cv608 device under devices/ in OpenIPC/builder that the NAND/UBI removal could leave without a filesystem; and ci-matrix.py --self-test still passes on 946afa4d.
Ultimate stays unbooted and I am not going to hold this for it. The lite kernel that did boot is the same one, the variants diverge by the iptables symbols its package fixup adds, and cv6xx ultimate cannot be assembled into a release image at all until the 7744 KiB rootfs question is settled -- so a boot test there is gated on work that is not this PR.
Three notes, none blocking.
The .ko strip comment overstates its precedent. INSTALL_MOD_STRIP=1 is not --strip-unneeded; the kernel's own mod_strip_cmd is $(STRIP) --strip-debug, which keeps the whole .symtab. --strip-unneeded is the more aggressive of the two and the modules demonstrably still load, so the code is fine -- but the comment claims an equivalence that is not there, and in this tree the comment is the reasoning. Worth saying which of the two the saving actually came from.
I am withdrawing my block-size suggestion. At 5076 of 5120 there are 44 KB left and 512 KiB blocks cost 64 KB, so what I proposed would now overflow the slot. The pinned squashfs cache is still ~5 MiB of a 48 MiB camera and that cost is real, but you have the measurements and the budget and I do not.
Both halves clear by about 2% -- 34 KB on the FIT, 44 KB on the rootfs, each just past the 32 KB headroom warning. That is the margin the warning exists to talk about, so this family is one ordinary shared-overlay commit from red in both directions at once.
Good work, and thank you for chasing the u-boot table rather than arguing with it.
cv6xx has no image for 8 MiB NOR parts. It was added as a lite/8 MB target in 679c2c7 and renamed to ultimate/16 MB two days later in 8d5d02d, because "the previous 8 MB lite variant cannot fit majestic + vendor MPP libs + 42 V5 kernel modules". 679c2c7 had already booked the choice -- "follow-up will either rebalance to 16 MB or trim rootfs". The rebalance happened; the trim did not. This does the trim, against the budget the cv610 u-boot actually enforces. Its compiled-in env is 256k(boot),64k(env),2048k(kernel),5120k(rootfs); the kernel slot is read whole by `sf read ${kernaddr} ${kernsize}` and root is mounted at the fixed rootfs offset, so an image boots from the release layout only if the FIT is under 2048 KB and the squashfs under 5120 KB. The Makefile's ceiling for this family is FLASH_SIZE * 1024 on the combined blob, which measures neither; the 8 MiB arm now checks both halves against their slots, where a PR sees it. Most of the kernel trim is one bug. The shared cv6xx kernel config builds 146 satellite and terrestrial television drivers into every image -- DVB frontends (cx24116, stv0900, si2168), tuners (tda18271, xc5000, r820t) and the analog TV, radio and SDR stacks. Nothing in the tree can reach them: no defconfig enables gst1-plugins-bad, whose DVB plugins are the only DVB reference in userspace, and no overlay opens a /dev/dvb device. They are not there by choice. 679c2c7 regenerated this config from hi3516cv{608,610}_defconfig in the BSP, which sets CONFIG_MEDIA_SUPPORT=y and never mentions digital TV. In 5.10, MEDIA_DIGITAL_TV_SUPPORT and its siblings carry a prompt only if MEDIA_SUPPORT_FILTER is set and otherwise default to y, so the expansion turned all six media classes on by itself. Reproducible: a bare `make hi3516cv610_defconfig` in openipc/linux yields 147 DVB and tuner symbols. rv1106 builds clean by a different route -- the Rockchip tree patches MEDIA_DIGITAL_TV_SUPPORT to carry an unconditional prompt; hi3519dv500 has the same root cause at a smaller scale, 15 symbols, and is left alone here. The sensor path on this SoC is HiSilicon MPP through the out-of-tree open_*.ko modules, not V4L2, and nothing in the tree captures from a USB camera on it, so MEDIA_SUPPORT goes entirely rather than being filtered down to the UVC class. REGMAP_I2C goes with it: its only selector was dvb-frontends. In the same spirit, and for the 2048 KB slot: the eBPF syscall (no bpftool, libbpf or cgroup user anywhere in the tree; classic BPF for tcpdump stays), io_uring (no caller in musl, busybox or majestic), the NAND/UBI/UBIFS stack (no cv6xx NAND board exists in the tree and the family's repack is NOR-only), the NFS client (cv6xx cannot NFS-root -- ROOT_NFS and IP_PNP are off -- and nothing mounts one at runtime), the USB gadget stack and ALSA, whose only user it was, and the zstd compressor, which only UBIFS and CRYPTO_ZSTD pulled in. hi3519dv500, the other V5 family, already builds without NAND, media, sound, BPF, io_uring and zstd. zImage goes from 2722 KB to 2045 KB; the FIT lands at 2014 KB of 2048. The rootfs side, to 5076 KB of 5120: - 15 ultimate-only packages dropped: aws-webrtc, divinus, exfat + exfatprogs, iptables, lame, libwebsockets, mosquitto, motors, quirc, uacme, vtund, wireguard (compat + tools) and zerotier-one. - 1 MiB squashfs blocks instead of the 128 KiB default. Measured on this rootfs: 128K 8704 KB, 256K 8512, 512K 8384, 1024K 8320. The price is about 2.5 MiB of RAM against 512K blocks, since the fragment cache and the xz dictionary scale with the block; the camera below has 19.9 MB available with them, and the 64 KB is an erase block this budget does not have. - Buildroot's squashfs EXTREME_COMP, which for xz on ARM is `-Xbcj arm,armthumb`: 52 KiB. The kernel's xz decoder already carries XZ_DEC_ARM and XZ_DEC_ARMTHUMB, so there is nothing to pay at read time. - The 42 vendor modules were installed unstripped: Buildroot's target-finalize skips *.ko on purpose and nothing else stripped them. `--strip-unneeded`, the operation INSTALL_MOD_STRIP=1 performs, takes 16 KB off the squashfs. Both cv6xx variants get that. - lite drops the eight modules nothing loads: load_hisilicon has no modprobe for adc, aiisp, devstat, spi_dma_transfer, user, user_proc or uvc, and open_svac3e -- the SVAC3 encoder -- is probed only on the 20g/00s/00g dies, for a codec majestic has no code for. 60 KB. ultimate keeps all of them. Four package symbols ultimate never has to state are spelled out here, because it gets them by side effect and lite does not: quirc and uacme select libcurl, libwebsockets and mbedtls drag in zlib and the TLS backend. sysupgrade shells out to curl over https, so a lite image without them has a sysupgrade that cannot fetch its manifest. Enabling only the libcurl parent is not enough -- that yields a curl linked TLS_NONE against an absent libz. zlib also settles majestic's own libz.so.1 and libevent_mbedtls dependencies, which on this family were previously left to the integrator to supply. Kept, because majestic will not start without them: the HiSilicon audio/VQE libraries its DT_NEEDED pulls in, opus, ogg, libevent, libcurl and protobuf-c. dropbear, ipctool, yaml-cli and uboot-tools survive the trim. The kernel config is the family's shared one, so ultimate builds the same kernel; its own differs from lite's by the netfilter symbols the iptables package fixup switches on (IP_NF_IPTABLES, IP_NF_FILTER, NETFILTER_XTABLES) and nothing else. ci-matrix.py's self-test carries three hardcoded row counts that a new board changes; they are bumped here so the merge gate passes. The config was generated with olddefconfig against openipc/linux 5.10.221 and then applied as a symbol-only edit: every comment, section header and blank line is byte-identical to the file it replaces, and the diff contains nothing but CONFIG_ lines. Section headers left standing with nothing under them are what a regeneration would tidy; leaving them keeps this diff reviewable.
946afa4 to
770560b
Compare
openipc-ai
left a comment
openipc-ai
left a comment
There was a problem hiding this comment.
Re-approving after the rebase: my earlier approval was dismissed when the branch moved.
Rebased 946afa4d onto master aa495696 (which had picked up #2441, #2442, #2444 and #2452 since this branched) and force-pushed as 770560b3. Clean rebase -- master's #2444 touched the Makefile in a different arm, so no conflict, and the resulting patch is byte-identical to yours apart from the blob hashes in that hunk header. Authorship unchanged.
Gates re-run on the rebased tree: ci-matrix.py --self-test ok (100 boards, 136 packages, 56 cases), lint-workflow-shell.py --self-test passed, and all 147 shipped scripts parse under busybox ash both before and after comment stripping.
The workflow runs needed maintainer approval -- fork PR -- so I approved them; the matrix is running now. Nothing else from me.
create_hisi laid u-boot at 0 and the combined firmware.bin verbatim at the
firmware offset, on a canvas hardcoded to 16384 KiB. Two things were wrong with
that, one new and one old.
The new one: a lite variant is flashed to an 8 MiB part, and an image whose
tail runs past the end of the chip cannot be written to one. The part size is
now the sixth argument, defaulting to 16384 so every existing caller keeps its
behaviour, and the cv6xx loop emits both variants at the size each is built
for.
The old one: the boot images this job pairs with the firmware carry their
partition table in a compiled-in env, and nothing rewrites it later.
Decompressed from boot-hi3516cv610-*-nor.bin (a HiSilicon boot table with a
gzip'd u-boot behind it):
cv6xx 256k(boot),64k(env),2048k(kernel),5120k(rootfs),7168k@0x50000(firmware),-(rootfs_data)
bootnor: sf read ${kernaddr}=0x50000 ${kernsize}=0x200000; root=/dev/mtdblock3
dv500 512k(boot),256k(env),6144k(kernel),8192k(rootfs),14336k@0xC0000(firmware),-(rootfs_data)
So u-boot reads the kernel as one fixed slot and mounts root at a fixed offset
behind it. firmware.bin is the FIT with the squashfs packed at the next 64 KiB
boundary (board post-image.sh), which only agrees with that table when the FIT
happens to end exactly at the slot. On every published cv6xx and dv500 image it
does not: root=/dev/mtdblockN points 704 KiB before the squashfs on cv6xx
ultimate and 512 KiB past it on dv500, and the cv6xx FIT is larger than its
2048 KiB slot, so `sf read` hands bootm a truncated image. sysupgrade already
gets this right (do_update_firmware splits the blob at the FIT boundary and
writes each half to its partition); a fresh whole-chip flash of the release
.bin did not.
Do what sysupgrade does: split at the FIT boundary (the FIT's total size is the
big-endian word at byte 4; the squashfs length is bytes_used in its
superblock), put the FIT at the firmware offset and the rootfs at the rootfs
partition, and measure each piece against the partition it goes into -- u-boot
against the boot partition (the env sits behind it, so the firmware offset is
the wrong bound), the FIT against the kernel slot, the rootfs against the
rootfs slot, the firmware partition against the part. Each failure prints a
::error:: and fails the step rather than uploading something that boots a
truncated kernel or overwrites rootfs_data. The exact-size check on the
assembled canvas stays as the backstop.
The layout table is a case on $soc inside the job; adding a family means
adding its u-boot's env line there. With this in place the cv6xx ultimate image
on master fails the FIT check (2709 KiB in 2048) and dv500 ultimate fails the
rootfs check (8832 KiB in 8192) -- both are the same table disagreement, now
reported instead of published. OpenIPC#2447 fits both cv6xx slots.
create_hisi laid u-boot at 0 and the combined firmware.bin verbatim at the
firmware offset, on a canvas hardcoded to 16384 KiB. Two things were wrong with
that, one new and one old.
The new one: a lite variant is flashed to an 8 MiB part, and an image whose
tail runs past the end of the chip cannot be written to one. The part size is
now the sixth argument, defaulting to 16384 so every existing caller keeps its
behaviour, and the cv6xx loop emits both variants at the size each is built
for.
The old one: the boot images this job pairs with the firmware carry their
partition table in a compiled-in env, and nothing rewrites it later.
Decompressed from boot-hi3516cv610-*-nor.bin (a HiSilicon boot table with a
gzip'd u-boot behind it):
cv6xx 256k(boot),64k(env),2048k(kernel),5120k(rootfs),7168k@0x50000(firmware),-(rootfs_data)
bootnor: sf read ${kernaddr}=0x50000 ${kernsize}=0x200000; root=/dev/mtdblock3
dv500 512k(boot),256k(env),6144k(kernel),8192k(rootfs),14336k@0xC0000(firmware),-(rootfs_data)
So u-boot reads the kernel as one fixed slot and mounts root at a fixed offset
behind it. firmware.bin is the FIT with the squashfs packed at the next 64 KiB
boundary (board post-image.sh), which only agrees with that table when the FIT
happens to end exactly at the slot. On every published cv6xx and dv500 image it
does not: root=/dev/mtdblockN points 704 KiB before the squashfs on cv6xx
ultimate and 512 KiB past it on dv500, and the cv6xx FIT is larger than its
2048 KiB slot, so `sf read` hands bootm a truncated image. sysupgrade already
gets this right (do_update_firmware splits the blob at the FIT boundary and
writes each half to its partition); a fresh whole-chip flash of the release
.bin did not.
Do what sysupgrade does: split at the FIT boundary (the FIT's total size is the
big-endian word at byte 4; the squashfs length is bytes_used in its
superblock), put the FIT at the firmware offset and the rootfs at the rootfs
partition, and measure each piece against the partition it goes into -- u-boot
against the boot partition (the env sits behind it, so the firmware offset is
the wrong bound), the FIT against the kernel slot, the rootfs against the
rootfs slot, the firmware partition against the part. Each failure prints a
::error:: and fails the step rather than uploading something that boots a
truncated kernel or overwrites rootfs_data. The exact-size check on the
assembled canvas stays as the backstop.
The layout table is a case on $soc inside the job; adding a family means
adding its u-boot's env line there. With this in place the cv6xx ultimate image
on master fails the FIT check (2709 KiB in 2048) and dv500 ultimate fails the
rootfs check (8832 KiB in 8192) -- both are the same table disagreement, now
reported instead of published. OpenIPC#2447 fits both cv6xx slots.
A refusal must not take the other SoCs' images with it. The step runs under
Actions' default `bash -e`, and the Sigmastar, Allwinner and Ingenic images are
assembled before the HiSilicon loops, so a bare `return 1` there would end the
step with their .bin files in target/ and the upload never reached. The
HiSilicon calls collect their failures instead, the step exits 1 after the last
one so the run is red, and Upload runs on `always()` so what did assemble
ships.
openipc-ai
left a comment
openipc-ai
left a comment
There was a problem hiding this comment.
CI is green on the rebased head: 121 checks pass, the only non-pass is the release-publish job that does not run on pull requests. Re-approving because my own thread replies each landed as a COMMENTED review and superseded the earlier approval.
create_hisi laid u-boot at 0 and the combined firmware.bin verbatim at the
firmware offset, on a canvas hardcoded to 16384 KiB. Two things were wrong with
that, one new and one old.
The new one: a lite variant is flashed to an 8 MiB part, and an image whose
tail runs past the end of the chip cannot be written to one. The part size is
now the sixth argument, defaulting to 16384 so every existing caller keeps its
behaviour, and the cv6xx loop emits both variants at the size each is built
for.
The old one: the boot images this job pairs with the firmware carry their
partition table in a compiled-in env, and nothing rewrites it later.
Decompressed from boot-hi3516cv610-*-nor.bin (a HiSilicon boot table with a
gzip'd u-boot behind it):
cv6xx 256k(boot),64k(env),2048k(kernel),5120k(rootfs),7168k@0x50000(firmware),-(rootfs_data)
bootnor: sf read ${kernaddr}=0x50000 ${kernsize}=0x200000; root=/dev/mtdblock3
dv500 512k(boot),256k(env),6144k(kernel),8192k(rootfs),14336k@0xC0000(firmware),-(rootfs_data)
So u-boot reads the kernel as one fixed slot and mounts root at a fixed offset
behind it. firmware.bin is the FIT with the squashfs packed at the next 64 KiB
boundary (board post-image.sh), which only agrees with that table when the FIT
happens to end exactly at the slot. On every published cv6xx and dv500 image it
does not: root=/dev/mtdblockN points 704 KiB before the squashfs on cv6xx
ultimate and 512 KiB past it on dv500, and the cv6xx FIT is larger than its
2048 KiB slot, so `sf read` hands bootm a truncated image. sysupgrade already
gets this right (do_update_firmware splits the blob at the FIT boundary and
writes each half to its partition); a fresh whole-chip flash of the release
.bin did not.
Do what sysupgrade does: split at the FIT boundary (the FIT's total size is the
big-endian word at byte 4; the squashfs length is bytes_used in its
superblock), put the FIT at the firmware offset and the rootfs at the rootfs
partition, and measure each piece against the partition it goes into -- u-boot
against the boot partition (the env sits behind it, so the firmware offset is
the wrong bound), the FIT against the kernel slot, the rootfs against the
rootfs slot, the firmware partition against the part. Each failure prints a
::error:: and fails the step rather than uploading something that boots a
truncated kernel or overwrites rootfs_data. The exact-size check on the
assembled canvas stays as the backstop.
The layout table is a case on $soc inside the job; adding a family means
adding its u-boot's env line there. With this in place the cv6xx ultimate image
on master fails the FIT check (2709 KiB in 2048) and dv500 ultimate fails the
rootfs check (8832 KiB in 8192) -- both are the same table disagreement, now
reported instead of published. OpenIPC#2447 fits both cv6xx slots.
A refusal must not take the other SoCs' images with it. The step runs under
Actions' default `bash -e`, and the Sigmastar, Allwinner and Ingenic images are
assembled before the HiSilicon loops, so a bare `return 1` there would end the
step with their .bin files in target/ and the upload never reached. The
HiSilicon calls collect their failures instead, the step exits 1 after the last
one so the run is red, and Upload runs on `always()` so what did assemble
ships.
create_hisi laid u-boot at 0 and the combined firmware.bin verbatim at the
firmware offset, on a canvas hardcoded to 16384 KiB. Two things were wrong with
that, one new and one old.
The new one: a lite variant is flashed to an 8 MiB part, and an image whose
tail runs past the end of the chip cannot be written to one. The part size is
now the sixth argument, defaulting to 16384 so every existing caller keeps its
behaviour, and the cv6xx loop emits both variants at the size each is built
for.
The old one: the boot images this job pairs with the firmware carry their
partition table in a compiled-in env, and nothing rewrites it later.
Decompressed from boot-hi3516cv610-*-nor.bin (a HiSilicon boot table with a
gzip'd u-boot behind it):
cv6xx 256k(boot),64k(env),2048k(kernel),5120k(rootfs),7168k@0x50000(firmware),-(rootfs_data)
bootnor: sf read ${kernaddr}=0x50000 ${kernsize}=0x200000; root=/dev/mtdblock3
dv500 512k(boot),256k(env),6144k(kernel),8192k(rootfs),14336k@0xC0000(firmware),-(rootfs_data)
So u-boot reads the kernel as one fixed slot and mounts root at a fixed offset
behind it. firmware.bin is the FIT with the squashfs packed at the next 64 KiB
boundary (board post-image.sh), which only agrees with that table when the FIT
happens to end exactly at the slot. On every published cv6xx and dv500 image it
does not: root=/dev/mtdblockN points 704 KiB before the squashfs on cv6xx
ultimate and 512 KiB past it on dv500, and the cv6xx FIT is larger than its
2048 KiB slot, so `sf read` hands bootm a truncated image. sysupgrade already
gets this right (do_update_firmware splits the blob at the FIT boundary and
writes each half to its partition); a fresh whole-chip flash of the release
.bin did not.
Do what sysupgrade does: split at the FIT boundary (the FIT's total size is the
big-endian word at byte 4; the squashfs length is bytes_used in its
superblock), put the FIT at the firmware offset and the rootfs at the rootfs
partition, and measure each piece against the partition it goes into -- u-boot
against the boot partition (the env sits behind it, so the firmware offset is
the wrong bound), the FIT against the kernel slot, the rootfs against the
rootfs slot, the firmware partition against the part. Each failure prints a
::error:: and fails the step rather than uploading something that boots a
truncated kernel or overwrites rootfs_data. The exact-size check on the
assembled canvas stays as the backstop.
The layout table is a case on $soc inside the job; adding a family means
adding its u-boot's env line there. With this in place the cv6xx ultimate image
on master fails the FIT check (2709 KiB in 2048) and dv500 ultimate fails the
rootfs check (8832 KiB in 8192) -- both are the same table disagreement, now
reported instead of published. OpenIPC#2447 fits both cv6xx slots.
A refusal must not take the other SoCs' images with it. The step runs under
Actions' default `bash -e`, and the Sigmastar, Allwinner and Ingenic images are
assembled before the HiSilicon loops, so a bare `return 1` there would end the
step with their .bin files in target/ and the upload never reached. The
HiSilicon calls collect their failures instead, the step exits 1 after the last
one so the run is red, and Upload runs on `always()` so what did assemble
ships.
The bounds checks #2448 added ran for the first time on 2026-09-19 and refused six images: hi3516cv6xx ultimate on all four DDR binnings, and hi3519dv500 ultimate on both. That is not a regression #2448 introduced -- it is the table disagreement the published images already carried, now reported instead of shipped -- but it leaves the job red on every run, and re-running against a fresh nightly does not clear it. That run assembled from nightly-20260918-49908b5, because the build had failed and the manifest never advanced, so cv6xx reported a FIT overflow from a blob predating #2447's kernel trim. Measured against the current nightly with create_hisi's own arithmetic, the error changes but does not go away: hi3516cv6xx_lite FIT 2048/2048 ok rootfs 5088/5120 ok hi3516cv6xx_ultimate FIT 2048/2048 ok rootfs 7096/5120 over by 1976 KiB hi3519dv500_ultimate FIT 5632/6144 ok rootfs 8820/8192 over by 628 KiB The cause is that there is one boot binary per DDR binning, not one per part size, and its compiled-in mtdparts is the same on an 8 MiB part and a 16 MiB one. Decompressing the gzip member inside the shipped u-boots (offset 40128 for cv610, 84768 for hi3519dv500) gives: cv610/cv608 256k(boot),64k(env),2048k(kernel),5120k(rootfs),7168k@0x50000(firmware) hi3519dv500 512k(boot),256k(env),6144k(kernel),8192k(rootfs),14336k@0xC0000(firmware) The cv6xx ultimate blob is 9152 KiB and the dv500 one 14464 KiB -- each larger than the whole firmware partition, never mind the rootfs slot inside it. FLASH_SIZE="16" buys them nothing, because the rest of the chip is rootfs_data. No trim closes a 1976 KiB gap, so until a u-boot ships whose table matches a 16 MiB part there is nothing to assemble. So emit cv6xx lite only, and drop the dv500 loop, which has no lite variant to fall back on. Both comments carry the partition numbers so restoring a loop is a two-line change. #2447 fits both cv6xx lite slots exactly (2048 in 2048, 5088 in 5120) and that image assembles and boots today. The .tgz for both variants still publishes from the build job, so sysupgrade is unaffected -- it splits the blob at the FIT boundary and writes each half to its own partition, and refuses an oversized half rather than mis-landing it.
2 participants
Problem
cv6xx has no image for 8 MiB NOR parts. It was added as a lite/8 MB target in 679c2c7 and renamed to ultimate/16 MB two days later in 8d5d02d, because "the previous 8 MB lite variant cannot fit majestic + vendor MPP libs + 42 V5 kernel modules". 679c2c7 had already booked the choice — "follow-up will either rebalance to 16 MB or trim rootfs". The rebalance happened; the trim did not, and cameras on 8 MiB parts have had nothing to flash since.
This does the trim, against the budget the cv610 u-boot actually enforces. Its compiled-in env is
256k(boot),64k(env),2048k(kernel),5120k(rootfs); the kernel slot is read whole bysf read ${kernaddr} ${kernsize}and root is mounted at the fixed rootfs offset, so an image boots from the release layout only if the FIT is under 2048 KB and the squashfs under 5120 KB. The Makefile's ceiling for this family isFLASH_SIZE * 1024on the combined blob, which measures neither; the 8 MiB arm now checks both halves against their slots, where a PR sees it.Most of the kernel trim is one bug. The shared cv6xx kernel config builds 146 satellite and terrestrial television drivers into every image — DVB frontends (
cx24116,stv0900,si2168), tuners (tda18271,xc5000,r820t), and the analog TV, radio and SDR stacks. Nothing in the tree can reach them: no defconfig enablesgst1-plugins-bad, whose DVB plugins are the only DVB reference in userspace, and no overlay opens a/dev/dvbdevice. They are not there by choice. 679c2c7 regenerated this config fromhi3516cv{608,610}_defconfigin the BSP, which setsCONFIG_MEDIA_SUPPORT=yand never mentions digital TV. In 5.10,MEDIA_DIGITAL_TV_SUPPORTand its siblings carry a prompt only ifMEDIA_SUPPORT_FILTERis set, and otherwise default toy— so the expansion turned all six media classes on by itself.rv1106builds clean by a different route (the Rockchip tree patchesMEDIA_DIGITAL_TV_SUPPORTto carry an unconditional prompt);hi3519dv500has the same root cause at a smaller scale (15 symbols) and is left alone.The sensor path on this SoC is HiSilicon MPP through the out-of-tree
open_*.komodules, not V4L2, and nothing in the tree captures from a USB camera on it, soMEDIA_SUPPORTgoes entirely rather than being filtered down to the UVC class.REGMAP_I2Cgoes with it: its only selector wasdvb-frontends. In the same spirit, and for the 2048 KB slot:NETselectsBPFindependently)ROOT_NFSandIP_PNPare off) and nothing mounts one at runtimeCRYPTO_ZSTDpulled inhi3519dv500, the other V5 family, already builds without NAND, media, sound, BPF, io_uring and zstd. zImage goes from 2722 KB to 2045 KB; the FIT lands at 2014 KB of its 2048.The rootfs, to 5076 KB of its 5120:
aws-webrtc,divinus,exfat+exfatprogs,iptables,lame,libwebsockets,mosquitto,motors,quirc,uacme,vtund,wireguardcompat + tools,zerotier-one).EXTREME_COMP, which for xz on ARM is-Xbcj arm,armthumb: 52 KiB. The kernel's xz decoder already carriesXZ_DEC_ARMandXZ_DEC_ARMTHUMB, so there is nothing to pay at read time.*.koon purpose and nothing else stripped them.--strip-unneeded, the operationINSTALL_MOD_STRIP=1performs, takes 16 KB off the squashfs. Both cv6xx variants get that.load_hisiliconhas no modprobe for adc, aiisp, devstat, spi_dma_transfer, user, user_proc or uvc, andopen_svac3e— the SVAC3 encoder — is probed only on the 20g/00s/00g dies, for a codec majestic has no code for. 60 KB. ultimate keeps all of them.Four package symbols ultimate never has to state are spelled out here, because it gets them by side effect and lite does not: quirc and uacme select libcurl, libwebsockets and mbedtls drag in zlib and the TLS backend. sysupgrade shells out to curl over https, so a lite image without them has a sysupgrade that cannot fetch its manifest. Enabling only the libcurl parent is not enough — that yields a curl linked
TLS_NONEagainst an absent libz.Kept, because majestic will not start without them: the HiSilicon audio/VQE libraries its
DT_NEEDEDpulls in, plus opus, ogg, libevent, libcurl and protobuf-c.dropbear,ipctool,yaml-clianduboot-toolssurvive.The kernel config is the family's shared one, so
ultimatebuilds the same kernel. I built ultimate's from this branch: it differs from the lite kernel by the netfilter symbols the iptables package fixup switches on (IP_NF_IPTABLES,IP_NF_FILTER,NETFILTER_XTABLES; 7.8 KB of zImage) and nothing else.ci-matrix.py's self-test carries three hardcoded row counts that a new board changes; they are bumped here so the merge gate passes.Hardware tested on
hi3516cv6xx (chip reports
hi3516cv613, DDR3 128 MB), SIMICAM A314D / Vatilon H80, 8 MiB NOR, GC4653 sensor.The camera runs the release image for this build as #2448's
create_hisiassembles it: the publishedboot-hi3516cv610-20s-nor.binat 0, this branch's FIT at 0x50000 and its squashfs at 0x250000, erased env, on the partition table that u-boot declares. All three were read back from flash byte-identical (md5s below). The env was left to u-boot's defaults andosmemset to 48M throughfw_setenv, which is the one step every cv610 install does. Two things the board adds that are not in this PR:rootfs_datawas pre-staged rather than left for/initto format, because this sensor has no driver in the tree (galaxycore_gc4653_2Lis in the ev200 V4 list, not the cv6xx V5 SDK's six —ultimatehas the same gap), so the overlay carries an out-of-treelibsns_gc4653_2l.soand this board's MIPI-port-1 bring-up scripts, plus the usual device files. Nothing in the squashfs is replaced by anything else.An ultimate image cannot be booted here: 8 MiB part. What the board did exercise is the shared kernel config in full; ultimate's kernel differs from it by the three iptables symbols above.
Evidence
Clean build of this branch:
The kernel change, before and after, in the file this PR edits:
On the camera, 2 minutes after the reboot that applied
osmem:The config was generated with
olddefconfigagainst openipc/linux 5.10.221 and applied as a symbol-only edit: the diff isCONFIG_lines only.Scope
general/package/all-patches/linux/general/overlay/or a sharedload_<vendor>script hardcodes a value specific to my boardLD_PRELOAD, no unbuildable binariesALL_BOARDS