Skip to content

install: write the UBI-only NAND layout for u-boot-xmedia SoCs - #147

Merged
widgetii merged 3 commits into
masterfrom
install/nand-ubi-layout
Oct 4, 2026
Merged

widgetii merged 3 commits into
masterfrom
install/nand-ubi-layout

Conversation

@widgetii

@widgetii widgetii commented Oct 4, 2026

Copy link
Copy Markdown
Member

The u-boot-xmedia SoCs (hi3516ev200/ev300/dv200, hi3518ev300, gk7205v500/v510/v530) moved to one NAND layout. It landed in OpenIPC/u-boot-xmedia#11 and #12, OpenIPC/firmware#2537, and OpenIPC/website#392 and #393:

0x000000 boot 768K   u-boot-<soc>-nand.bin
0x0C0000 env  256K
0x100000 ubi  rest   rootfs.ubi.<board>: UBIFS rootfs (kernel inside, /boot/fitImage) + rootfs_data

defib still installed the retired split layout on these SoCs and downloaded the retired -universal bootloader for them.

Install (install --nand)

For these SoCs install --nand now:

  • writes u-boot-<soc>-nand.bin into 0..0xc0000;
  • writes rootfs.ubi.<board> with nand erase.part ubi and nand write.trimffs, then reads it back and CRC-checks it;
  • runs env default -a, restores the MAC, and saves.

Never sent: setenv mtdparts, mtdids, bootcmd, bootargs, or ubi part/create/write.

Kernel and rootfs-data stages: no-ops. The kernel and an empty rootfs_data are inside the UBI image.

Erase safety:

  • The installer checks that mtdparts puts ubi at 0x100000 before erasing.
  • It falls back to nand erase 0x100000 (to the chip end) where erase.part is unavailable.
  • Other chips keep the split-layout path.

Bootloader names

These seven SoCs resolve per flash type, to u-boot-<soc>-nor.bin or u-boot-<soc>-nand.bin.

  • install follows --nand. burn gains --nand and defaults to NOR. restore --flash-type follows the flash type.
  • The -universal image is never downloaded for them, and a stale cached one doesn't stand in for the new build.
  • Every other chip is unchanged.

Web UI

  • Takes the -nor build for these SoCs.
  • Now cuts the SPL where the compressed payload starts (detectSplSize, a port of HiSiliconStandard._detect_spl_size). The u-boot-xmedia images put it at 0x4400, while the profile says 0x6000. Sending the profile length writes into SRAM the boot ROM uses for its own state.

Verified

On a hi3516ev300 (W25N01GV, 128 MiB, 5 factory bad blocks), defib install -c hi3516ev300 --nand --power-cycle with the NAND package and the u-boot-xmedia master NAND image ran end to end:

  • burned U-Boot to RAM from the boot ROM;
  • U-Boot OK;
  • Erased ubi: 0x100000, 0x7F00000 bytes;
  • Flash verified: E2E60EAA;
  • Restoring factory ethaddr, Environment saved;
  • rebooted into OpenIPC, which booted /boot/fitImage and came up on the network.

Separately, defib burn -c hi3516ev300 -f u-boot-hi3516ev300-nand.bin loaded the u-boot-xmedia image from the boot ROM; defib found its gzip SPL boundary at 0x4400.

Tests

  • pytest: 1020 passed. ruff and mypy clean.
  • New tests/test_nand_ubi_install.py: the exact console sequence, the erase fallback, the missing-mtdparts path, refusing a relocated ubi, an env-only run, and a NOR install taking -nor over a cached -universal.
  • tests/test_firmware.py covers names per flash type.
  • Web tests: 92 pass. The new detectSplSize gives 0x4400 on the real hi3516ev300 image.

Not verified on hardware

  • The gk7205v5xx and hi3516dv200 paths.
  • The Web UI recovery with the -nor images.
  • The erase.part fallback on a U-Boot without it.

OpenIPC retired the HiSilicon split NAND layout (1M boot, 1M env, 8M raw
kernel, ubi) and the u-boot-hi3516ev200 "universal" build that carried
it. hi3516ev200, hi3516ev300, hi3518ev300, hi3516dv200 and
gk7205v500/v510/v530 now boot u-boot-xmedia, published per flash type in
the OpenIPC/firmware latest release as u-boot-<soc>-nor.bin and
u-boot-<soc>-nand.bin. The NAND build uses one layout:

  0x000000  boot  768K   u-boot-<soc>-nand.bin
  0x0C0000  env   256K
  0x100000  ubi   rest   rootfs.ubi.<board> (rootfs volume with the
                         kernel as /boot/fitImage, plus rootfs_data)

and its default environment defines mtdids/mtdparts, bootcmd and
bootargs for it.

firmware: PER_FLASH_TYPE_UBOOT lists those SoCs. asset_name(),
firmware_url(), has_firmware(), get_cached_path() and download_firmware()
take an optional flash_type ("nor"/"nand", NOR when unknown) and resolve
u-boot-<soc>-<type>.bin for them; other chips ignore it. The universal
image is never downloaded for these SoCs. A universal image already in the
cache is no longer reported by get_cached_path() (it would shadow the
published build), but download_firmware() still falls back to it for NOR
when the download fails. gk7205v5x0 NOR shares its cache name with
download_v500_donor(), so either source satisfies the other.

Callers: install picks the build from --nand; burn gains --nand (NOR by
default); restore uses the NAND build for --flash-type nand and warns that
"auto" means NOR for these SoCs. agent upload/flash only take the SPL,
whose DDR init is the same in both builds, so they keep the NOR default.

install: with --nand on those SoCs the plan becomes
  uboot   tftp u (padded to 0xC0000), crc32, nand erase 0x0 0xc0000,
          nand write <ram> 0x0 0xc0000
  kernel  no-op: the kernel is inside the UBI image
  rootfs  tftp r (rootfs.ubi.<board>), crc32, printenv mtdparts,
          nand erase.part ubi, nand write.trimffs <ram> 0x100000 <len>,
          nand read + crc32 readback when crc32 is available
  rootfs-data  no-op: the UBI image carries an empty rootfs_data
  env     (if uboot was written) env default -a, restore ethaddr, saveenv
  reset
with no setenv mtdparts/bootcmd/bootargs and no ubi create/write.
write.trimffs is required: a plain nand write programs the image's 0xFF
pages and UBIFS programming them again breaks ECC (OpenIPC/firmware#2519).
erase.part erases the whole partition, so chips over 128 MiB lose every
stale block. It runs only when the live mtdparts puts ubi at 0x100000; a
different offset aborts before any erase. Without mtdparts, or when the
U-Boot lacks erase.part (no "Erasing at" in the reply), the installer
erases with `nand erase 0x100000`, which U-Boot runs to the chip end. The
reported erase offset must be 0x100000 and the image must fit the
reported size. The firmware package member is rootfs.ubi.<board>
(never rootfs.ubifs.*); a package without one is rejected with a pointer
to openipc.<board>-nand-<variant>.tgz.

The split layout stays only for `install --nand` on other chips: it is
chip-agnostic code the existing hi3516cv300 test drives, and nothing in
the tree says those chips moved.

Tests: offline FakeNandUBoot console checks the exact hi3516ev300 NAND
command sequence, the erase.part fallback, the missing-mtdparts path, the
refusal on a foreign ubi offset, an env-only run, the NOR install picking
u-boot-hi3516ev300-nor.bin, package member selection, and the per-flash
naming for all seven SoCs.

Untested on hardware: the full install. The command sequence follows the
manual procedure verified on a hi3516ev300 with a defib-burned
u-boot-hi3516ev300-nand.bin; the erase.part fallback and the readback CRC
step have not run on a camera.
The seven u-boot-xmedia SoCs (hi3516ev200/ev300/dv200, hi3518ev300,
gk7205v500/v510/v530) publish their U-Boot per flash type now. The
-universal image the Web UI looked for is the retired u-boot-hi3516ev200
build, or absent. A recovery loads U-Boot into RAM and needs no flash
layout, so the Web UI takes the -nor build (fwAssetName), as `defib burn`
does without --nand. A stale -universal asset of these SoCs is ignored
rather than taken in its place.

Those images put their compressed payload earlier than the reference
SPL the profiles describe (gzip at 0x4400 against FILELEN 0x6000). The Web
UI sent the profile's length, and bytes past the payload land in SRAM the
bootrom uses for its own state. detectSplSize ports
HiSiliconStandard._detect_spl_size: it finds the LZMA or gzip header from
0x4000, rounds down to 1K and caps at SRAMLIMIT where a profile has one.
On the real u-boot-hi3516ev300-nand.bin it gives 0x4400, the same as
defib's Python. Web tests: 92 pass.
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Install the UBI-only NAND layout for u-boot-xmedia SoCs

✨ Enhancement 🐞 Bug fix 🧪 Tests 📝 Documentation 🕐 40+ Minutes

Grey Divider

AI Description

• Install u-boot-xmedia NAND systems with UBI-only images, guarded erasure, CRC verification, and
 default environments.
• Select flash-specific U-Boot builds across the CLI and web UI instead of retired universal builds.
• Detect web SPL boundaries to protect SRAM; add install, asset-selection, and browser tests.
Diagram

graph TD
  A["Install CLI"] --> B["Install orchestrator"] --> C["U-Boot resolver"] --> D["UBI package loader"] --> E{"Offset safe?"} -->|yes| F["NAND erase/write"] --> G["Default environment"]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Create and populate UBI volumes through U-Boot
  • ➕ Lets UBI manage volume creation and writes.
  • ➖ Requires additional U-Boot commands and volume reconstruction instead of installing the published whole-partition image.
2. Infer flash type instead of requiring explicit selection
  • ➕ Could reduce accidental NOR/NAND build selection.
  • ➖ Flash type may be unavailable during boot-ROM recovery, making automatic selection unreliable.

Recommendation: Keep explicit flash-type selection and write the published UBI image with nand write.trimffs. This matches the upstream layout without reconstructing volumes; retain the offset guard and readback verification because NAND erasure is high risk.

Files changed (12) +1253 / -72

Enhancement (5) +493 / -63
app.pyRoute CLI recovery commands to the selected U-Boot build +33/-9

Route CLI recovery commands to the selected U-Boot build

• Adds burn --nand, passes flash type through burn and restore firmware resolution, and explains the new install --nand behavior.

src/defib/cli/app.py

firmware.pyResolve U-Boot assets by flash type for seven SoCs +103/-22

Resolve U-Boot assets by flash type for seven SoCs

• Names, downloads, and caches separate NOR and NAND builds for u-boot-xmedia SoCs while preserving other chips' naming. Prevents stale universal cache entries from shadowing published builds, with a NOR-only offline fallback.

src/defib/firmware.py

firmware.pyLoad NAND packages containing a single UBI image +51/-10

Load NAND packages containing a single UBI image

• Adds UBI-only package selection and validation without requiring a separate uImage. Rejects missing, ambiguous, invalid, or checksum-mismatched UBI payloads.

src/defib/install/firmware.py

layout.pyDefine the UBI-only NAND layout and erase guards +73/-0

Define the UBI-only NAND layout and erase guards

• Defines boot and environment boundaries and adds helpers to parse live mtdparts offsets and NAND erase results.

src/defib/install/layout.py

orchestrator.pyInstall and verify the UBI-only NAND layout +233/-22

Install and verify the UBI-only NAND layout

• Selects the new path for the seven SoCs while retaining legacy behavior elsewhere. Guards erasure, writes the whole UBI image with trimffs, verifies readback when CRC is available, skips redundant stages, and restores U-Boot defaults with the MAC preserved.

src/defib/install/orchestrator.py

Bug fix (2) +52 / -6
index.htmlUse the correct web recovery asset and SPL length +3/-2

Use the correct web recovery asset and SPL length

• Downloads the selected asset name when metadata is unavailable and uploads only the detected SPL portion, respecting any profile SRAM limit.

web/index.html

protocol.jsSelect xmedia NOR assets and detect compressed SPL boundaries +49/-4

Select xmedia NOR assets and detect compressed SPL boundaries

• Filters release assets so xmedia recovery takes the NOR build rather than a stale universal image. Detects gzip or LZMA payload boundaries to avoid writing beyond the SPL into boot-ROM SRAM.

web/protocol.js

Tests (3) +672 / -2
test_firmware.pyTest flash-specific asset selection and cache behavior +106/-2

Test flash-specific asset selection and cache behavior

• Covers all seven SoCs, default NOR naming, NAND downloads, rejection of stale universal cache entries, and the NOR-only offline fallback.

tests/test_firmware.py

test_nand_ubi_install.pyTest UBI-only NAND installation and safety paths +520/-0

Test UBI-only NAND installation and safety paths

• Adds a simulated U-Boot console to check the full command sequence, erase fallback, unsafe-offset refusal, environment-only operation, and NOR asset selection. Also tests NAND package validation and layout parsers.

tests/test_nand_ubi_install.py

protocol.test.jsTest web asset selection and SPL boundary detection +46/-0

Test web asset selection and SPL boundary detection

• Checks xmedia and legacy asset naming, stale-asset filtering, gzip and LZMA detection, profile fallback, and SRAM-limit capping.

web/protocol.test.js

Documentation (2) +36 / -1
CLAUDE.mdDocument flash-specific assets and NAND layouts +8/-1

Document flash-specific assets and NAND layouts

• Explains which SoCs use per-flash U-Boot builds and distinguishes their UBI-only layout from the legacy split layout.

CLAUDE.md

README.mdDocument u-boot-xmedia NAND installation +28/-0

Document u-boot-xmedia NAND installation

• Adds the partition map, an install example, flash-type selection rules, and the behavior of UBI and environment stages.

README.md

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Web recovery overruns SRAM on hi3520dv200-class chips ✓ Resolved
Description
standardSendFirmware passes profile.SRAMLIMIT to detectSplSize, but the embedded web
PROFILES omit that field, leaving the browser without the SRAM cap used by the Python path. When a
compressed-payload boundary exceeds a chip’s limit—for example, 0x4400 versus 0x3B00 on
hi3520dv200—the browser uploads bytes beyond the SRAM window, reaching boot-ROM state and causing
the chip to drop back into boot mode mid-upload; hi3518ev200, hi3516cv200, and hi3516dv100 also have
0x3B00 limits in their Python profiles.
Code

web/index.html[R300-301]

+  const splMax = detectSplSize(firmware, parseInt(profile.FILELEN[1], 16),
+    profile.SRAMLIMIT ? parseInt(profile.SRAMLIMIT, 16) : null);
Evidence
The Python profile and detector cap the detected boundary using SRAMLIMIT, including for
hi3520dv200’s documented 0x4400 LZMA boundary and approximately 0x3B00 SRAM window. The embedded
web profile instead lists hi3520dv200 with "FILELEN":["0x0040","0x2300"] but no SRAMLIMIT;
consequently, the web call passes no cap to detectSplSize, and firmware.slice(0, splMax) uploads
the raw detected boundary.

src/defib/protocol/hisilicon_standard.py[456-485]
src/defib/profiles/data/hi3520dv200.json[1-1]
web/index.html[185-185]
web/index.html[296-316]
web/protocol.js[255-273]
web/index.html[185-186]
web/index.html[297-318]
web/protocol.js[255-272]
src/defib/protocol/hisilicon_standard.py[449-478]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The web UI calls `detectSplSize` for HiSilicon-standard chips, but its embedded `PROFILES` omit `SRAMLIMIT`. Unlike the Python path, it therefore does not cap boundaries beyond SRAM, such as hi3520dv200’s `0x4400` boundary against its `0x3B00` limit.
## Fix Focus Areas
- web/index.html[185-186]
- web/index.html[297-301]
- web/protocol.js[255-273]
## Recommended Fix
Regenerate the web `PROFILES` blob from `src/defib/profiles/data` with `SRAMLIMIT` retained, including `0x3B00` for hi3520dv200, hi3518ev200, hi3516cv200, and hi3516dv100. Add a parity test in `web/profile-parity.test.js` comparing `SRAMLIMIT` with the Python profiles, and test the web send path when a detected boundary exceeds the limit. As a safety net for profiles missing `SRAMLIMIT` whose Python profiles define one, cap the detected size at the previous profile length, or restrict detection to `XMEDIA_SOCS`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. NAND installs can inspect the wrong partition ✓ Resolved
Description
mtdparts_partition_offset() returns the first partition named ubi across all devices, and
flash_ubi_image() treats that offset as the NAND partition’s offset. When an earlier SPI device
also names a partition ubi, a valid NAND install can be refused or nand erase.part ubi can be
selected without checking the NAND partition.
Code

src/defib/install/orchestrator.py[R1317-1320]

+                ubi_off = (
+                    mtdparts_partition_offset(mtd_value, "ubi")
+                    if mtd_value is not None
+                    else None
Evidence
The helper scans semicolon-separated devices and returns on the first matching name; the installer
uses that result to guard a NAND erase. The new tests establish that multi-device partition strings
are accepted.

src/defib/install/layout.py[74-99]
src/defib/install/orchestrator.py[1315-1334]
tests/test_nand_ubi_install.py[493-503]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The erase guard may use the first `ubi` partition from a different device in a multi-device `mtdparts` value.
## Fix Focus Areas
- src/defib/install/layout.py[74-99]
- src/defib/install/orchestrator.py[1315-1334]
## Recommended Fix
Resolve `ubi` specifically within the active NAND device’s partition list before authorizing `nand erase.part ubi`; add a test where both SPI and NAND devices have a `ubi` partition.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. A wrong-board image can reach NAND ✓ Resolved
Description
load_firmware_bundle(..., ubi_only=True) accepts any single rootfs.ubi.* member without
comparing its board suffix with the requested chip. A valid UBI image from another board passes the
header and optional checksum checks, then the rootfs stage writes it to persistent NAND along with
its embedded kernel.
Code

src/defib/install/firmware.py[R60-63]

+                if _is_ubi_member(name):
+                    ubi_members.append(name)
+                    rootfs_name = name
+                    rootfs = stream.read()
Evidence
The new loader selects by filename prefix and validates only image presence, count, header and any
supplied checksum. Its caller supplies no target for comparison, and the selected bytes reach the
new raw UBI write path.

src/defib/install/firmware.py[28-30]
src/defib/install/firmware.py[59-85]
src/defib/install/orchestrator.py[224-235]
src/defib/install/orchestrator.py[1431-1433]
tests/test_nand_ubi_install.py[460-464]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new UBI-only loader can select and flash an image built for a different board.
## Fix Focus Areas
- src/defib/install/firmware.py[59-85]
- src/defib/install/orchestrator.py[224-227]
## Recommended Fix
Pass the requested target into UBI-only bundle validation and check the selected image’s board against an explicit compatibility mapping. Include the documented gk7205v510/v530 to gk7205v500 package mapping.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can describe a rule in plain language on the Rules page and Qodo drafts it for you

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread src/defib/install/orchestrator.py
Comment thread src/defib/install/firmware.py
Comment thread web/index.html Outdated
Review on #147.

- load_firmware_bundle checks that rootfs.ubi.<board> is built for the
  chip being installed, through ubi_nand_boards: the chip itself, plus
  gk7205v500 for gk7205v510/v530, which share that build. The image
  carries its kernel, so one built for another board would put that
  board's system on this camera's NAND.
- mtdparts_partition_offset(..., nand_only=True) looks for `ubi` only on
  NAND devices (an mtd-id with "nand" in it). A `ubi` on a SPI device
  listed first can no longer pass, or fail, the erase guard for the NAND
  one.
- The Web UI cuts the SPL at its payload only for the u-boot-xmedia
  images. Its profiles carry no SRAMLIMIT, so on other chips, where a
  payload can sit past the SRAM window (hi3520dv200, 0x4400 against
  0x3B00), it keeps sending the profile length as it always did.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant