bootloader,flash: pre-seed VolatileVars.bin for overlay/DTBO loading

[guanquan (GuanquanTian)]

Summary

Add tooling to pre-seed VolatileVars.bin with the VendorDtbOverlays UEFI variable at build time, and wire it into the flat-meta flash pipeline on a per-board basis.

Why

Overlay-based DTBO loading depends on the VendorDtbOverlays UEFI variable being present in VolatileVars.bin. Today this file is only ever generated by the firmware itself during the UEFI boot stage — there's no way to seed it ahead of time.

This means the variable doesn't exist on first boot (or after any reset that wipes the ESP), so overlay/DTBO loading can't be relied upon without an extra runtime step.

In the flat-meta flash pipeline, a single CI job can build one shared efi.bin and reuse it across several boards (see targets.json). Only some of those boards need VolatileVars.bin pre-seeded, so the injection needs to be per-board rather than per-platform/suite, and it needs to work against an existing efi.bin (not just a freshly-built one).

What it does

• bootloader/gen-volatile-vars.py — builds an EBBR-style VolatileVars.bin (EFI_VARIABLE_FILE + EFI_VARIABLE_ENTRY layout consumed by UpdateVariableFromRTVolatileBin()). Ships a built-in default entry for VendorDtbOverlays (camx). Accepts an optional --config extra.json to add/override entries (matched by guid+name) without touching the script.
• bootloader/build-efi-esp.sh — new --seed-volatile-vars / --seed-volatile-vars-config flags to bake VolatileVars.bin into a freshly built ESP image.
• bootloader/patch-volatile-vars.sh — writes VolatileVars.bin into an existing FAT32 image in place via mtools (mcopy), no loop device/mount/root required. Used when one efi.bin is built once and eused across multiple boards.
• flash/parse-boards-config.py / flash/gen-flash-dirs.sh — new "seed_volatile_vars" field (bool, default false) and "seed_volatile_vars_config" field (optional JSON path, default "") in targets.json. When seed_volatile_vars is true for a board, gen-flash-dirs.sh patches that board's own efi.bin copy after staging it (forwarding seed_volatile_vars_config as --config if set), leaving other boards sharing the same source efi.bin unaffected.

Usage

1. Add VolatileVars.bin to an already-generated efi.bin:
   ./build-efi-esp.sh --out efi.bin
   ./patch-volatile-vars.sh --efi efi.bin

2. Generate VolatileVars.bin separately:
   python3 gen-volatile-vars.py --out VolatileVars.bin

3. Generate and add VolatileVars.bin during the compilation of efi.bin:
   ./build-efi-esp.sh --seed-volatile-vars --out efi.bin

[Bjordis Collaku (bjordiscollaku)]

Reviewed the change. The path that runs today (targets.json volatile_vars: true to patch-volatile-vars.sh to the built-in camx default) is what you validated, building and on target, and looks solid. Nothing below is blocking. Each item sits on a path that flow does not reach: the --config handling in gen_volatile_vars.py, and the build-efi-esp.sh --volatile-vars build-time path. They are latent, worth sorting before those paths get used.

build-efi-esp.sh

• python3 dependency check: when --volatile-vars is set, python3 is added to missing[], but the apt-get install line does not include it and there is no re-check afterward. On a host without python3, the script builds the full image and then fails at the generation step. Adding python3 to the install list resolves it. This is specific to the build-efi-esp.sh --volatile-vars build-time path; the targets.json / patch-volatile-vars.sh path already hard-fails cleanly on a missing python3, so it is low-priority.

• VolatileVars generation: --volatile-vars-config is not validated for existence in Step 3, and generation runs after the image is fully built. A mistyped path fails at the very end and leaves an efi.bin that appears complete but is missing VolatileVars.bin. Generating the blob to a WORKDIR temp file up front (as with STANDALONE_EFI) and installing it here, plus a -f check in Step 3, would fail fast instead.

gen_volatile_vars.py

• _decode_data: base64.b64decode without validate=True silently drops invalid characters. For example, b64decode("SGV#sbG8=") returns b'Hello', and an all-invalid value yields an empty variable (delete semantics). Passing validate=True addresses this while the path is still unused.

• merge_entries: the override key is guid.lower() on the raw string, so a braced or uppercase form of the same GUID does not match the default. Confirmed: {882F8C2B-...} produces two entries rather than an override. Keying on uuid.UUID(guid).bytes normalizes it.

• Filename: every other script in these directories uses hyphens (build-efi-esp.sh, patch-volatile-vars.sh, parse-boards-config.py); this is the only one with underscores. Since it is never imported as a Python module, renaming it to gen-volatile-vars.py would keep the naming consistent.

[guanquan (GuanquanTian)]

Bjordis Collaku (@bjordiscollaku)

Thanks for the thorough review — all five points below are addressed:

build-efi-esp.sh

• python3 dependency check: Added python3 to the apt-get install line, so the --volatile-vars build-time path now installs it alongside the other missing tools instead of failing later at generation.
• VolatileVars generation ordering: Generation now happens in its own step before the image is created/formatted (mirrors the STANDALONE_EFI pattern — written to a WORKDIR temp file, then installed into the mounted image afterward). Also added a -f existence check for --volatile-vars-config during input validation, so a mistyped path fails fast instead of leaving a "complete-looking" efi.bin missing VolatileVars.bin.

gen_volatile_vars.py

• _decode_data: Added validate=True to base64.b64decode, so invalid characters now raise instead of being silently dropped (which previously could produce a truncated/empty variable — delete semantics).
• merge_entries GUID matching: Switched the override key to uuid.UUID(guid).bytes instead of guid.lower(), so braced/uppercase forms of the same GUID now correctly match the default entry instead of producing a duplicate.
• Filename: Renamed to gen-volatile-vars.py (hyphenated) to match the naming convention of the other scripts in these directories.

Also renamed --volatile-vars/--volatile-vars-config → --seed-volatile-vars/--seed-volatile-vars-config (and the corresponding volatile_vars/volatile_vars_config → seed_volatile_vars/seed_volatile_vars_config in targets.json) for consistency with the rest of the naming.