feat(rp2040): deploy parity with Arduino-Pico, PlatformIO, and official picotool

[zackees]

Parent: #1040

Goal

Bring fbuild's Pico 1 deploy path to behavioral and safety parity with the proven Arduino-Pico and PlatformIO implementations while keeping fbuild as the only user-facing build/deploy command. The acceptance target is a stock Amazon-purchased Raspberry Pi Pico 1, including first deployment with no serial port and subsequent 1200-bps reset deployments.

This issue was produced from a source-level comparison against:

• Arduino-Pico's managed picotool UF2 recipe and uf2conv.py uploader:
  • https://github.com/earlephilhower/arduino-pico/blob/47c2cd2b0be80c357809f7acdfa50cb5b7ccfce4/platform.txt
  • https://github.com/earlephilhower/arduino-pico/blob/47c2cd2b0be80c357809f7acdfa50cb5b7ccfce4/tools/uf2conv.py
• PlatformIO-compatible platform-raspberrypi upload orchestration:
  • https://github.com/maxgerhardt/platform-raspberrypi/blob/aa70b802be8851668053d4f09734e4089fe41932/builder/main.py
• Raspberry Pi's official picotool / elf2uf2 implementation:
  • https://github.com/raspberrypi/picotool
  • https://github.com/raspberrypi/picotool/blob/master/elf2uf2/elf2uf2.cpp
• Official UF2 behavior: the virtual volume automatically dismounts after a successful copy:
  • https://www.raspberrypi.com/documentation/microcontrollers/debug-probe.html

Gap inventory

Phase 1: canonical UF2 artifact and conversion

• Emit an inspectable sibling firmware.uf2; do not treat BIN as the canonical deploy artifact.
• Use the pinned, framework-managed pqt-picotool for ELF -> UF2 with the exact Arduino-Pico 4.5.3 recipe (picotool uf2 convert <ELF> <UF2> --family rp2040). Never search PATH or require a host install. Add --platform rp2040 memory validation only after the pinned framework tool supports it; picotool 4.0.1 rejects that newer flag.
• Package the matching Windows, Linux x86_64/aarch64, and macOS x86_64/arm64 picotool archives with verified checksums.
• Prefer ELF conversion. Preserve PT_LOAD physical addresses, sparse pages, overlap/range validation, and official touched-sector padding. The current handwritten dense-image encoder silently zero-fills gaps and is not equivalent to official elf2uf2.
• Validate existing UF2 input magic, block structure, and RP2040 family ID; reject wrong-family or malformed input before copying.
• Retain a narrowly tested BIN conversion fallback at the documented 0x10000000 offset only when no ELF exists.

Phase 2: ROM-volume transfer semantics

• Copy the completed local artifact to exactly NEW.UF2, matching Arduino-Pico.
• Match Arduino-Pico's direct whole-buffer open(..., wb) / f.write(buf) transfer to NEW.UF2; do not use whole-file copy APIs, repeated small writes, metadata preservation, reopen/fsync, rename, retry, or readback against the ROM volume. Preserve full-byte accounting and require an observed eject before accepting any final-write error.
• Wait for the selected BOOTSEL volume to disappear and treat disappearance as the expected success transition.
• Distinguish an accepted transfer/eject race from a true partial-copy or corrupt-volume failure; surface the original actionable OS error when acceptance cannot be established.

Phase 3: stock-board discovery and reset

• Preserve the already-in-BOOTSEL fast path so first deployment requires no COM port, manual BOOTSEL press, Zadig, or vendor driver.
• Match Arduino-Pico's bounded 1200-bps/DTR reset behavior for subsequent deployments.
• Snapshot devices/ports before reset, wait for a new matching BOOTSEL volume, and correlate the selected runtime device to the selected volume where platform metadata permits.
• Reject multiple plausible targets rather than selecting the first device or flashing every mounted UF2 volume.

Phase 4: CDC handoff and result correctness

• Snapshot the pre-deploy serial set and return the actual post-deploy port, including when Windows assigns a different COM name.
• Do not return the stale original port merely because CDC reacquisition timed out.
• A CDC timeout must be an actionable deployment/recovery failure for firmware expected to expose JSON-RPC, not success: true with port: None.
• Wait for volume disappearance before accepting runtime CDC so a stale/pre-reset port cannot win the race.

Phase 5: USB identity policy

• Remove production RP2040 VID/PID literals from the deploy and port-selection paths.
• Consume generated FastLED/boards catalogue data during the fbuild build; no built-in VID/PID fallback is allowed in production.
• Test fixtures may embed IDs. Broader repo-wide migration remains tracked by chore(usb): migrate all VID/PID catalogue data to FastLED/boards #1047; FastLED/boards discovery/publication remains tracked by research: audit and publish Pico/Pico 2 USB identities boards#46.

Phase 6: deterministic tests and portability

• Add injectable/fake transfer behavior for auto-eject during/after close and for genuine copy failures.
• Cover malformed/wrong-family UF2, invalid ELF, overlapping/out-of-range segments, zero/multiple volumes, boot timeout, disappearance timeout, changed COM, ambiguous CDC, and CDC timeout.
• Cover Windows drive roots, Linux /media and /run/media, and macOS /Volumes discovery without relying on the test host's OS.
• Ensure the editable fbuild development install stages both CLI and daemon binaries so FastLED hardware tests cannot silently run a stale daemon.

Attached-board diagnostic (not normal flash acceptance)

• Load a flash-independent RAM UF2 through fbuild and prove ROM-volume eject/re-enumeration, a new USB arrival epoch, and zero new Disk 51/153 events. Merged in feat(rp2040): support RAM-only UF2 loads #1072 at 8f7e8609e3fe4d28f7edb7fa97fdc1eb8437c06d; artifact SHA-256 0b28e37067a718504d6fc2841524d751b52bd7e3a5bbae0aea08d74f080e3e97.
• Compare RAM-target and flash-target UF2 behavior on the unchanged cable, motherboard port, Windows MSC stack, and ROM USB session. The RAM path succeeds repeatedly; only the external-QSPI erase/program path stalls, isolating a board-level QSPI fault.

Phase 7: attached Pico 1 acceptance

• On a known-good stock Pico 1 whose blank/invalid flash naturally enumerates as ROM RPI-RP2, record 2E8A:0003, INFO_UF2.TXT, exact fbuild commit, command, volume transition, runtime VID:PID/COM, and JSON-RPC response. No physical BOOTSEL press is part of acceptance.
• Run the applicable pin-free runtime suites from feat(autoresearch): add reusable pin-free JSON-RPC smoke mode FastLED#3628-#3630. RPC smoke, SIMD 85/85, IEEE-754 35/35, and watchdog recovery passed on genuine RP2040 CDC; coroutine was removed as an unrelated unsupported optional runtime.
• Pass ten consecutive deploy -> reboot -> new/current CDC -> JSON-RPC cycles with no manual action. Verified on COM12 / 2E8A:000A / serial 5303284720C4641C.

Current remaining gate: the original attached serial E0C9125B0D9B has a nonfunctional external-QSPI path. A working replacement (5303284720C4641C) now passes subsequent 1200-bps deploy, runtime CDC, JSON-RPC, watchdog recovery, and 10/10 soak. The user explicitly requested a PlatformIO A/B before fbuild on that replacement, so the as-received blank/invalid-flash -> fbuild-only first-deploy checkbox remains unproven and must not be rewritten as passed.

Out of scope

• GPIO/pin validation until headers are soldered (test(rp2040): GPIO, PIO, and LED loopback bring-up after headers are soldered FastLED#3631).
• Direct PlatformIO, Arduino IDE, pio, or user-invoked picotool commands. Their source is reference material only; fbuild remains the sole interface.
• A separate deploy repository unless the managed-tool/package approach proves impossible and repository creation is separately approved.

Completion

Land coherent fbuild milestone PRs, admin-merge after focused tests and attached-board evidence, then publish one fbuild release after all fbuild-owned gaps are resolved. Finally restore FastLED from the editable link to the released pin and rerun the complete Pico 1 acceptance path.

[zackees]

Software milestone merged in #1050 at c086773. Reopening because this issue's attached Pico 1 acceptance and 10-cycle JSON-RPC soak are intentionally not yet proven: the currently mounted RPI-RP2 volume is reported by Windows as Warning / Full Repair Needed. A fresh physical reconnect is required before hardware evidence is trustworthy. No corrupted-mount result will be used to close the issue.

[zackees]

Formatting follow-up #1051 was admin-merged at c84f916 after the pinned Rust 1.94.1 formatter identified four files in #1050. Direct per-file soldr rustfmt --check now passes. This is mechanical only; hardware acceptance remains open pending a fresh healthy RPI-RP2 mount.

[zackees]

Current-source sweep completed on 2026-07-14 against:

• Arduino-Pico 0643904164d16e2599e5c6e95f0eea5c9d6297d4 (current master)
• maxgerhardt/platform-raspberrypi aa70b802be8851668053d4f09734e4089fe41932 (current develop)
• platformio/platform-raspberrypi 6fc5dce8ef6a98a7d3a37414e29b688bdba9ddf9 (current develop)
• current Raspberry Pi picotool documentation/source

Important correction/acceptance invariant: Arduino-Pico's current default upload method is still Default (UF2) (boards.txt selects uf2conv), and its default firmware does not require the opt-in ENABLE_PICOTOOL_USB code. That define belongs only to Arduino-Pico's separate Upload Method -> Picotool option. fbuild's stock-board acceptance path is therefore ROM UF2 mass-storage plus the normal 1200-bps reset hook; it must never require a board pre-flashed for the opt-in picotool transport. A blank/as-received Pico whose flash is not bootable naturally enters ROM USB boot, so first deploy must work without a CDC port.

Additional gaps found in the current sources and live FastLED/boards integration:

• Run the already-in-BOOTSEL preflight before touching any supplied/stale serial port. maxgerhardt's current builder explicitly returns early when an RP device is already in BOOTSEL; fbuild currently touches first and can disturb an unrelated COM device.
• Support stable USB-serial selection/correlation (SER=<serial> or an idiomatic fbuild equivalent) across COM renumbering. The current maxgerhardt builder resolves a serial-number selector to exactly one interface; ambiguity must fail closed.
• Accept an explicitly selected UF2 volume/path for multi-board recovery, while continuing to reject an ambiguous unselected set.
• Match Arduino-Pico's Linux recovery coverage for an unmounted ROM device (/dev/disk/by-id/usb-RPI_RP2*-part1 + bounded udisksctl mount best effort), without requiring PlatformIO.
• Reject overlapping UF2 target addresses, not only duplicate block numbers, and reject accidental ELF/unknown input instead of ever encoding its file bytes as a raw flash BIN.
• Derive RP2040/RP2350 family from resolved board MCU metadata, not a board_id.contains(pico2) heuristic; this is required for non-Pico RP2350 boards and Pico 2 aliases.
• Feature-probe the managed picotool conversion flags. Arduino-Pico current master publishes pqt-picotool 4.1.0 while fbuild's currently pinned 4.5.3 core uses the older 4.0.1 package; never pass newer --platform flags to a pinned binary that rejects them.
• Make typed FastLED/boards profile ingestion tolerate legal upstream identity collisions while keeping curated role conflicts fail-closed (fix: preserve upstream USB identity collisions boards#51, merged).
• Verify the live 2.77 MB usb-profiles.json against _meta.json SHA/schema and handle case-folded board/alias collisions without last-record-wins behavior (current fbuild milestone in validation).

The original Phase 1-7 checklist remains authoritative. I am iterating these deltas in the same sweep and will attach focused tests plus stock-board evidence here before closing.

[zackees]

Correction to the earlier hardware blocker comment: Windows Get-Volume reports this attached RPI-RP2 volume as Warning / Full Repair Needed, but that is not evidence of a corrupt Pico. Raspberry Pi documents RPI-RP2 as a synthetic/fake FAT volume with no mutable storage backing it; reads after writes are intentionally not normal filesystem semantics. The authoritative stock-device evidence is healthy USB enumeration plus a readable ROM marker:

• USB 2E8A:0003
• serial E0C9125B0D9B
• mass-storage interface present
• INFO_UF2.TXT readable on G:

The prior "fresh reconnect required" assumption is withdrawn. We will not run filesystem repair against the ROM's virtual disk. Hardware acceptance can proceed through fbuild's validated NEW.UF2 transfer/eject path.

Reference: https://www.raspberrypi.com/news/the-ventura-problem/ (Raspberry Pi explains that RPI-RP2 is a fake volume and why ordinary filesystem readback/error expectations do not apply).

[zackees]

Attached stock Pico 1 acceptance found and closed two additional source-parity gaps:

• fix: accept canonical UF2 magic for RP-series deploy #1055 corrected fbuild's first UF2 magic word. The canonical on-wire bytes are 55 46 32 0A; the old internal constant encoded 55 45 32 0A, so fbuild rejected managed picotool's valid UF2 and generated invalid raw-BIN fallback UF2. A byte-literal regression test now prevents the internal constant and validator from agreeing on the same typo.
• fix: add stock-ROM picotool fallback for RP-series deploy #1056 keeps Arduino-Pico's default ROM UF2 mass-storage route primary, then falls back to fbuild's pinned managed picotool if the host rejects the synthetic FAT write. The fallback talks to stock ROM BOOTSEL and does not require opt-in Arduino picotool firmware or any board pre-flash. It never searches PATH and adds no fbuild VID/PID literals.

Live evidence on this host:

• stock ROM identity and marker remain healthy: 2E8A:0003, serial E0C9125B0D9B, UF2 Bootloader v3.0, Model: Raspberry Pi RP2, Board-ID: RPI-RP2
• managed firmware.uf2 now passes fbuild validation
• Windows rejects G:\NEW.UF2 with error 1392 while reporting the synthetic volume dirty
• the stock ROM's second interface is present as RP2 Boot but has no bound driver (CM_PROB_FAILED_INSTALL), so official picotool reports that WinUSB may need to be bound through Zadig

This is a host transport condition, not evidence that a purchased board needs different firmware. Raspberry Pi's official getting-started guide documents WinUSB on RP2 Boot (Interface 1) for picotool on Windows, and current picotool explicitly targets devices in BOOTSEL mode by default. Arduino-Pico current default remains UF2 copy; its opt-in application picotool support is a separate post-first-flash choice.

References:

• https://github.com/raspberrypi/picotool
• https://datasheets.raspberrypi.com/pico/getting-started-with-pico.pdf
• https://github.com/earlephilhower/arduino-pico/blob/master/platform.txt
• https://github.com/earlephilhower/arduino-pico/blob/master/tools/uf2conv.py
• https://forums.raspberrypi.com/viewtopic.php?t=326610 (same Windows copy error; removable-drive scanning software was the reported cause)

The remaining attached-board gate is host access: either a fresh BOOTSEL enumeration must restore writable mass-storage behavior, or WinUSB must be bound to Interface 1 so the new managed fallback can run. No filesystem repair will be attempted against the synthetic ROM volume.