Skip to content

fix(stm32f4): Hangs during flashing firmware - #7415

Merged
pfeerick merged 11 commits into
mainfrom
richardclli/fix-f4-flash-hangs
Oct 4, 2026
Merged

pfeerick merged 11 commits into
mainfrom
richardclli/fix-f4-flash-hangs

Conversation

@richardclli

@richardclli richardclli commented Jun 1, 2026 •

Copy link
Copy Markdown
Member

Trying to fix the problem by chance the flashing will stop in the middle. Using opencode + deepseek v4 to discover the fix

Summary by CodeRabbit

  • Bug Fixes
    • Improved reliability of flash erase and programming by protecting critical sequences from interruption, reducing risk of corruption during firmware or storage updates.
    • Ensured hardware operations properly wait for completion, flush instruction/data caches, and respect timeouts, improving device stability and predictability during flash-related tasks.

@richardclli richardclli self-assigned this Jun 1, 2026
@coderabbitai

coderabbitai Bot commented Jun 1, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Adds flash helper routines for waiting and flushing caches, and wraps STM32 flash unlock/erase/program/lock sequences with interrupt masking, DSB barriers, and cache flushes for both F2/F4 register-driven and HAL-based paths.

Changes

Flash Operation Interrupt Masking & Helpers

Layer / File(s) Summary
Flash helper utilities
radio/src/targets/common/arm/stm32/flash_driver.cpp
Adds FLASH_TIMEOUT_MS, flash_drv_wait_last_op (DWT timeout + flag clearing), and flash_drv_flush_caches (I/D cache invalidation).
Erase: F2/F4 register path
radio/src/targets/common/arm/stm32/flash_driver.cpp
stm32_flash_erase_sector (F2/F4) now disables interrupts around unlock/erase, waits via flash_drv_wait_last_op, clears flags, issues __DSB(), re-enables interrupts before flushing caches, then locks flash.
Erase: HAL-based path
radio/src/targets/common/arm/stm32/flash_driver.cpp
Non-F2/F4 stm32_flash_erase_sector disables interrupts around HAL_FLASHEx_Erase, captures sector error, and re-enables interrupts after locking.
Program: F2/F4 register path
radio/src/targets/common/arm/stm32/flash_driver.cpp
F2/F4 stm32_flash_program disables interrupts during unlock and the programming loop, adds __DSB() barriers, re-enables interrupts after the loop and clearing control bits, then locks flash.
Program: HAL-based path finalization
radio/src/targets/common/arm/stm32/flash_driver.cpp
Completes non-F2/F4 stm32_flash_program interrupt masking by disabling interrupts before HAL programming and re-enabling them before exiting the block.

🎯 4 (Complex) | ⏱️ ~45 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description lacks required template elements: no issue reference (Fixes #), minimal summary detail, and insufficient explanation of the actual changes or rationale. Add the issue reference, expand the summary to explain the specific problem (interrupt/cache deadlock) and solution, and cite any testing methodology or results more formally.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title directly addresses the main issue: preventing hangs during STM32F4 flash firmware operations through interrupt masking and cache synchronization fixes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch richardclli/fix-f4-flash-hangs

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 Infer (1.2.0)
radio/src/targets/common/arm/stm32/flash_driver.cpp

In file included from radio/src/targets/common/arm/stm32/flash_driver.cpp:22:
radio/src/targets/common/arm/stm32/flash_driver.h:26:10: fatal error: 'hal/flash_driver.h' file not found
26 | #include "hal/flash_driver.h"
| ^~~~~~~~~~~~~~~~~~~~
1 error generated.
Error: the following clang command did not run successfully:
/opt/infer-linux-x86_64-v1.2.0/lib/infer/facebook-clang-plugins/clang/install/bin/clang-18
@/tmp/coderabbit-infer/182eddc77287b80f6c59b61c3b0b6c2da0dba705-dfee4c55f90f26fc/tmp/clang_command_.tmp.b7d1fd.txt
++Contents of '/tmp/coderabbit-infer/182eddc77287b80f6c59b61c3b0b6c2da0dba705-dfee4c55f90f26fc/tmp/clang_command_.tmp.b7d1fd.txt':
"-cc1" "-load"
"/opt/infer-linux-x86_64-v1.2.0/lib/infer/infer/bin/../../facebook-clang-plugins/libtooling/build/FacebookClangPlugin.dylib"
"-add-plugin" "BiniouASTExporter" "-plugin-arg-BiniouASTExporter" "-"
"-plugin-arg-BiniouASTExporter" "PREPEND_CURRENT_DIR=1"
"-plugin-arg-BiniouASTExpor

... [truncated 1215 characters] ...

"
"-internal-isystem" "/usr/local/include" "-internal-isystem"
"/usr/lib/gcc/x86_64-linux-gnu/12/../../../../x86_64-linux-gnu/include"
"-internal-externc-isystem" "/usr/include/x86_64-linux-gnu"
"-internal-externc-isystem" "/include" "-internal-externc-isystem"
"/usr/include" "-Wno-ignored-optimization-argument" "-Wno-everything"
"-fdeprecated-macro" "-ferror-limit" "19" "-fgnuc-version=4.2.1"
"-fskip-odr-check-in-gmf" "-fcxx-exceptions" "-fexceptions"
"-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-o"
"/tmp/coderabbit-infer/dfee4c55f90f26fc/file.o" "-x" "c++"
"radio/src/targets/common/arm/stm32/flash_driver.cpp" "-O0"
"-fno-builtin" "-include"
"/opt/infer-linux-x86_64-v1.2.0/lib/infer/infer/bin/../lib/clang_wrappers/global_defines.h"
"-Wno-everything"


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@richardclli richardclli added this to the 2.11.7 milestone Jun 1, 2026
@richardclli
richardclli marked this pull request as draft June 1, 2026 00:41
@richardclli richardclli changed the title fix(stm32): disable interrupts during flash erase/program to prevent … fix(stm32f4): Hangs during flashing firmware Jun 1, 2026
@richardclli
richardclli force-pushed the richardclli/fix-f4-flash-hangs branch from 3ff0440 to b14c674 Compare June 1, 2026 05:05
@richardclli
richardclli changed the base branch from 2.11 to main June 1, 2026 05:10
@richardclli
richardclli marked this pull request as ready for review June 1, 2026 05:11

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
radio/src/targets/common/arm/stm32/flash_driver.cpp (1)

179-192: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Same issue: restore previous interrupt state instead of unconditionally enabling.

Apply the same PRIMASK save/restore pattern here for consistency and correctness.

Proposed fix
   int ret = 0;
+  uint32_t primask = __get_PRIMASK();
   __disable_irq();
   stm32_flash_unlock();
   while (address < end_addr) {
     if (_FLASH_PROGRAM(address, p_data) != HAL_OK) {
       ret = -1;
       break;
     }

     address += sizeof(uint32_t) * FLASH_PROG_WORDS;
     p_data += FLASH_PROG_WORDS;
   }

   stm32_flash_lock();
-  __enable_irq();
+  __set_PRIMASK(primask);
   return ret;
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@radio/src/targets/common/arm/stm32/flash_driver.cpp` around lines 179 - 192,
The block currently unconditionally calls __disable_irq() and later
__enable_irq(); change it to save and restore the prior interrupt state using
the PRIMASK pattern: capture the current PRIMASK (via __get_PRIMASK() or
equivalent) before disabling, call __disable_irq(), perform
stm32_flash_unlock(), the programming loop using _FLASH_PROGRAM, and
stm32_flash_lock(), then restore the saved PRIMASK (via __set_PRIMASK(saved) or
equivalent) instead of calling __enable_irq() directly so the interrupt state is
preserved; update the surrounding code in flash_driver.cpp where
__disable_irq(), stm32_flash_unlock(), stm32_flash_lock(), and __enable_irq()
are used.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@radio/src/targets/common/arm/stm32/flash_driver.cpp`:
- Around line 148-155: The code unconditionally calls __enable_irq() after using
__disable_irq(), which can enable interrupts that were previously disabled by
the caller; change the pattern to save and restore the caller's PRIMASK instead:
at function entry (around where __disable_irq() is currently called) read the
current PRIMASK via __get_PRIMASK(), then call __disable_irq(), perform
stm32_flash_unlock(), HAL_FLASHEx_Erase(...), stm32_flash_lock(), and finally
restore the original interrupt state by calling __set_PRIMASK(saved_primask)
instead of unconditionally calling __enable_irq(); update usage around
stm32_flash_unlock()/stm32_flash_lock()/HAL_FLASHEx_Erase to use this
save/restore PRIMASK pattern.

---

Outside diff comments:
In `@radio/src/targets/common/arm/stm32/flash_driver.cpp`:
- Around line 179-192: The block currently unconditionally calls __disable_irq()
and later __enable_irq(); change it to save and restore the prior interrupt
state using the PRIMASK pattern: capture the current PRIMASK (via
__get_PRIMASK() or equivalent) before disabling, call __disable_irq(), perform
stm32_flash_unlock(), the programming loop using _FLASH_PROGRAM, and
stm32_flash_lock(), then restore the saved PRIMASK (via __set_PRIMASK(saved) or
equivalent) instead of calling __enable_irq() directly so the interrupt state is
preserved; update the surrounding code in flash_driver.cpp where
__disable_irq(), stm32_flash_unlock(), stm32_flash_lock(), and __enable_irq()
are used.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5d211f90-8d5f-4c97-b58a-b508b0fe27d7

📥 Commits

Reviewing files that changed from the base of the PR and between 00d7544 and b14c674.

📒 Files selected for processing (1)
  • radio/src/targets/common/arm/stm32/flash_driver.cpp

Comment on lines +148 to +155
__disable_irq();
stm32_flash_unlock();
if (HAL_FLASHEx_Erase(&eraseInit, &sector_errors) != HAL_OK) {
ret = -1;
}

stm32_flash_lock();
__enable_irq();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Unconditional __enable_irq() may unexpectedly alter caller's interrupt state.

If interrupts were already disabled before entering this function, __enable_irq() will enable them unexpectedly. Use PRIMASK save/restore pattern to preserve the original interrupt state.

Proposed fix
   int ret = 0;
   uint32_t sector_errors = 0;

+  uint32_t primask = __get_PRIMASK();
   __disable_irq();
   stm32_flash_unlock();
   if (HAL_FLASHEx_Erase(&eraseInit, &sector_errors) != HAL_OK) {
     ret = -1;
   }

   stm32_flash_lock();
-  __enable_irq();
+  __set_PRIMASK(primask);
   return ret;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
__disable_irq();
stm32_flash_unlock();
if (HAL_FLASHEx_Erase(&eraseInit, &sector_errors) != HAL_OK) {
ret = -1;
}
stm32_flash_lock();
__enable_irq();
uint32_t primask = __get_PRIMASK();
__disable_irq();
stm32_flash_unlock();
if (HAL_FLASHEx_Erase(&eraseInit, &sector_errors) != HAL_OK) {
ret = -1;
}
stm32_flash_lock();
__set_PRIMASK(primask);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@radio/src/targets/common/arm/stm32/flash_driver.cpp` around lines 148 - 155,
The code unconditionally calls __enable_irq() after using __disable_irq(), which
can enable interrupts that were previously disabled by the caller; change the
pattern to save and restore the caller's PRIMASK instead: at function entry
(around where __disable_irq() is currently called) read the current PRIMASK via
__get_PRIMASK(), then call __disable_irq(), perform stm32_flash_unlock(),
HAL_FLASHEx_Erase(...), stm32_flash_lock(), and finally restore the original
interrupt state by calling __set_PRIMASK(saved_primask) instead of
unconditionally calling __enable_irq(); update usage around
stm32_flash_unlock()/stm32_flash_lock()/HAL_FLASHEx_Erase to use this
save/restore PRIMASK pattern.

@richardclli

richardclli commented Jun 1, 2026 •

Copy link
Copy Markdown
Member Author

Tested to flash > 10 times using my PL18U, no more hangs. However, the probability of seeing hangs during flashing is quite low. Maybe need more to test to confirm if it is gone.

As the flash operation can still works properly. No harm to merge anyway.

@pfeerick

pfeerick commented Jun 3, 2026

Copy link
Copy Markdown
Member

Maybe need more to test to confirm if it is gone.

Probably quite a few more... I think we were counting something like 1 in 30 flashes... very strange gremlin

@pfeerick pfeerick added backport/2.11 To be backported to a 2.11 release also. backport/2.12 To be backported to a 2.12 release also. labels Jun 3, 2026
@richardclli

richardclli commented Jun 4, 2026 •

Copy link
Copy Markdown
Member Author

Maybe need more to test to confirm if it is gone.

Probably quite a few more... I think we were counting something like 1 in 30 flashes... very strange gremlin

So it is still there, right? Let me see if deepseek can discover more suspect codes.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
radio/src/targets/common/arm/stm32/flash_driver.cpp (1)

187-211: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Preserve the caller’s PRIMASK in the new IRQ-masked paths too.

These blocks still pair __disable_irq() with unconditional __enable_irq(). If the caller entered with interrupts already masked, these helpers reopen IRQs on exit and break the caller’s critical section; please apply the same PRIMASK save/restore pattern already requested for the HAL erase path.

Suggested pattern
+  uint32_t primask = __get_PRIMASK();
   __disable_irq();
   ...
-  __enable_irq();
+  __set_PRIMASK(primask);

Also applies to: 263-287, 294-307

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@radio/src/targets/common/arm/stm32/flash_driver.cpp` around lines 187 - 211,
The IRQ-masking here unconditionally calls __disable_irq() then __enable_irq(),
which can re-enable interrupts if the caller already had PRIMASK set; instead
save and restore PRIMASK around the critical region: call __get_PRIMASK() into a
local variable, then __disable_irq(), keep the existing __DSB()/operation
sequence (stm32_flash_unlock(), FLASH CR ops, flash_drv_wait_last_op(),
CLEAR_BIT, __DSB(), flash_drv_flush_caches(), stm32_flash_lock()), and at the
end restore the saved state with __set_PRIMASK(saved_primask) rather than
unconditionally calling __enable_irq(); apply the same change to the other
similar blocks that use __disable_irq()/__enable_irq() in this file.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@radio/src/targets/common/arm/stm32/flash_driver.cpp`:
- Around line 263-287: The loop can exit on failure while FLASH_CR_PG remains
set; ensure FLASH_CR_PG is cleared before breaking: in the loop around
flash_drv_wait_last_op() (and immediately before any early return/break), clear
FLASH_CR_PG (use CLEAR_BIT(FLASH->CR, FLASH_CR_PG)) so the controller is not
left in program mode; keep the existing
__DSB()/__enable_irq()/stm32_flash_lock() epilogue but perform the
CLEAR_BIT(FLASH->CR, FLASH_CR_PG) in the failure branch right after detecting
!flash_drv_wait_last_op() and before ret = -1; this change affects the block
that sets FLASH_CR_PG and calls flash_drv_wait_last_op() in flash_driver.cpp.

---

Duplicate comments:
In `@radio/src/targets/common/arm/stm32/flash_driver.cpp`:
- Around line 187-211: The IRQ-masking here unconditionally calls
__disable_irq() then __enable_irq(), which can re-enable interrupts if the
caller already had PRIMASK set; instead save and restore PRIMASK around the
critical region: call __get_PRIMASK() into a local variable, then
__disable_irq(), keep the existing __DSB()/operation sequence
(stm32_flash_unlock(), FLASH CR ops, flash_drv_wait_last_op(), CLEAR_BIT,
__DSB(), flash_drv_flush_caches(), stm32_flash_lock()), and at the end restore
the saved state with __set_PRIMASK(saved_primask) rather than unconditionally
calling __enable_irq(); apply the same change to the other similar blocks that
use __disable_irq()/__enable_irq() in this file.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: fc7d5f4a-6394-42c7-bd70-f3cb38730cfb

📥 Commits

Reviewing files that changed from the base of the PR and between b14c674 and 182eddc.

📒 Files selected for processing (1)
  • radio/src/targets/common/arm/stm32/flash_driver.cpp

Comment on lines +263 to +287
__disable_irq();
__DSB();
stm32_flash_unlock();

while (address < end_addr) {
CLEAR_BIT(FLASH->CR, FLASH_CR_PSIZE);
FLASH->CR |= FLASH_PSIZE_WORD;
FLASH->CR |= FLASH_CR_PG;

*(__IO uint32_t*)address = *p_data;

if (!flash_drv_wait_last_op()) {
ret = -1;
break;
}

CLEAR_BIT(FLASH->CR, FLASH_CR_PG);

address += sizeof(uint32_t);
p_data++;
}

__DSB();
__enable_irq();
stm32_flash_lock();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Always clear FLASH_CR_PG in the failure path.

Line 274 breaks out before FLASH_CR_PG is cleared. If flash_drv_wait_last_op() times out or reports an error, the controller stays in program mode through the rest of the epilogue, which is unsafe for the next flash access.

Suggested fix
   while (address < end_addr) {
     CLEAR_BIT(FLASH->CR, FLASH_CR_PSIZE);
     FLASH->CR |= FLASH_PSIZE_WORD;
     FLASH->CR |= FLASH_CR_PG;
@@
     if (!flash_drv_wait_last_op()) {
       ret = -1;
       break;
     }
 
     CLEAR_BIT(FLASH->CR, FLASH_CR_PG);
@@
   }
 
+  CLEAR_BIT(FLASH->CR, FLASH_CR_PG);
   __DSB();
   __enable_irq();
   stm32_flash_lock();
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@radio/src/targets/common/arm/stm32/flash_driver.cpp` around lines 263 - 287,
The loop can exit on failure while FLASH_CR_PG remains set; ensure FLASH_CR_PG
is cleared before breaking: in the loop around flash_drv_wait_last_op() (and
immediately before any early return/break), clear FLASH_CR_PG (use
CLEAR_BIT(FLASH->CR, FLASH_CR_PG)) so the controller is not left in program
mode; keep the existing __DSB()/__enable_irq()/stm32_flash_lock() epilogue but
perform the CLEAR_BIT(FLASH->CR, FLASH_CR_PG) in the failure branch right after
detecting !flash_drv_wait_last_op() and before ret = -1; this change affects the
block that sets FLASH_CR_PG and calls flash_drv_wait_last_op() in
flash_driver.cpp.

@richardclli

Copy link
Copy Markdown
Member Author

Bootloader Flash Hangs — Root Cause Analysis & Fix

Symptom

When flashing firmware via the bootloader on STM32F2/F4 targets, the
device sometimes hangs mid-operation (erase or program). The hang is
permanent — no timeout, no watchdog reset, no progress. The user must
power-cycle the radio and retry. The symptom is intermittent and has
no single reproducible trigger.


Previous Attempt (commit b14c67441)

Added __disable_irq() / __enable_irq() around HAL_FLASHEx_Erase()
and HAL_FLASH_Program() to prevent interrupt handlers from reading
flash memory during erase/program (which stalls the bus on F4).

Why it wasn't enough: The fix correctly prevents ISR interference,
but it introduced two new failure paths (see Root Causes below) that
turn any transient flash controller glitch into a permanent deadlock
with no recovery mechanism.


Root Cause #1: Cache flush with I-cache disabled while IRQs are masked

Call chain during sector erase

stm32_flash_erase_sector()
  __disable_irq()                           ← IRQs OFF
  HAL_FLASHEx_Erase()
    FLASH_Erase_Sector()                    ← set SER | SNB | STRT
    FLASH_WaitForLastOperation()            ← poll BUSY (timeout dead, see #2)
    FLASH_FlushCaches()                     ← *** PROBLEM ***
      ICEN = 0                              ← I-cache OFF
      ICRST = 1 → 0                         ← reset cache
      ICEN = 1                              ← I-cache ON
      [same for D-cache]
  stm32_flash_lock()
  __enable_irq()                            ← IRQs ON (too late)

Why this hangs

FLASH_FlushCaches() temporarily disables the instruction cache.
During the window where ICEN = 0, every CPU instruction fetch goes
directly to the flash memory bus with 5 wait states. This is expected
behavior on F4 — BUT:

  • The cache disable/reset/re-enable sequence writes to FLASH->ACR,
    which is in the flash controller register block
  • If the flash controller has any lingering state from the just-completed
    erase (timing glitch, power dip, silicon variation), the ACR write
    may not complete cleanly
  • The CPU stalls on its next instruction fetch with no way to recover:
    • IRQs are disabled → no SysTick, no timer ISR, no watchdog feed
    • I-cache is off → every fetch hits flash, which may be stalled
    • System is permanently deadlocked

Fix

Re-enable IRQs before flushing caches:

  __disable_irq()
  DSB()
  unlock()
  set SER | SNB | STRT           ← start erase
  wait BUSY (DWT cycle counter)  ← hardware timeout guard
  clear SER | SNB                ← erase done
  DSB()
  __enable_irq()                 ← *** IRQs ON before cache flush ***
  flush_caches()                 ← safe: IRQs can fire if anything stalls
  lock()

Now if the cache flush encounters a timing glitch, a SysTick or timer
interrupt can break the stall, or the hardware watchdog (if enabled)
can reset cleanly.


Root Cause #2: Broken HAL timeouts when IRQs are disabled

The problem

FLASH_WaitForLastOperation() uses HAL_GetTick() for its timeout:

tickstart = HAL_GetTick();
while (BUSY) {
    if (HAL_GetTick() - tickstart > FLASH_TIMEOUT_VALUE)  // 50000 = 50s
        return HAL_TIMEOUT;
}

HAL_GetTick() calls timersGetMsTick() which reads _ms_ticks —
a static variable incremented only in the 1ms timer ISR
(timers_driver.cpp:86).

When __disable_irq() is active:

IRQs OFF
  → 1ms timer ISR never fires
  → _ms_ticks never increments
  → HAL_GetTick() returns the same value every time
  → (HAL_GetTick() - tickstart) == 0 always
  → 0 > 50000 is never true
  → timeout check is dead code

If FLASH_SR_BSY ever stays set (voltage sag, worn flash cell, silicon
errata), the polling loop runs forever with no escape.

Fix

Replace FLASH_WaitForLastOperation() with flash_drv_wait_last_op()
that uses the DWT->CYCCNT cycle counter:

static bool flash_drv_wait_last_op()
{
    CoreDebug->DEMCR |= CoreDebug_DEMCR_TRCENA_Msk;
    DWT->CTRL |= DWT_CTRL_CYCCNTENA_Msk;

    uint32_t start = DWT->CYCCNT;
    uint32_t timeout_cycles = FLASH_TIMEOUT_MS * (SystemCoreClock / 1000);

    while (__HAL_FLASH_GET_FLAG(FLASH_FLAG_BSY)) {
        if ((DWT->CYCCNT - start) > timeout_cycles) {
            clear_flags();
            return false;   // timeout → caller retries or resets
        }
    }

    clear_flags();
    check_errors();
    return true;
}

Why the DWT cycle counter works when HAL_GetTick() does not

Property HAL_GetTick() DWT->CYCCNT
Clock source 1ms timer (APB peripheral) Core clock (direct feed)
Increments with IRQs off? No (ISR-dependent) Yes (hardware counter)
Resolution 1 ms 1 CPU cycle (~6ns at 168MHz)
Max timeout (32-bit) ~49 days ~25.5 seconds at 168MHz
Initialized by timersInit() delaysInit() (already called at boot)

The DWT is a Cortex-M3/M4 debug peripheral. Its cycle counter runs
directly from the CPU clock independent of the interrupt controller.
delaysInit() in delays_driver.cpp already configures it, so it is
ready before any flash operation begins.

The 15-second timeout (FLASH_TIMEOUT_MS) comfortably covers even
the worst-case F4 sector erase (embedded flash: ~2-4 seconds typical,
up to ~8 seconds worst-case per STM reference manual).


Additional improvements in the fix

1. Data synchronization barriers

__DSB() is placed after __disable_irq() to ensure PRIMASK takes
full effect before touching flash control registers, and before
__enable_irq() to ensure all flash register writes complete before
any pending ISR fires.

2. Direct register access (F2/F4 only)

The erase and program sequence now writes FLASH->CR directly instead
of going through the HAL. This eliminates two dead code paths:

  • The HAL's internal __HAL_LOCK (spinlock, pointless with IRQs off)
  • The HAL's pre-operation FLASH_WaitForLastOperation (always succeeds
    since we always clear BUSY+PG+SER before returning)

For F4 program, each word cycle:

set PSIZE | PG
write data to flash address   ← starts programming
DWT-wait BUSY                 ← real timeout
check error flags
clear PG

3. Error flag handling

On timeout or error, all pending error flags (WRPERR, PGAERR,
PGPERR, PGSERR, EOP) are cleared in FLASH->SR. This prevents
residual flags from corrupting the next flash operation.

4. H7/H7RS path unchanged

The STM32H7 and H7RS flash controllers use a completely different
architecture (read-while-write, 256-bit flash words, different cache
hierarchy). The original HAL-based implementation is preserved for
those targets.


Verification: how to test

  1. Flash firmware via bootloader (both BIN menu and UF2 paths) on
    affected hardware (e.g., Taranis X9D, Horus X12S, RadioMaster
    TX16S, etc.)
  2. Repeat 50+ times to check for intermittent hangs
  3. Verify that if an error does occur (e.g., power pulled mid-erase),
    the system either completes cleanly after power-restore or reports
    the error rather than hanging forever
  4. On the next power-up, check WAS_RESET_BY_WATCHDOG_OR_SOFTWARE()
    to confirm no spurious watchdog resets occurred

@richardclli

Copy link
Copy Markdown
Member Author

@pfeerick See if this time works

@richardclli

Copy link
Copy Markdown
Member Author

I tried in v16 for a whole day long and cannot make it hangs in the middle. See if you can repeat the hangs anymore @pfeerick

@richardclli

Copy link
Copy Markdown
Member Author

@raphaelcoeffic Maybe you want to take a look about the rationale and maybe you want to make some changes.

@pfeerick pfeerick modified the milestones: 2.11.7, 2.11.8 Aug 5, 2026
@pfeerick pfeerick mentioned this pull request Aug 30, 2026
1 task done
@pfeerick
pfeerick self-requested a review August 30, 2026 08:51
@pfeerick pfeerick added the firmware (fw) General radio firmware issue, not colorlcd or B&W specific label Aug 30, 2026
@pfeerick pfeerick mentioned this pull request Aug 30, 2026
8 tasks done
@pfeerick

pfeerick commented Sep 2, 2026 •

Copy link
Copy Markdown
Member

I've done 40-50 flashes on TX16S (mk1), and while it hasn't hung during flashing a single time, I think there is room for improvement - I'll do a PR to be merged into this for review once I've had some time to test it further. Namely, since around v2.11.0, the bootloader has a major flaw where it can silently skip writing the first block after erasing, thus while it seems to now complete without hangs, you could have a have a unbootable radio (will either not boot at all, or will hang on boot - depending on the exact failure path). More nasty stuff that was hidden in the silicon errata. Also, the FLASH_TIMEOUT_MS is actually 2s maximum per the reference manual, given the driver settings in use, so will drop it from 15s to double the maximum, bringing the failure window down to around 48s rather than 3 minutes.

Needless to say, won't be in 2.12.4, hopefully 2.12.5 and also next 2.11.x release

@pfeerick
pfeerick force-pushed the richardclli/fix-f4-flash-hangs branch from 182eddc to 96f78eb Compare September 2, 2026 07:02
@3djc

3djc commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

I have pushed a branch with 2 additional commits you might want to have a look at: https://github.com/EdgeTX/edgetx/tree/3djc/f4-flash-error-reporting

@philmoz

philmoz commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

I have pushed a branch with 2 additional commits you might want to have a look at: https://github.com/EdgeTX/edgetx/tree/3djc/f4-flash-error-reporting

This branch fixes both flashing problems for me on TX16S and T15.
With current main on both of these radios flashing firmware from bootloader and flashing bootloader from firmware both fail 100% of the time (radio will not boot requiring DFU refresh).
With this branch both radios work correctly again.
Also tested rebasing this branch to current main just to be sure.

@pfeerick

pfeerick commented Sep 10, 2026 •

Copy link
Copy Markdown
Member

Looks like f4-flash-error-reporting finishes off / somewhat duplicates what I was working on last week:

https://github.com/pfeerick/edgetx/tree/pfeerick/fix-flash-write-error-propagation

It is still missing a few things though (noting now so they don't get missed):

  • does not add the ES0206 2.2.15 errata workaround - if we're fixing flashing related stuff... mitigating this is not optional as it affects all silicon versions
  • readFirmwareFile()'s result is still ignored - a mid-transfer SD read error will report success
  • it is probably still a good idea to do the memcmp as hardening - it adds almost negligible time to the flash, and is our own independent validation that the bytes were actually written if the flash controller incorrectly reports success...
  • FLASH_TIMEOUT_MS still needs turning down - datasheet max is 2s, so the 4s I turned it down to is still overly generous given this has it set to 15s! Simply means errors will be reported sooner rather than sitting there pointlessly waiting.

@pfeerick pfeerick mentioned this pull request Sep 24, 2026
19 of 21 tasks
@pfeerick
pfeerick force-pushed the richardclli/fix-f4-flash-hangs branch from 96f78eb to 640a71e Compare September 25, 2026 04:41
richardclli and others added 9 commits October 1, 2026 00:46
… flash erase/program

Replace HAL_WaitForLastOperation (dead when IRQs off) with DWT cycle
counter for real timeout. Re-enable IRQs before cache flush in erase
to prevent deadlock from FLASH_FlushCaches running with I-cache
temporarily disabled and IRQs masked. Add DSB barriers for pipeline
synchronization.
FLASH_SR error flags (WRPERR/PGAERR/PGPERR/PGSERR/OPERR) are sticky and
survive a reset. One left behind by whatever wrote the flash before us -
a DFU session, the previous firmware - makes the very next erase or
program report a failure it did not cause, which the caller then treats
as a genuine write error.

Clear them once after unlocking, before starting the operation, so the
check that follows only ever sees flags this driver produced.

Also clear FLASH_CR_PG when a word write fails. Leaving it set meant the
next erase ran with PG and SER both set, turning one failed word into a
sequence error for the rest of the session.
flashWrite() returned void and silently gave up when the sector erase
failed, so the bootloader kept advancing the progress bar and finished on
"Writing complete" over an image whose first 128KB was never written. The
radio then would not boot, which reads to users as a bricked radio rather
than a failed flash (#7726, #7748, #7758).

Make flashWrite() return whether the page was written and propagate that
up: firmwareWriteBlock() now reports FW_IN_PROGRESS/FW_DONE/FW_ERROR, and
the bootloader stops on ST_FLASH_ERROR, leaving the bar where the write
gave up and pointing at a DFU flash as the way out.

The in-firmware bootloader update had the same shape - it showed the
success popup unconditionally, even after an SD read error or an
incompatible file - so gate that on the writes having actually worked.
The Senduwing H17 board was added after flashWrite() was changed to
report failures, so its board.h still declared the old void signature
and conflicted with flash_driver.h.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
firmwareWriteBlock() discarded the result of readFirmwareFile(), so a
disk error mid-flash (after which FatFs latches fp->err and every later
read returns 0 bytes) was indistinguishable from end of file. The
bootloader flashed a truncated image and showed "Writing complete".

Report a non-OK read, or a short file before firmwareSize, as FW_ERROR
so the ST_FLASH_ERROR screen is shown instead.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
….2.15)

On dual bank STM32F42x/43x devices a read (data access or code
execution) from one bank while the other bank is being written can
corrupt the ART data cache if DCEN is set; subsequent cache hits then
return corrupted data. The bootloader executes from bank 1 with the
data cache enabled and a TX16S image crosses into bank 2 at roughly
57% of the write, so the tail of every flash was exposed. The
documented workaround is to disable the data cache before the write
and reset it before enabling it again; previously the caches were only
flushed after an erase, and not at all after a program.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The erase/program status only tells us what the flash controller
reported. Read the page back and compare it with the source buffer so
a write that the controller wrongly reports as successful is still
caught and reported as a flash failure. The cost is a 256 byte memcmp
per page, which is negligible next to programming it.

On cores with a data cache (H7) the page is invalidated first, as the
firmware may have read that area (e.g. the bootloader version) before
updating it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…maximum

FLASH_TIMEOUT_MS was 15 s. DS9405 Rev 13 Table 48 gives a maximum of 2 s
for a 128 KB sector erase at x32 parallelism (the driver's setting) and
100 us per word, and the driver never issues a bank or mass erase, so
4 s is ample and shortens the time to the error screen when the
controller does not complete an operation.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@pfeerick
pfeerick force-pushed the richardclli/fix-f4-flash-hangs branch from 640a71e to 707f734 Compare October 1, 2026 00:46
pfeerick and others added 2 commits October 2, 2026 03:02
The interrupt masking added for F4 was kept on the H7/H7RS HAL path when
F2/F4 moved to the register-level driver. There it stops the HAL timeout
from ever firing: FLASH_WaitForLastOperation() counts HAL_GetTick(), which
only advances in the ms timer interrupt, so a flash operation that never
completes becomes an infinite loop. This is the same defect the DWT timeout
fixes on F4.

Masking protects nothing on these targets. RM0433 Rev 8 4.3.7 only stalls
reads of the bank being written, and the bootloader runs its code and
vectors from ITCM/DTCM (the firmware from SDRAM), so nothing fetches from
internal flash during the erase. ES0392 Rev 15 has no flash erratum that
calls for it, and ST's FLASH_EraseProgram example does not mask either.

Restore main's behaviour for the HAL path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lags

The comment claimed the error flags survive a reset. RM0090 Rev 22 3.9.6
gives FLASH_SR a reset value of 0, so they do not. They are sticky until
written with 1, though, so a flag raised earlier in the same session (a
stray write to flash sets PGSERR) would still abort the next operation,
which is why clearing them first is worthwhile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pfeerick added a commit to pfeerick/edgetx that referenced this pull request Oct 3, 2026
…sh error screen

On 2.11 the colorlcd bootloader's lcd is a BitmapBuffer*, not an object
as on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pfeerick

pfeerick commented Oct 4, 2026

Copy link
Copy Markdown
Member

I wanna pay rise after testing this further! 🤣

Long story short, 2.11.8 backport branch with this PR merged into it, built by CI over at pfeerick#27. Only one single failure, which I pinned down to a spontaneous (!?) change to the firmware content on the TX16s SD card after I think it was 23 cycles of of 90 firmware flashes, which did not repeat again. Since it was the fault of storage, I consider this to be a complete success... Not one single bootloader firmware update lockup in 90 consecutive alternating cycles on TX16S or on T15.

PR #7415 hardware test results (bootloader build 7b0bc59/28a2d92)

Radio MCU Test Runs Result Median time
T15 F4 dual-bank Install the PR bootloader 1/1 1 passed
T15 F4 dual-bank Flash firmware from the bootloader 90/90 90 completed and boots, rate < 1 in 30 (95%) 0:25.2
T15 F4 dual-bank Read back the flash and compare it with the file 3/3 3 match (eof on firmware.bin)
T15 F4 dual-bank Flash over DFU, then straight from the bootloader 2/2 2 completed and boots
T15 F4 dual-bank Update the bootloader from firmware 3/3 3 passed
T15 F4 dual-bank Pull the SD card mid-flash 1/1 1 not possible on this radio
Zorro F4 single-bank Install the PR bootloader 1/1 1 passed
Zorro F4 single-bank Flash firmware from the bootloader 30/30 30 completed and boots, rate < 1 in 10 (95%) 0:05.7
Zorro F4 single-bank Read back the flash and compare it with the file 3/3 3 match (eof on firmware.bin)
Zorro F4 single-bank Flash over DFU, then straight from the bootloader 2/2 2 completed and boots
Zorro F4 single-bank Update the bootloader from firmware 3/3 3 passed
Zorro F4 single-bank Pull the SD card mid-flash 1/1 1 error screen shown
PA01 H7 Install the PR bootloader 1/1 1 passed
PA01 H7 Flash firmware.uf2 over USB from the bootloader 20/20 20 completed and boots, rate < 1 in 6 (95%) 0:11.8
PA01 H7 Repeat the UF2 flash from another host OS 2/2 2 completed and boots
TX16S F4 dual-bank Install the PR bootloader 1/1 1 passed
TX16S F4 dual-bank Flash firmware from the bootloader 90/90 89 completed and boots, 1 completed, won't boot 0:23.9
TX16S F4 dual-bank Read back the flash and compare it with the file 3/3 3 match (eof on firmware.bin)
TX16S F4 dual-bank Flash over DFU, then straight from the bootloader 2/2 2 completed and boots
TX16S F4 dual-bank Update the bootloader from firmware 3/3 3 passed
TX16S F4 dual-bank Pull the SD card mid-flash 1/1 1 error screen shown
x9d+ F2 Install the PR bootloader 1/1 1 passed
x9d+ F2 Flash firmware from the bootloader 30/30 30 completed and boots, rate < 1 in 10 (95%) 0:07.4
x9d+ F2 Read back the flash and compare it with the file 3/3 3 match (eof on firmware.bin)
x9d+ F2 Flash over DFU, then straight from the bootloader 2/2 2 completed and boots
x9d+ F2 Update the bootloader from firmware 3/3 3 passed
x9d+ F2 Pull the SD card mid-flash 1/1 1 error screen shown

@pfeerick
pfeerick merged commit 5204f6c into main Oct 4, 2026
39 checks passed
@pfeerick
pfeerick deleted the richardclli/fix-f4-flash-hangs branch October 4, 2026 10:25
pfeerick pushed a commit that referenced this pull request Oct 4, 2026
Co-authored-by: 3djc <3djc@gh.com>
Co-authored-by: Peter Feerick <5500713+pfeerick@users.noreply.github.com>

(cherry picked from commit 5204f6c)

Port notes for 2.11:
- The h750 board.h and c14/board.h hunks are dropped (those boards
  don't exist on 2.11). Git's rename detection paired rm-h750/board.h
  with nv14/board.h, which is left unchanged.
- The simulator stub change is applied to simpgmspace.cpp instead of
  simulib.cpp (simulib.cpp is main-only, added by #6435).
- The colorlcd boot menu's flash error screen uses lcd-> instead of
  lcd., because 2.11's colorlcd bootloader lcd is a BitmapBuffer*.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pfeerick pushed a commit that referenced this pull request Oct 4, 2026
Co-authored-by: 3djc <3djc@gh.com>
Co-authored-by: Peter Feerick <5500713+pfeerick@users.noreply.github.com>

(cherry picked from commit 5204f6c)

Port notes for 2.12: the simulator stub change is applied to
simpgmspace.cpp instead of simulib.cpp (simulib.cpp is main-only,
added by #6435).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pfeerick pfeerick mentioned this pull request Oct 4, 2026
47 of 49 tasks
pfeerick added a commit to brano2378/edgetx that referenced this pull request Oct 5, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport/2.11 To be backported to a 2.11 release also. backport/2.12 To be backported to a 2.12 release also. firmware (fw) General radio firmware issue, not colorlcd or B&W specific

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants