Skip to content

File downloads: paced by the channel, with a real fallback to smaller blocks - #56

Merged
robdobsn merged 1 commit into
mainfrom
feature/transfer-flow-control
Nov 15, 2026
Merged

robdobsn merged 1 commit into
mainfrom
feature/transfer-flow-control

Conversation

@robdobsn

Copy link
Copy Markdown
Owner

Why

  • Downloads were paced by a fixed 100 ms between blocks on every link (50 kB/s over WiFi, 5 kB/s over BLE).
  • Over the USB serial console, downloads with 5,000-byte blocks failed: the console offered blocks its transport could not send, they were dropped silently, and nothing fell back to a smaller size.

What changed

  • Pacing: the next block is sent when the last has left the channel's outbound queue (CommsCoreIF::outboundQueuedCount(), new) and the transport is ready (outboundCanAccept()), within the ack window.
  • Start: the first block waits for the client's first dfAck, or 200 ms. Without the 100 ms gap, a block sent straight after the reply to dfStart could reach a client before it had handled that reply, and clients ignore blocks for a download they have not set up (seen as a 3 s pause at the start of some WiFi downloads). Released raftjs already acks at once.
  • Fallback: when nothing is acknowledged for 3 s, the block size is halved (down to 64 bytes) before resending.
  • CommsChannelMsg::setAckedByProtocol(): blocks are marked as covered by the protocol's own acks, so a transport need not confirm each packet (BLE uses notifications for them).

Results (ESP32-S3 Axiom, with the matching RaftSysMods and RaftWebServer PRs)

Link Before After
USB serial failed 320-330 kB/s
WiFi 48 kB/s 125 kB/s (190-200 with the TCP settings in the wiki)
BLE 5 kB/s 50-52 kB/s

A TinyS3 built without PSRAM gives 230 kB/s over serial and 44 kB/s over BLE.

Testing

  • linux_unit_tests: new test_file_download_okto; make check passes.
  • Hardware: Axiom018 and a TinyS3 over serial, WiFi and BLE with raftpy; released and new raftjs over WiFi.

Notes

  • Depends on nothing else to build. The speed-up on serial and BLE needs the RaftSysMods PR, and WiFi needs the RaftWebServer PR (without it a burst can overflow the WebSocket send queue).
  • Details: devdocs/file-transfer-flow-control.md in raftpy.

🤖 Generated with Claude Code

…l fallback to smaller blocks

- Blocks are sent when the last has left the channel's outbound queue (CommsCoreIF::outboundQueuedCount) and
  its transport is ready (outboundCanAccept), within the ack window, instead of one every 100 ms
- The first block waits for the client's first dfAck (clients ack as soon as they are ready to receive), or
  200 ms: a block sent straight after the reply to dfStart can reach a client before it has handled the reply
- When nothing is acknowledged for 3 s the block size is halved (down to 64 bytes) before resending, in case
  the channel offers blocks its transport can't deliver
- Blocks are marked as acknowledged by the protocol (CommsChannelMsg::setAckedByProtocol), so a transport
  needn't confirm each packet (BLE sends them as notifications)
- test_file_download_okto

Measured on an ESP32-S3 (Axiom): USB serial 320 kB/s (downloads of 5000-byte blocks failed before), WiFi
125 kB/s (48), BLE 50 kB/s (5), with the matching RaftSysMods and RaftWebServer changes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@robdobsn
robdobsn merged commit 0f21b0f into main Nov 15, 2026
1 check passed
@robdobsn
robdobsn deleted the feature/transfer-flow-control branch November 15, 2026 18:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant