Skip to content

RaftSerialConsoleSysMod

Rob Dobson edited this page Nov 15, 2026 · 7 revisions

SerialConsole SysMod

Built-in SysMod that exposes the REST API over the console port - a UART, or the USB-Serial-JTAG port when that is the console (CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG). Lets a developer type API requests directly into the serial console and receive JSON responses, while simultaneously carrying a binary protocol channel (RICSerial / RICJSON) for tool-driven access.

Source: RaftSysMods/components/SerialConsole/

What it provides

  • ASCII command entry: type subscription?topic=devjson&rateHz=10 + Enter.
  • Deferred responses: an endpoint that replies later (e.g. devman/cmdraw, devman/typefile/check, datalog?action=start) can defer its reply, which is written when it is ready - so other commands' replies can come first. Replies include the request, so they can be matched.
  • Binary channel multiplexed on the same UART using ProtocolOverAscii: RICSerial (HDLC) frames are re-encoded so every byte has its top bit set, so they share the port with log text and typed input (bytes below 0x80). See RICREST Protocol → RICSerial framing for the encoding. Requests in these frames can have deferred responses like those on any RICREST channel.
  • Registers a Comms Channel so the binary side participates fully in the channel system.
  • XON/XOFF flow control hint via getXonXoff().

SysType configuration

{
    "name": "SerialConsole",
    "enable": 1,
    "uartNum": 0,
    "baudRate": 115200,
    "rxBuf": 4096,
    "txBuf": 4096,
    "crlfOnTx": 1,
    "protocol": "RICSerial"
}
Key Default Meaning
enable false Master switch
uartNum 0 ESP32 UART number
baudRate the console's (CONFIG_ESP_CONSOLE_UART_BAUDRATE) Baud rate - when not set the UART is left at the rate it is already running at
rxBuf 1024 on a UART, 4096 on USB-Serial-JTAG Driver receive buffer size
uartRxBuf 1024 UART driver receive buffer size when the console is on a UART and USB-Serial-JTAG is the secondary console (rxBuf is then USB-Serial-JTAG's)
txBuf 4096 Driver transmit buffer size (on a UART console this also buffers log output - see Notes)
outBlockMax 5000 with PSRAM, 1200 without Largest file block offered to a download on this channel. A block is held encoded (up to 4 times its size for the worst data) until the host has read it, so the limit is memory, not txBuf
crlfOnTx true Send CRLF on tx line endings
protocol "RICSerial" Protocol codec attached to the binary side

Receiving

Each loop() reads whatever has arrived (up to 1024 bytes) in one go. The protocol frame bytes in it are decoded from OverAscii and passed to the comms channel as one message; text goes to the command line. Processing stops after a complete command line (to run it), and any bytes left are kept for the next loop.

Before RaftSysMods 1.26.0 SerialConsole read at most 100 bytes a loop, one at a time, and a host's protocol frame of more than about 1 KB on the wire was lost (it arrived as many small messages that filled the channel's queue, and the USB-Serial-JTAG driver dropped bytes while its 1024-byte buffer was full). With the bulk read and the 4096-byte USB-Serial-JTAG buffer, file uploads over serial in 800-byte blocks sent one or two at a time work; larger batches (several KB on the wire at once) can still overflow the receive buffer, and are recovered, slowly, by the upload's timer acks.

Sending protocol frames

Outbound frames are OverAscii-encoded (at most 2 bytes per byte). The encoded frame is built on the stack when it fits in 1000 bytes, and in a heap (PSRAM where available) buffer when it is larger, so a large reply (e.g. a big filesection) can't overflow the stack.

On a USB-Serial-JTAG console a frame of any size is sent (RaftSysMods versions after 1.32.0). What fits in the driver's transmit buffer is written at once, in pieces, and the rest from loop() as the host reads, as for large text replies. While anything is waiting the channel reports that it isn't ready (outboundCanAccept), so the comms core holds later messages in the channel's queue and a file download sends its next block when the last has gone. If a frame is waiting and the host reads nothing for 2 s, what is waiting is dropped.

Before, the frame was written in one piece with a 1 ms wait. The driver takes a write whole or not at all, so a frame over txBuf bytes once encoded (4096 by default), or larger than the room in the buffer at the time, was lost without the sender knowing: any reply over about 4 KB, and download blocks over about 2.5 KB of random data (900 bytes of the worst data: HDLC then OverAscii can make a byte four), though the channel offered 5000-byte blocks.

On a UART console frames are written with uart_write_bytes, which waits for room: nothing is lost, but the main loop waits while a large frame is sent.

REST endpoints exposed

Endpoint Method Purpose
console GET Console control (e.g. show registered endpoints)

JSON commands (receiveCmdJSON)

Sent from another SysMod with getSysManager()->sendCmdJSON("SerialConsole", ...):

Command Effect
{"cmd":"releasePort"} Stop reading from and writing to the port (console input, replies, and the binary channel - sendMsg returns false) until reclaimed. The driver stays installed.
{"cmd":"reclaimPort"} Resume normal operation.
{"cmd":"set","baudRate":n,"txBuf":n,"rxBuf":n} UART consoles only: change baud rate or driver buffer sizes.

Releasing the port

releasePort is for an owner that needs the console's port as a clean byte pipe for a while - the RaftROS serial link uses it on USB-Serial-JTAG. It only stops SerialConsole itself; other console output (Raft and ESP-IDF logging, printf, ROM printf) is silenced by RaftConsolePort, which the owner claims at the same time.

Notes

  • This is the recommended way to test Raft REST endpoints during bring-up — no network required.
  • Logging output (LOG_I etc.) shares the same port; the binary protocol uses MSB-set bytes so it does not collide with log text.
  • On a USB-Serial-JTAG console SerialConsole installs the USB-Serial-JTAG driver (with rxBuf / txBuf as its ring sizes) and an ESP-IDF log hook that writes to it.
  • Log output on a UART console is buffered (RaftSysMods versions after 1.31.0). When the ESP-IDF console is on the UART that SerialConsole uses (CONFIG_ESP_CONSOLE_UART, e.g. a development build with the console on UART0 and USB-Serial-JTAG as the secondary console), SerialConsole switches ESP-IDF's console to use the UART driver (uart_vfs_dev_use_driver()). Console output (printf, Raft and ESP-IDF logging) is then copied into the driver's transmit buffer (txBuf) and sent by its interrupt, and the writing task only waits if the buffer is full. Without this, ESP-IDF's console waits for room in the UART's 128-byte FIFO for each character without yielding - about 87 us per character at 115200 baud once the FIFO is full, during which the writing task holds stdout's lock and other tasks that log wait behind it. Output before SerialConsole's setup (the boot log) and panic output are still written directly. When the console isn't on a UART (e.g. USB-Serial-JTAG only) nothing changes and no UART driver or buffer is allocated for it.
  • Large replies on USB-Serial-JTAG. The driver takes a write whole or not at all, so a reply larger than the transmit ring (txBuf) used to be lost. Replies are now written in pieces of up to 1 KB without waiting; what doesn't fit is kept (in PSRAM, up to 64 KB) and written from loop() as the host reads, and later console output queues behind it, so the main loop is never held up and replies arrive whole (e.g. a 9.5 KB devman/typefile/check reply). If more than 64 KB is waiting (nothing on the host is reading), further output is dropped and a warning logged. Log lines from other tasks write to the port directly, so one can appear between the pieces of a large reply. (On a UART console, replies are written by uart_write_bytes, which waits for room.)
  • When nothing on the host is reading the USB port (no terminal has it open), the driver's transmit ring fills and stays full. Each console write then waits up to 10 ms (RAFT_LOGGER_USB_JTAG_WRITE_TIMEOUT_MS) and the text is dropped. In RaftCore versions after 1.58.1, once a write has timed out, later writes don't wait (and are dropped) until one succeeds again. A burst of log lines therefore costs one wait rather than 10 ms per line on the logging task, which is often the main loop. When a host starts reading again, output resumes with no lines lost after that point.
    • In RaftSysMods versions after 1.23.1 the ESP-IDF log hook writes through the same code (LoggerCore::writeToConsole()), so this applies to ESP-IDF logging as well as Raft logging.
    • Main-loop timing measured with a terminal attached can differ from timing with nothing reading the port, so measure both.

Related

Clone this wiki locally