Feature request: TX FIFO for UARTs

[digitalentity]

UART Write() functions are blocking (by design), however for most use-cases a "fire and forget" approach is more appropriate - e.g. when using an UART as a data logger or a one-way control.

This feature request is for adding a RingBuffer to UART's Write() and WriteBuf() functions and make the actual transmission happen in an interrupt handler. The current implementation could remain as WriteSync() to allow usage from the ISR and panic().

[eliasnaur]

In Go, blocking is usually taken care of by way of separate goroutines. Why not here?

[digitalentity]

Let's consider this code:

func asyncWrite(s []byte) {
	go uart.Write(s)
}

Looks clean idiomatic go, but let's look at the uart.Write implementation:

// Write data over the UART's Tx.
// This function blocks until the data is finished being sent.
func (uart *UART) Write(data []byte) (n int, err error) {
	for i, v := range data {
		err = uart.writeByte(v)
		if err != nil {
			return i, err
		}
	}
	uart.flush() // flush() blocks until all data has been transmitted.
	return len(data), nil
}

This function blocks until the entire slice is written. Now the gotcha - our scheduler is not preemptive, meaning the entire programm will be blocked until the write is complete. The go uart.Write() is also blocking.

Even if we do a hack like this:

func (uart *UART) Write(data []byte) (n int, err error) {
	for i, v := range data {
		err = uart.writeByte(v)
		if err != nil {
			return i, err
		}
		runtime.Gosched() // <-- yield every character
	}
	uart.flush() // flush() blocks until all data has been transmitted.
	return len(data), nil
}

this doesn't entirely solve the problem:

• not all TinyGo platforms support goroutines, which makes the code which makes asynchronous IO non-portable. The code above will simply not compile for e.g. nRF51.
• we pay a scheduler pass overhead every character which means higher CPU usage and low throughput - on 115200 one byte takes 86 uS to transmit. Unless another goroutine can release control to the scheduler in time, the Write() will have to wait and send nothing.

Also, a goroutine means stack allocation, which means additional memory and GC pressure on an already resource-constrained embedded platform.

An interrupt (or even DMA where supported), on the other hand, will have a much less overhead:

• We allocate a fixed-sized ring buffer for TX - meaning memory requirements are known upfront and no GC pressure because the buffer is always referenced.
• Interrupt is served in the MSP context but does not necessarily cause the scheduler pass. We still pay for the context switch to/from the MSP stack, but we don't pay the scheduler overhead and we only incur the latency of the context switch per character, which means much higher throughput.
• Interrupts are supported on most (if not all) platforms, making the IO code concurrent even with scheduler.none.

[aykevl]

I think the default should remain at "write and wait until completion", and we should add a new API for "write and complete later". See: tinygo-org/tinygo-site#342 for a similar proposal (that we never managed to find agreement on).

[eliasnaur]

Let's consider this code:

func asyncWrite(s []byte) {
	go uart.Write(s)
}

Looks clean idiomatic go

This is not idiomatic. A caller calls Write, and the caller blocks, allowing other goroutines to run on the processor. No new goroutines are created.

but let's look at the uart.Write implementation:

// Write data over the UART's Tx.
// This function blocks until the data is finished being sent.
func (uart *UART) Write(data []byte) (n int, err error) {
	for i, v := range data {
		err = uart.writeByte(v)
		if err != nil {
			return i, err
		}
	}
	uart.flush() // flush() blocks until all data has been transmitted.
	return len(data), nil
}

This function blocks until the entire slice is written. Now the gotcha - our scheduler is not preemptive, meaning the entire programm will be blocked until the write is complete. The go uart.Write() is also blocking.

Even if we do a hack like this:

func (uart *UART) Write(data []byte) (n int, err error) {
	for i, v := range data {
		err = uart.writeByte(v)
		if err != nil {
			return i, err
		}
		runtime.Gosched() // <-- yield every character
	}
	uart.flush() // flush() blocks until all data has been transmitted.
	return len(data), nil
}

I argue that the scheduler should be preemptive, but even if not, good implementation of Write will surely use DMA to transfer data to the peripheral, and explicitly notify the scheduler that it's blocked, pending a completion interrupt (or a timer if the peripheral is deterministic).

this doesn't entirely solve the problem:

* not all TinyGo platforms support goroutines, which makes the code which makes asynchronous IO non-portable. The code above will simply not compile for e.g. nRF51.

Why doesn't nRF51 support goroutines?

* we pay a scheduler pass overhead every character which means higher CPU usage and low throughput - on 115200 one byte takes 86 uS to transmit. Unless _another_ goroutine can release control to the scheduler in time, the `Write()` will have to wait and send nothing.

With DMA you pay for at most two scheduler transitions.

Also, a goroutine means stack allocation, which means additional memory and GC pressure on an already resource-constrained embedded platform.

No new goroutines are created.

An interrupt (or even DMA where supported), on the other hand, will have a much less overhead:

* We allocate a fixed-sized ring buffer for TX - meaning memory requirements are known upfront and no GC pressure because the buffer is always referenced.

Note that with asynchronous Write you have ensure the buffer is pinned until completion. With synchronous Write, completion happens inside Write, simplifying pinning to approximately defer pin(buffer).

Another thing: with asynchronous Write, how do you notify the caller of any errors and the number of bytes successfully written?

[digitalentity]

A caller calls Write, and the caller blocks, allowing other goroutines to run on the processor. No new goroutines are created.

This is not how things work currently. A caller calls Write and busy-waits until the call returns. I agree that the Write itself should yield to the scheduler in this case, but this is not how things work currently.

Why doesn't nRF51 support goroutines?

Sorry, meant AVR (although it's quite an old platform, it still supports interrupt-driven UART ops).

With DMA you pay for at most two scheduler transitions.

Yes, on platforms where DMA can be used. On STM32 in some specific cases you unfortunately need DMA for more time-sensitive stuff (e.g. real-time sequence generation via a timer) and DMA channels are not always available for all of the use-cases).

Another thing: with asynchronous Write, how do you notify the caller of any errors and the number of bytes successfully written?

That's a very good argument in favor of blocking writes. On a Big Go file IO Write() is sychronous, but the underlying filesystem may be doing write-back caching effectively resulting in "all bytes written, no errors" because the actual write to disk happens lazily under the hood.

We don't have the OS, but we can provide the same semantics - write to the buffer is a success. actual write to the peripheral happens asynchronously.

On a side note: for any practical embedded use-case I don't really care who yields as long as I don't get extra GC pressure and can utilize the full bandwidth of the peripheral (which in case of a UART means minimal gaps between bytes).

[eliasnaur]

A caller calls Write, and the caller blocks, allowing other goroutines to run on the processor. No new goroutines are created.

This is not how things work currently. A caller calls Write and busy-waits until the call returns. I agree that the Write itself should yield to the scheduler in this case, but this is not how things work currently.

Then let's improve Write before adding another API.

Why doesn't nRF51 support goroutines?

Sorry, meant AVR (although it's quite an old platform, it still supports interrupt-driven UART ops).

Point taken. In which cases does AsyncWrite help AVR? It seems to me a platform has too few resources for goroutines also doesn't have the resources to keep an UART write running while the processor does other things.

With DMA you pay for at most two scheduler transitions.

Yes, on platforms where DMA can be used. On STM32 in some specific cases you unfortunately need DMA for more time-sensitive stuff (e.g. real-time sequence generation via a timer) and DMA channels are not always available for all of the use-cases).

Surely AsyncWrite runs into the same issue? In other words, whatever AsyncWrite can do to offload the main processor, Write can do the same, and notify the scheduler to allow other goroutines to run.

Another thing: with asynchronous Write, how do you notify the caller of any errors and the number of bytes successfully written?

That's a very good argument in favor of blocking writes. On a Big Go file IO Write() is sychronous, but the underlying filesystem may be doing write-back caching effectively resulting in "all bytes written, no errors" because the actual write to disk happens lazily under the hood.

We don't have the OS, but we can provide the same semantics - write to the buffer is a success. actual write to the peripheral happens asynchronously.

An streaming I/O peripheral is not quite the same as io.File. For files, you often only need to know whether a (series of) Write went through or not, and you have Close() error to report any delayed errors (for durability, the OS usually provides afsync API for ensuring the writes have reached the storage medium).
For an I/O peripheral, the Close API is almost never called because you're continuously streaming new data to it, and even when an error occurs you want to retry.

[digitalentity]

For an I/O peripheral, the Close API is almost never called because you're continuously streaming new data to it, and even when an error occurs you want to retry.

Point taken.

Let's also consider the established patterns in embedded OSes, e.g.:

• ChibiOS (uses queues): https://github.com/ChibiOS/ChibiOS/blob/master/os/hal/src/hal_serial.c#L50-L54
• FreeRTOS (uses queues): https://github.com/FreeRTOS/FreeRTOS/blob/main/FreeRTOS/Demo/CORTEX_STM32F103_GCC_Rowley/Drivers/STM32_USART.c#L243-L258

They typically have buffered/queued API and use DMA/IRQ under the hood to talk to the peripheral. They also return an error if the TX FIFO is exhausted. Actual HW errors are often propagated to the next write attempt.

[eliasnaur]

Note that operating systems that don't offer lightweight threads need to supply concurrency in some other way, such as with asynchronous writes. The point of goroutines is to get rid of most other special-cased approaches to concurrency.

[digitalentity]

I would argue that FreeRTOS's threads and TinyGo's goroutines have similar ctx switching overhead, at least on Cortex-M arch. The Go's approach to concurency is rather different to what is considered established practices in embedded development.

The async-first approach, for example, allows me to generate a constant stream of data in one thread. Essentially I am preparing a new batch while the previous one is being transmitted.

func PrepAndSend() {
    var buf [100]byte
    for {
        n := CPUIntensiveDataPreparation(buf)
        err := uart.WriteAsync(buf[0:n-1]) // Data is copied into the internal TX buffer, I am free to reuse buf immediately
        // I get an error here in case of tx queue overflow.
    }
}

With a blocking Write I need an additional goroutine to do the sending, make sure I don't run two sending goroutines at a time and deal with double-buffering myself:

func PrepAndSend() {
    var (
		buf [100]byte
		mu sync.Mutex
		isSending bool
	)

    for {
        n := CPUIntensiveDataPreparation(buf)
		mu.Lock()
		if isSending {
			mu.Unlock()
			continue
		}
		isSending = true
		mu.Unlock()
		go func() {
			buf2 := buf[0:n-1] // Need a copy of the buffer
			uart.Write(buf2)
			mu.Lock()
			isSending = false
			mu.Unlock()
		}()
    }
}

I am sure there is a more go-friendly way to do this, but from a distance having to manage IO concurrency manually is a huge and unnecessary hassle. With production embedded IO (not Arduino-style) one usually expects IO asynchrony to be built-in and not something you have to manage yourself.

On a side note: I think providing async alternatives to the go idiomatic patters might drive adoption of TinyGo higher for new projects.

[eliasnaur]

The async-first approach, for example, allows me to generate a constant stream of data in one thread. Essentially I am preparing a new batch while the previous one is being transmitted.

func PrepAndSend() {
var buf [100]byte
for {
n := CPUIntensiveDataPreparation(buf)
err := uart.WriteAsync(buf[0:n-1]) // Data is copied into the internal TX buffer, I am free to reuse buf immediately
// I get an error here in case of tx queue overflow.
}
}

With a blocking Write I need an additional goroutine to do the sending, make sure I don't run two sending goroutines at a time and deal with double-buffering myself

I don't understand how you magically avoid the double buffer problem with WriteAsync. You write "the internal TX buffer" which is the second buffer. In your model every async driver must now manage a buffer, in particular its size and lifetime.

Here's a more idiomatic PrepAndSend using channels (untested):

func PrepAndSend() error {
    // Double-buffering.
    avail := make(chan []byte, 2)
    defer close(avail)
    processed := make(chan []byte, cap(avail))
    for range cap(avail) {
         avail<-make([]byte, 100)
    }

    go func() {
        for buf := range avail {
            n := CPUHeavy(buf)
            processed<-buf[:n]
        }
    }()
    for buf := range processed {
        err := uart.Write(buf)
        if err != nil {
            return err
        }
        avail<-buf[:cap(buf):cap(buf)]
    }
    return nil
}

[digitalentity]

I don't understand how you magically avoid the double buffer problem with WriteAsync. You write "the internal TX buffer" which is the second buffer. In your model every async driver must now manage a buffer, in particular its size and lifetime.

I don't avoid the second buffer, I avoid allocating it every time I need to send something. The driver allocates once on creation of the interface. With this I can operate at essentially 0 bytes free in the heap without risking an OOM.

Here's a more idiomatic PrepAndSend using channels (untested):

Good point about the channels and double-buffering. Still looks significantly more complex and harder to understand than async IO though.

And the main problem is still that Write is not scheduler friendly - which is probably the first thing we should fix.

[aykevl]

I agree with @digitalentity that we need async I/O. I've run into this before, and in that particular case goroutines weren't even a proper solution (for complex reasons) - only async I/O could solve it. In some cases you need fine grained control over blocking behavior, and goroutines are not fine grained enough.

Also, goroutines have a big overhead and are best avoided for high performance code (explicit async I/O will be much faster). Just because goroutines are lightweight on desktop-class systems doesn't mean they're also lightweight on microcontrollers.

[digitalentity]

What is the decision here?

I would be in favor of extending the interface:

type SerialIO interface {
  WriteByte(c byte) error  // Blocking
  Write(data []byte) (n int, err error)  // Blocking
  Buffered() int
  ReadByte() (byte, error)  // Blocking

  WriteAsync(data []byte) (n int, err error)  // Async
  ReadAsync(data []byte, n int) (n int, err error) // Async
  Sync() // Blocking

Behavior of the new functions is as follows:

WriteAsync(data []byte) (n int, err error)

• Writes up to len(data) bytes to the FIFO and returns immediately
• Returns n - number of bytes actually written to FIFO (handles the case when caller wants to write more data than FIFO can take), caller can then decide whether to wait or drop the rest.
• Non-nil error is returned when transmission can't be reasonably expected to complete (e.g. USB VCP + cable not connected)

ReadAsync(data []byte, n int) (n int, err error)

• Reads up to min(n, len(data)) bytes and returns number of bytes read.
• May return n == 0 if the receive FIFO is empty

Sync()

• Waits until all the data in the transmission FIFO physically leave the MCU, effectively making the IO synchronous.

I would also suggest making the IO async by default and add the Sync() API, but this will not be backwards-compatible.