audiodev: a live audio path for stompbox use — an input pull-source node and a low-buffer duplex I2S loop

[bdbarnett]

Stakes

The effects program is rebuilding all 46 audioeffects classes with documented, near-zero algorithmic latency, and one intended use is a stompbox on a stage — a single effect or a chain. On today's audio path those effects cannot be played live, whatever they do internally:

• lib/audiodev/sample_out.py pumps playback in 40 ms chunks with a two-chunk lookahead.
• The "low" latency profile queues 100 ms into machine.I2S's ibuf (LOW_LATENCY_QUEUE_MS); capture is opened at queue_ms=150 (board_peripherals.audio_in()).
• The README's own measurement on the host: 52 ms steady at "low", 418 ms at the default.
• PCMInput exists (I2S, SDL, wasm) but nothing wraps a live input as an audioif pull-source, so an input cannot head an effect graph today.
• On the P4 panel, playback and capture cannot run together — both open I2S(0) (P4 panel: playback and capture cannot run together — both directions open I2S(0) #23).

A stompbox built on this path would sit at a hundred milliseconds or more round trip, against the roughly ten a player starts to feel. The effect graph itself is not the problem: a filter, a drive, a modulation or a delay's dry path add zero; only the convolver's 256-frame partition, an opt-in limiter lookahead and the pitch shifter's window add any, and those are the effects program's to minimise and document.

The ask — a platform spike, not scheduled

1. A live-input pull-source: an object over PCMInput that speaks the audiosample protocol (audiocore.get_buffer / reset_buffer) so a live input can be the source of any audioeffects class or Rack.
2. A tight duplex loop: I2S-in → graph → I2S-out with DMA-sized buffers (a few milliseconds, not a hundred), most likely pumped in C the way the output pump already is arranged; full duplex on the P4 per P4 panel: playback and capture cannot run together — both directions open I2S(0) #23.
3. A loopback latency harness: a click through a wire, output jumpered to input, measured on the P4 and the S3 at each buffer setting. The effects program's Phase 1 will measure today's path this way, wire and five-effect chain, and link the numbers here so the gap is a number.
4. The target: a round trip under 10 ms is the working assumption; the effects survey sources the perceptible-delay figure and this issue adopts it.

Boundary

The effects program (workspace docs/effects-vision.md §9a, docs/effects-roadmap.md Phase 0/1, and hardware concept C2 in docs/hardware-products.md) owns per-class latency budgets and reporting; this issue is the platform half. It is filed so the seam is real before anyone builds the pedal; it is not on a schedule.

Trigger: the Phase 1 round-trip numbers land, or a stompbox build starts.

Related: #23 (P4 duplex), #24 (mck= ignored on esp32), usbif's M5 (a USB-mic capture path that already exists and could head a graph the same way).

[bdbarnett]

Parked here under the effects roadmap's §2a (2026-09-07): the round-trip latency measurement of Phase 1. Stakes first: the effects program's live-stompbox use (vision §9a) needs the platform path's round trip measured on a board — a click through a wire and through a five-effect chain at the smallest buffer settings that do not starve — and today no board can make the loop: the P4 cannot play and capture together (#23, both directions open I2S(0)) and the S3 panel has no codec at all. The measurement kit carries the ROUNDTRIP measurement written against the board runner (audiocomponents f9e3a34, tools/effect_measurements.py), so the day either board has a duplex path the number is one run away. Reopens on #23's fix or a board with a codec and a loopback.

[claude]

Two things: one that makes item 3 much cheaper, and one that argues for a position this issue currently leaves implicit.

The loopback harness can run this week, on hardware we already own

Item 3 asks for a click-through-a-wire measurement on the P4 and the S3, and the issue hands measuring today's path to the effects program's Phase 1. But the LilyGO T-Embed can do it now, with no patch, no purchase and nothing blocked on #23: its audio_out is a MAX98357A on I2S(1) and its pcm_in is an ES7210 on I2S(0) — separate peripherals, separate pins. That is exactly why tools/audio_rig/README.md names it the rig board and says in the same breath that the P4 panel cannot do this.

So the deciding number does not have to wait on a duplex patch. Three runs:

1. Round trip with nothing else running.
2. Round trip while the ST7789 redraws at whatever rate a UI would.
3. Round trip while the encoder is being spun.

The second is the one to want most. It measures what a display actually costs the audio thread, and right now that question is answered only by the 28 ms worst pump gap recorded in the P4 panel's board config — which is a MIPI-DSI number and should not be generalised to an SPI display. The board also carries machine.Encoder on a PCNT hardware counter, so run 3 doubles as evidence about whether control input needs its own processor.

Read the results with two limits in view. AUDIO_IN is declared at 16 kHz only, because the ES7210 driver's profiles pin the MCLK ratio — so read the answer in buffer periods and scale, rather than taking the milliseconds at face value. And it is mic-to-speaker, so it says nothing about analog levels, which is fine, because analog levels are not the thing in doubt.

One clock domain is the target, and PyDevices/usbif#13 is why

Worth stating as a position here rather than leaving it to be rediscovered. PyDevices/usbif#13 — "USB audio drops out every ~4 s at 48 kHz: no device-computed feedback, so host and sink clocks drift" — is the same failure this issue's architecture choice can walk into, in a different transport. Two free-running clock domains eventually need either a feedback path or an asynchronous sample rate converter, and finding out which at the end of the build is the expensive way to find out.

So: one clock domain is the target architecture for the live path, not a nice-to-have. A separate ADC and DAC on two I2S ports is a legitimate way to get an early measurement and needs no firmware work at all — which is precisely what makes it easy to let quietly become permanent. It should not. The duplex path is PyDevices/micropython-pydevices#5, and there is an open question on that issue about whether the P4's two I2S peripherals are in fact frequency-locked, which would change this calculus if the answer is yes.

---

Generated by Claude Code

[claude]

Item 2 is smaller than it reads, because the audio path is already allocation-free

"Most likely pumped in C the way the output pump already is arranged" turns out to be the right instinct, and cheaper than the sentence suggests. I went through the sources to check whether a pull inside a DMA callback would allocate, and it doesn't — not because anyone planned for a callback, but because the node palette was written that way.

Take audiofilters.Filter. It has three m_malloc calls, at src/audiofilters/Filter.c:40, :43 and :48, and all three are inside common_hal_audiofilters_filter_construct. audiofilters_filter_get_buffer (:120) allocates nothing; it fills a buffer the constructor already owns and hands back a pointer to it. I swept the get_buffer bodies across audiofilters/ and audiodelays/ and found the same thing everywhere. synthio is the same shape — synthio_synth_synthesize works out of synth->buffers[], malloc'd at construction, plus stack scratch.

The allocation that people remember is real but it's in the control path. synthio_block_assign_slot_maybe calls mp_obj_new_float, and you reach it from constructors and from setters — assigning a parameter from Python makes a float. Nothing on the audio path does.

That inverts the problem. It isn't "make the pull allocation-free so it can run in a callback". It's "get the already-allocation-free pull off the interpreter thread", which is a different and much smaller job.

The protocol is already runtime-neutral

audioif_sample_source_t in src/shared/audioif_sample.h is a plain C pull interface — a get_buffer/reset_buffer function-pointer pair and a context, returning a pointer to a node-owned buffer. It has no MicroPython in it. The MicroPython binding is an adapter over it (src/audiocore/__init__.c), not the thing itself. A FreeRTOS task can hold an audioif_sample_source_t and pull the graph without the interpreter being involved at all.

And the pump already exists

usbif/src/usbif_i2s.c is a C FreeRTOS task that moves received USB audio into an I2S sink with no interpreter in the path, under MIT, in our own tree. Its header states the design principle better than I would: Python configures and observes; C moves isochronous bytes. What it buys, in its own words, is headroom — the interpreter can be busy elsewhere, or garbage collecting, without the audio noticing.

It is also where the duplex change lands. usbif_i2s.c:257 is i2s_new_channel(&chan_config, &usbif_i2s_tx, NULL) — that NULL is the RX handle, and PyDevices/micropython-pydevices#5 is the argument for filling it in.

What it costs

Three things, and none of them is the effects library.

The first is the thesis. The audio path becomes C, and the scripting stays in the control path. That's the same bargain usbif already struck and the same one audioif's node palette is built on, so it isn't new, but it is worth saying out loud rather than discovering later.

The second is concurrency you don't have today. Right now the pull and the Python that rewires the graph are the same thread, so they can't collide. Once the pump is on its own core, play(), deinit() and a knob turn all race it. GC isn't the danger here — MicroPython's collector is non-moving, so a pointer the pump holds stays valid across a collection — but a graph being rewired underneath a pull is.

The third is that real-time discipline stops being a convention and becomes a rule. Today a node that allocated in get_buffer would cause a hiccup. In a DMA callback it causes a dropout, and the gates would have to say so.

What it means for audioeffects

Nothing has to be rewritten. I checked all 46 effect classes and all 53 instruments in audiocomponents, and none of them computes a sample. An effect class builds a graph of audioif C nodes in its constructor and exposes the tail as .output; after that the Python object is idle until a macro moves.

One file looked like an exception and isn't. audioeffects/vibrato.py defines two Python objects with a _get_buffer, and they never run on a board: every pull dispatches through the native protocol_audiosample slot (audioif/src/audiocore/__init__.c:99), so a native node refuses a Python source, and vibrato catches that and falls back to the C path. They exist so the CPython shim reproduces block timing hardware gets for free.

Two classes do need the concurrency handoff, though, and they're worth naming now because they're the concrete shape of point two above. Compressor._route re-play()s the mixer's wet voice on entering or leaving patch 14 (audiocomponents/lib/audioeffects/compressor.py:437-442), and Phaser builds a whole new cascade on macro 5 and re-play()s the shaper on macro 7 (phaser.py:250-276). A play() on a live node re-inits it and swaps a source pointer. The pump must not pull through one of those mid-flight — the control thread stages the change and the pump adopts it at a block boundary.

What's actually missing

Of the four pieces this needs, three exist: the allocation-free node palette, the runtime-neutral pull protocol, and a working C pump precedent. What's missing is duplex I2S (PyDevices/micropython-pydevices#5), the live-input source node that is item 1 of this issue, and the parameter handoff described above.

None of this is measured on hardware — it's read from the sources. The gc.mem_free() confirmation across a few hundred pulls on a board is worth doing before anyone commits to it.

---

Generated by Claude Code