Stage: transient text-cell video

Status: protocol, relay, authority, producer session, reference FFmpeg/Chafa projector, optional synchronized AAC track, selected v1 visual profile, production-client Stage surface, cross-device acceptance, and guided publishing workflow implemented

Stage is one live text-cell presentation surface attached to a channel. It is intended for short clips, deliberate movie-night broadcasts, camera adapters, and other explicitly started live displays. People may call it ASCII video, but the protocol permits a curated fixed-cell Unicode glyph table so block and shade characters can improve representation without making arbitrary Unicode part of the relay.

Product boundary

A channel has at most one active Stage publisher.
Stage frames use a separate authenticated WebSocket and are not IRC PRIVMSG, NOTICE, or TAGMSG traffic.
Frames never become retained events, history, unread counts, delivery/read cursors, reactions, replies, search results, status-event spam, backups, or content-reset state.
The daemon relays already-quantized frames. It does not decode video, choose glyphs, quantize colors, resize, transcode, record, or generate thumbnails.
One publisher provides one rendition initially. Clients scale it to their presentation surface; multiple simultaneous renditions are deferred.
Audio is an optional, codec-negotiated track on the same transient Stage session. It is never an IRC message or retained server object. The initial profile is AAC-LC in fragmented MP4; viewers that do not advertise that MIME type continue to receive video alone.
Agent faces require no server-specific concept. A future adapter can produce ordinary source frames for the same projector library.

Reference pipeline

file / camera / generated frames
             |
          FFmpeg
      /             \
 video decode     AAC-LC/fMP4
 scale, PTS      fragments, PTS
      |
           Chafa
 curated glyph and color cells
      |             /
    Telex Stage projector
 typed packets and timestamps
             |
      producer WebSocket
             |
       Telex Stage relay
 bounded packet recovery window
   plus current initialization
             |
       viewer WebSockets
             |
       dedicated Canvas

FFmpeg owns source decoding and the reference AAC-LC encode. Chafa is the reference quantizer because its stable C API exposes the selected glyph and foreground/background color for each cell. The file projector produces a constant-rate visual rendition and derives visual presentation timestamps from the selected frame rate. Its audio rendition starts at the same epoch. A later live or variable-rate adapter can preserve source timestamps in the same Stage fields. The Stage protocol remains encoder-neutral; a compatible Rust, Go, Python, browser, camera, or generated producer does not need Chafa.

Stable envelope

telex.stage.v1 uses a codec-independent initialization description followed by typed binary packets. The stable binary header contains:

magic and wire version;
message kind and bounded flags;
extensible header length;
unsigned 64-bit packet sequence;
unsigned 64-bit presentation timestamp in the advertised timebase;
unsigned 32-bit duration;
unsigned 32-bit opaque payload length; and
reserved/extension bytes for compatible evolution.

The 32-bit wire length can describe almost 4 GiB and is not the deployment allowance. Each server separately admits bounded geometry, frame size, frame rate, and aggregate bytes per second. Those operational limits can change without changing the protocol. The relay validates the stable envelope and passes codec bytes opaquely.

Packet kind 1 is a complete independent visual frame. Kind 2 is one audio initialization segment (ftyp plus moov for the selected profile), and kind 3 is one complete audio media fragment (moof plus mdat). Video and audio share a strictly increasing transport sequence while each track preserves its own monotonic presentation clock.

A reconnecting or late viewer receives the current initialization state, the bounded recent frame window, and—when it advertised the active audio MIME type—the audio initialization plus a bounded recent fragment window. Viewers that fall outside that window skip forward instead of making the publisher or other viewers wait. The envelope reserves compatible flags and codec identifiers so independently compressed full frames or a later adaptive codec do not require a new WebSocket transport.

Because the server does not transcode, "negotiation" means compatibility checking: the publisher announces the single active codec and viewers either support it or report that they cannot render the Stage. All bundled clients must support the eventual baseline codec.

Implemented transport

Stage reuses the control listener and account/app-key authentication without putting a credential into a WebSocket URL.

POST /v1/stage-sessions uses HTTP Basic authentication and a JSON body:
```
{"channel":"#movie-night","role":"viewer"}
```
role is viewer or publisher. Success returns a 30-second, one-use st1. ticket, canonical channel and conversation identifiers, and current geometry/frame-rate/frame-size limits.
The client opens /v1/stage with the exact WebSocket subprotocol telex.stage.v1.

The first client message consumes the ticket:

{"type":"hello","api_version":"1","ticket":"st1.…"}

A publisher includes its validated init object in this message. A viewer must not. A viewer may instead advertise a bounded exact-match list:

{
  "type": "hello",
  "api_version": "1",
  "ticket": "st1.…",
  "audio_mime_types": ["audio/mp4; codecs=\"mp4a.40.2\""]
}

The server sends ready. Viewers then receive stage.start, supported binary packets, and stage.end as a publisher starts, advances, and leaves.

The relay keeps the active initialization, a short bounded video window, one audio initialization segment, and approximately two seconds of recent audio fragments. No Stage state is durable. Publisher and viewer authority is rechecked while the socket remains open. The server permits one publisher per canonical channel, bounds global active Stages and ingress, and never interprets MP4 or codec payloads. Audio packets are withheld from viewers that did not advertise the publisher's exact MIME type.

The shared JavaScript StageSession performs the ticket exchange, requires the selected subprotocol, validates server limits, accepts browser ArrayBuffer/Blob frames, and drops publisher output under transport backpressure instead of extending latency. Required audio packets are retried within a short bound because dropping an arbitrary fragmented-MP4 segment would corrupt the track; video remains disposable.

The browser StageViewerSurface composes the session with a bounded timestamp scheduler, device-pixel-ratio-aware Canvas renderer, and Media Source audio player. The production web client loads that surface only when a person explicitly opens stage in a joined channel. The Stage dock contains the complete source geometry within its viewport, shares the workspace with the timeline and composer, and returns focus to the channel control when closed. If browser autoplay policy holds the audio, the dock exposes a terse explicit play action rather than advancing silent video. The same dock provides local volume and mute controls without changing the stream seen by other viewers.

Initial cell codec

telex.cells version 1 uses a stream-initial glyph dictionary of at most 256 entries. Each entry is one non-control Unicode scalar in this version. A cell carries an eight-bit glyph index, foreground color index, and background color index.

Two color models are defined for comparison:

xterm256 uses the fixed terminal palette and exactly three bytes per cell;
palette256 prefixes every independent frame with one to 256 packed RGB colors, then uses the same three-byte cells.

The per-frame custom palette adds at most 770 bytes and avoids binding the visual baseline to xterm's fixed colors. A future codec may add RGB colors, different glyph indexing, compression, or another complete representation inside the same opaque Stage envelope.

The reference symbol profiles are:

ascii: space through tilde;
terminal: the ASCII profile plus ░▒▓█▀▄▌▐; and
video: a selected compact luminance ramp ( .:-=+*#%@░▒▓▀▄▌▐) that preserves visible character structure without collapsing into a full-block pixel mosaic.

Arbitrary combining sequences, emoji clusters, and ambiguous terminal-width text are deliberately outside telex.cells v1. Canvas uses the declared table as a fixed glyph atlas rather than relying on operating-system terminal width. The envelope remains codec-agnostic, so a future Braille or compressed profile can be introduced as another advertised codec rather than changing Stage transport.

Selected v1 presentation profile

The reference video publisher defaults to:

Setting	Selected behavior
geometry	160 columns; rows derived from source display aspect
frame rate	24 fps
glyph profile	`video`
color model	`xterm256`
Chafa color mode	foreground glyph color on black

The row calculation accounts for the reference 8x16 cell shape. It does not force every source into 16:9 or transmit black padding cells:

Source display aspect	Cell geometry	Raw cell rate
5:4	160x64	about 5.9 Mbit/s
comparison clip (704x570)	160x65	about 6.0 Mbit/s
16:9	160x45	about 4.1 Mbit/s

The raw rates use three xterm cell bytes at 24 fps and exclude the small Stage header plus WebSocket/TLS overhead. Relay egress multiplies by active viewers. An explicit admitted row count still overrides automatic geometry, and the protocol permits any bounded per-Stage geometry. This is aspect-ratio support, not server-side quality adaptation or multiple renditions.

On the comparison machine, projecting the 25.75-second 704x570 H.264 source at 160x65 and 30 fps completed in about 2.4 seconds, over ten times real time. The 200-column comparison would raise the selected 24 fps raw stream to about 9.3 Mbit/s without enough visual gain to justify making it the default. Sixty fps would raise it to about 15 Mbit/s and likewise remains an explicit opt-in.

Timing and rendering

Presentation timestamps are relative to one stream epoch. The relay preserves a bounded two-second-class packet window at the selected profile. For a silent Stage, the web client adds a one-second playout lead, retains at most 48 queued frames, and schedules against a monotonic browser clock. For an audio Stage, playback waits until both its AAC buffer and visual frame queue hold the playout lead; the media element then becomes the presentation clock. Either path drops late superseded video frames rather than accumulating delay. A discontinuity flag resets the silent schedule after seeking or a source-clock jump. stage.end drains the bounded presentation before clearing the surface.

The bundled client uses a purpose-built Canvas renderer rather than a terminal emulator. It owns fixed-cell geometry and the declared glyph table, a device-pixel-ratio-aware backing surface, explicit background/foreground drawing, newest-frame scheduling, resize without logical-grid mutation, and bounded memory. Canvas2D is the baseline. The backing surface honors device pixel ratio only within a fixed dimension and pixel-allocation ceiling so a maximum admitted Stage cannot force an unbounded mobile Canvas. The protocol permits a later WebGL renderer without change.

Production hosting must admit both the control HTTPS origin and its Stage WebSocket counterpart in connect-src (for example, https://irc.example.net:8098 and wss://irc.example.net:8098). Allowing the IRC WebSocket alone is insufficient because Stage deliberately uses the control listener rather than the /irc transport. The policy must also permit blob: in media-src; Media Source attaches its in-memory fragmented MP4 stream to the audio element through a blob URL.

Initial audio profile

The optional baseline audio track is:

Setting	Selected behavior
container	fragmented MP4 (`empty_moov` + `default_base_moof`)
codec	AAC-LC (`mp4a.40.2`)
sample rate	48 kHz
channels	stereo output
target bitrate	128 kbit/s
fragment target	500 ms

At the selected bitrate the audio adds roughly 16 KiB/s, or 0.128 Mbit/s, to publisher ingress and to relay egress per listening viewer, plus small MP4/WebSocket/TLS overhead. That is minor beside the 4–6 Mbit/s default cell video. FFmpeg automatically mixes or duplicates a source channel layout into the declared stereo output. The reference file publisher normalizes by default to -16 LUFS with a -1.5 dB true-peak ceiling; --raw-audio preserves source levels when mastering must remain untouched. A source without audio remains a silent Stage; --silent explicitly suppresses audio even when the source has it.

AAC/fMP4 is a negotiated profile rather than an eternal protocol assumption. The initialization's exact MIME type and the viewer capability list permit a later Opus/WebM or other profile without changing the typed Stage envelope. The web client prefers WebKit's ManagedMediaSource when it is available and supports the selected MIME type, then falls back to standard MediaSource. Stage has no transferable AirPlay URL, so the client explicitly disables remote playback on its in-memory audio element as required for managed media source availability on iPhone Safari. Audible playback still follows browser autoplay policy and exposes the existing explicit play action when a user gesture is required.

Authority

Stage reuses Telex authority rather than introducing a second credential system:

browser clients exchange their existing memory-only account credentials for a short-lived one-use Stage ticket;
a human viewer must have current access to the attached channel;
a human publisher must have channel owner/operator authority;
automated publication requires a separate explicit stage.publish channel grant rather than inheriting ordinary message.publish; and
leaving, banning, disabling, changing a human password, revoking an automation app key or grant, or ending the publisher terminates the corresponding live authority.

The one-use-ticket and exact-origin pattern matches the native gateway. No credential belongs in the WebSocket URL or Stage frame payload. An automated publisher can hold only stage.publish; it does not need or inherit native message, history, roster, invocation, or content access.

Implementation and comparison sequence

The comparison result selects the required baseline; it does not alter the stable relay envelope.

The deployed reference clip passed Android Chrome as an installed PWA over 5G, macOS Helium, and iPhone Safari on 2026-07-27. The iPhone path uses ManagedMediaSource, explicitly disables unavailable remote playback, and falls back to the Stage play action when audible autoplay requires a user gesture.

examples/stage/synthetic-publisher.js exercises the complete ticket, WebSocket, relay, timing, frame, and backpressure path with generated cells. examples/stage/video-publisher.js implements the selected adaptive FFmpeg/Chafa file pipeline. examples/stage/frame-preview.js projects an exact source frame through the same helper into a self-contained Canvas preview. examples/stage/stage-publisher.js ties those primitives into explicit doctor, preview, publish, encode, sample, inspect, play, and station commands. Its compact, balanced, and detail presets keep 24 fps and automatic aspect-derived rows while selecting 120, 160, or 200 columns. They remain overridable publisher defaults rather than daemon policy or additional wire codecs. The Stage publishing guide documents the standalone kit, deterministic sample, archive validation, playlist/checkpoint behavior, container, XDCC exchange, and narrow stage.publish provisioning path.

FilesExpand file tree

stage.md

Latest commit

History