feat: stream HTTP responses (SSE/chunked) and generalise scheduled push to core - #102
Draft
outofcoffee wants to merge 4 commits into
Draft
outofcoffee wants to merge 4 commits into
outofcoffee wants to merge 4 commits into
Conversation
Add `stream: true` on rest/openapi resources to deliver output incrementally (HTTP chunked / Server-Sent Events) instead of buffering a single body: - a `responses` list is flushed one chunk at a time, paced by each block's `delay` (e.g. an OpenAI-style token stream ending in [DONE]); - a `schedule` on a streamed resource pushes further responses over the open connection until its limit is hit or the client disconnects (SSE keepalive, progress, live data). The sending and scheduling machinery that was specific to the websocket plugin is generalised into a new core package, `internal/emit`: - `Sink` abstracts "send one processed body" (a websocket frame, or an HTTP flushed chunk); - `EmitResponses` is the shared process-then-emit loop; - `ScheduleHost` runs connection- or request-scoped schedules via the existing `internal/scheduler`, emitting through a Sink. The websocket plugin is refactored onto these primitives (a `frameSink` plus `ScheduleHost`), removing its bespoke send/schedule code. OpenAPI inherits streaming for free because it already serves responses through the REST pipeline. When the connection cannot flush (AWS Lambda adapter), the chunks are concatenated into one buffered response and schedules are skipped. Validation: multiple `responses`, or a `schedule`, on an HTTP resource now require `stream: true`; `stream: true` with nothing to stream is rejected; the schedule-entry checks are shared between websocket and HTTP resources. Adds unit tests for the emit package (including the buffered fallback and schedule limit/cancel), streaming validation tests, an `examples/rest/ sse-streaming` example and `docs/streaming.md`. Verified end to end on the native engine: incremental SSE pacing, scheduled push with limit and client-disconnect, and OpenAPI streaming. Claude-Session: https://claude.ai/code/session_01NGFQaVnKN4wv322agMownn
A websocket connection is inherently a streaming, multi-frame transport, so it is implicitly 'stream: true'. The real misconfiguration is an explicit 'stream: false', which asserts something the plugin cannot honour — but a plain bool cannot tell 'false' from 'unset', so the previous code instead warned on the harmless-and-consistent 'stream: true'. Make BaseResource.Stream a *bool (tri-state) with a StreamEnabled() helper: - websocket: 'stream: true'/unset are accepted silently; explicit 'stream: false' is now a startup error. - rest/openapi: unset and 'stream: false' both mean the default (buffered); 'stream: true' opts into incremental streaming, as before. - other plugins: only an explicit 'stream: true' is rejected as unsupported. Adds validation tests for the websocket true/false cases and updates the streaming doc. Full suite passes. Claude-Session: https://claude.ai/code/session_01NGFQaVnKN4wv322agMownn
Use a plain progress-style SSE stream in the fixed-sequence example instead of OpenAI chat-completion chunks, and drop the "OpenAI compatible" aside, so the guide teaches the feature generically. The concrete OpenAI-shaped case stays in the examples/rest/sse-streaming example. Claude-Session: https://claude.ai/code/session_01NGFQaVnKN4wv322agMownn
Reword doc comments that narrated how the code came to be (e.g. "the generalisation of the websocket plugin's connection-scoped schedules") to describe what the code does now, and drop speculative "leaves room for later" rationale. Claude-Session: https://claude.ai/code/session_01NGFQaVnKN4wv322agMownn
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds incremental HTTP streaming to the
restandopenapiplugins viastream: true, and generalises the websocket plugin's multiple-response and scheduled-frame machinery into a protocol-agnostic core (internal/emit) that HTTP and websocket now share.Two capabilities on a streamed resource:
responseslist is written and flushed one chunk at a time, paced by each block'sdelay. Ideal for an OpenAI-style token stream ending in[DONE].scheduleon a streamed resource pushes further responses over the open connection until itslimitis reached or the client disconnects (SSE keepalive, progress, live data).The generalisation (DRY)
New core package
internal/emit:Sink— abstracts "send one processed response body" (a websocket text frame, or an HTTP flushed chunk).EmitResponses— the shared process-then-emit loop (per-block delay/file/content/template).ScheduleHost— runs connection- or request-scoped schedules through the existinginternal/scheduler, emitting via aSink.The websocket plugin is refactored onto these (a
frameSink+ScheduleHost), deleting its bespoke send/schedule code. OpenAPI inherits streaming for free because it already serves responses through the REST pipeline. The top-level engine-lifetimescheduleris unchanged (no client sink; avoids an import cycle).Behaviour & safety
Transfer-Encoding: chunked); status/headers taken from the first chunk. SSE framing is authored in the content, so the same mechanism serves NDJSON or any chunked format.r.Context()).responsesor ascheduleon an HTTP resource requirestream: true;stream: truewith nothing to stream is rejected; schedule-entry checks are shared between websocket and HTTP.Tests & docs
internal/emit(emit loop, buffered fallback, schedule limit + context-cancel), streaming validation tests. Full suite: 30 packages pass;go test -raceclean on the concurrency-sensitive packages.examples/rest/sse-streamingexample anddocs/streaming.md.Verified end-to-end on the native engine: 50 ms SSE pacing to
[DONE]; scheduled push honouringlimitand stopping on client disconnect; and OpenAPI streaming through the shared pipeline.