Skip to content

feat(binding-llm): SSE content decoder for text/event-stream - #2554

Open
jfallows wants to merge 5 commits into
developfrom
claude/issue-2479-2qhkfm
Open

jfallows wants to merge 5 commits into
developfrom
claude/issue-2479-2qhkfm

Conversation

@jfallows

@jfallows jfallows commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Description

Implements LlmContentDecoderSpi for text/event-stream, decoding blank-line-delimited SSE framing (data:/event:/id: fields, comment lines, CRLF/CR/LF line endings) into event DATA + FLUSH frames, per the WHATWG SSE parsing algorithm. Framing decode only, independent of any dialect-specific (OpenAI, Anthropic, etc.) payload interpretation — feeds both same-dialect (framing-only) and cross-dialect (framing + payload transform) routes.

  • LlmSseContentDecoder (io.aklivity.zilla.runtime.binding.llm.internal.decode) — decodes one stream's SSE framing, reporting data() for accumulated data: field bytes (joined with \n across multiple data: lines in one event) and flush(event, id, ...) on each blank-line dispatch. Comment lines (:...) and unrecognized fields (e.g. retry:) are ignored.
  • LlmSseContentDecoderFactorySpi — registers the decoder for content-type text/event-stream, wired via META-INF/services and a provides clause added to module-info.java (the uses declaration already existed from feat(binding-llm): LlmContentDecoderSpi internal SPI, keyed by content-type #2553).
  • LlmSseContentDecoderTest / LlmSseContentDecoderFactorySpiTest — 19 new unit tests covering multi-line data joining, event-name reset semantics, last-event-id persistence across dispatches (and NUL-byte rejection / buffer growth for it), comment/unrecognized-field handling, and the CR/CRLF line-ending edge cases including the ambiguous trailing-CR-at-buffer-edge case (deliberately left unconsumed since it may still turn out to be part of a CRLF pair once more bytes arrive).

Representational-slack note (per the issue's open question): this decoder normalizes field ordering and does not preserve field-value whitespace beyond the single optional space after the colon that the SSE spec itself strips. Re-encoding from the decoded data/flush frames alone cannot byte-exactly reproduce arbitrary source framing — a later round-trip test should assert semantic equality, not assertEquals(bytes).

Verified ./mvnw clean install -pl incubator/binding-llm.conf,incubator/binding-llm passes end-to-end: checkstyle, license headers, and all 27 unit tests pass with full jacoco coverage.

This branch is built on top of claude/issue-2478-76f5ry (#2553, the LlmContentDecoderSpi internal SPI), so this diff includes #2553's commits until that merges to develop — at which point this PR's diff will shrink to just this SSE decoder commit.

Fixes #2479

🤖 Generated with Claude Code

https://claude.ai/code/session_013kK9z4ywuPJe33jm6W1fik


Generated by Claude Code

Scaffold incubator/binding-llm.spec per AGENTS.md conventions and define
LlmBeginEx, LlmDataEx, and the LlmFlushEx union, modelled on
binding-mcp.spec's idl.

LlmBeginEx carries dialect only; model routing is deferred. LlmDataEx has
no fields: content flows through the DATA frame's own payload octets and
INIT/FIN through its existing flags, so nothing survives in the extension
once block identity moves to the FLUSH plane. LlmFlushEx is a 7-case union
covering message start, block start/end, finish, usage, keepalive, and an
opaque native/raw case for re-encoding events a same-dialect route doesn't
recognize.

Fixes #2476

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AuZoMsETwEczJx3cbkb8EJ
Scaffolds incubator/binding-llm and incubator/binding-llm.conf, modelled
on binding-mcp's SERVER/CLIENT BindingContext structure. LlmBindingInfo
is annotated @Incubating so type: llm config loading is gated behind
ZILLA_INCUBATOR_ENABLED via FeatureFilter, matching the AmqpBindingInfo/
PgsqlBindingInfo/RisingwaveBindingInfo precedent.

Fixes #2477

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0142buJWS7C89AKr9uDJtSy4
…t-type

Registers by content-type and hands back a per-stream LlmContentDecoder;
stays in an internal, unexported package for now with no concrete
implementation registered yet.

Fixes #2478

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mw32oxEw24fLt5Ypakj6pH
@jfallows
jfallows force-pushed the claude/issue-2479-2qhkfm branch from 3147f77 to 3ceda3d Compare September 11, 2026 20:05
Closes the non-streaming half of #2478's own scope: "Non-streaming
application/json goes through the same abstraction as one event, rather
than a special-cased branch." Only text/event-stream had an
LlmContentDecoderSpi implementation; application/json requests
(non-streaming dialect responses) had no decoder to dispatch to.

LlmJsonContentDecoder treats the entire buffered document as a single
event (one data + one flush call, no framing loop), mirroring
LlmSseContentDecoder's structure and unit-test conventions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mw32oxEw24fLt5Ypakj6pH
@jfallows
jfallows force-pushed the claude/issue-2479-2qhkfm branch from 3ceda3d to b30e1f4 Compare September 12, 2026 01:36
…odecSpi

LlmContentDecoderSpi and LlmContentEncoderSpi (the latter previously
living downstream in #2571) let a content-type register a decoder with
no matching encoder, or vice versa, since each was ServiceLoader-discovered
and dispatched independently. LlmContentCodecSpi makes "this content-type
is fully supported" one type-enforced fact: a single contentType() key
with both supplyDecoder() and supplyEncoder(), one META-INF/services
registration per content-type, one LlmContentCodecFactory dispatching
both directions.

Pulls the internal/encode/ base package and its text/event-stream
implementation (LlmContentEncoder, LlmSseContentEncoder) forward from
#2571 so the collapse can happen where decode already lives, rather than
forking that package ahead of its own introduction there; #2571 will
need to rebase on top of this and drop its now-duplicate copies.

LlmSseContentDecoder, LlmSseContentEncoder, LlmJsonContentDecoder widen
from package-private to public (unchanged otherwise) since their new
LlmSseContentCodecSpi/LlmJsonContentCodecSpi providers construct them
from the sibling internal.codec package. Adds LlmJsonContentEncoder
(new): the application/json inverse of LlmJsonContentDecoder, copying
content bytes through unchanged with no framing on flush.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mw32oxEw24fLt5Ypakj6pH
@jfallows
jfallows force-pushed the claude/issue-2479-2qhkfm branch from b30e1f4 to 0d82f5d Compare September 14, 2026 22:57

Copy link
Copy Markdown
Contributor Author

Rebased onto the latest claude/issue-2478-76f5ry (#2553). That branch's refactor(binding-llm): collapse decoder/encoder SPIs into LlmContentCodecSpi commit already pulled this PR's LlmSseContentDecoder forward verbatim (widened to public) and re-registered it under the new LlmSseContentCodecSpi, replacing the LlmContentDecoderSpi/LlmSseContentDecoderFactorySpi abstraction this PR introduced.

After resolving the rebase conflicts (keeping the upstream codec-based registration and dropping the now-dead LlmSseContentDecoderFactorySpi + its test, which implemented the deleted LlmContentDecoderSpi), this branch has zero unique commits left relative to #2553 — issue #2479's SSE decoder is fully covered there.

Leaving this open with a zero-diff per instruction; a maintainer may want to close this as superseded by #2553 once that merges.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

binding-llm: SSE content decoder (text/event-stream)

2 participants