feat(binding-llm): event sequence mapper for anthropic and openai dialects by jfallows · Pull Request #2558 · aklivity/zilla

jfallows · 2026-09-09T18:23:21Z

Description

The mapper that translates between a dialect's native event sequence and the canonical vocabulary (from #2557), in both directions:

LlmAnthropicEventMapper — decodes/encodes Anthropic's message_start / content_block_start / content_block_delta / content_block_stop / message_delta / message_stop sequence. Holds input_tokens from message_start until the paired finish + usage emission at message_delta; tracks the currently open block's type to know when content_block_stop needs a canonical blockEnd (tool calls only) and to route content_block_delta payloads (text_delta vs input_json_delta) on encode.
LlmOpenAiEventMapper — decodes/encodes OpenAI's chunk sequence. Translates OpenAI's tool-call-only index space into the canonical (Anthropic-shaped) block index via a per-stream map, and synthesizes blockEnd lazily — deferred until the next tool call starts or the stream finishes, since OpenAI has no explicit block-close event.

Both hold per-stream mutable state (held tokens, block-index map, open-block tracking); a fresh instance is required per stream, matching the existing LlmContentDecoder convention — instances are never shared across streams.

Unit-tested against both worked-example tables from the issue (message role/content/tool-call cardinality changes in each direction), plus the held-usage/already-consumed and lazy-blockEnd edge cases. ./mvnw clean verify -pl incubator/binding-llm passes: checkstyle, license headers, all tests, 100% JaCoCo instruction coverage.

Also wires binding-llm's pom.xml to depend on binding-llm.spec and generate its own flyweight types (mirroring binding-mcp's pattern), with the matching JaCoCo exclusion for generated types.

This branch is built on top of claude/issue-2481-ip27lo (#2557, the canonical event vocabulary), which itself stacks on #2554 and #2553 — so this diff includes those PRs' commits until they merge to develop, at which point this PR's diff will shrink to just this mapper commit.

Fixes #2482

🤖 Generated with Claude Code

https://claude.ai/code/session_019BHBkLjV2tcbNxMEgxkpSw

Generated by Claude Code

Scaffold incubator/binding-llm.spec per AGENTS.md conventions and define LlmBeginEx, LlmDataEx, and the LlmFlushEx union, modelled on binding-mcp.spec's idl. LlmBeginEx carries dialect only; model routing is deferred. LlmDataEx has no fields: content flows through the DATA frame's own payload octets and INIT/FIN through its existing flags, so nothing survives in the extension once block identity moves to the FLUSH plane. LlmFlushEx is a 7-case union covering message start, block start/end, finish, usage, keepalive, and an opaque native/raw case for re-encoding events a same-dialect route doesn't recognize. Fixes #2476 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuZoMsETwEczJx3cbkb8EJ

@Incubating

Scaffolds incubator/binding-llm and incubator/binding-llm.conf, modelled on binding-mcp's SERVER/CLIENT BindingContext structure. LlmBindingInfo is annotated @Incubating so type: llm config loading is gated behind ZILLA_INCUBATOR_ENABLED via FeatureFilter, matching the AmqpBindingInfo/ PgsqlBindingInfo/RisingwaveBindingInfo precedent. Fixes #2477 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0142buJWS7C89AKr9uDJtSy4

…t-type Registers by content-type and hands back a per-stream LlmContentDecoder; stays in an internal, unexported package for now with no concrete implementation registered yet. Fixes #2478 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mw32oxEw24fLt5Ypakj6pH

Closes the non-streaming half of #2478's own scope: "Non-streaming application/json goes through the same abstraction as one event, rather than a special-cased branch." Only text/event-stream had an LlmContentDecoderSpi implementation; application/json requests (non-streaming dialect responses) had no decoder to dispatch to. LlmJsonContentDecoder treats the entire buffered document as a single event (one data + one flush call, no framing loop), mirroring LlmSseContentDecoder's structure and unit-test conventions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mw32oxEw24fLt5Ypakj6pH

…odecSpi LlmContentDecoderSpi and LlmContentEncoderSpi (the latter previously living downstream in #2571) let a content-type register a decoder with no matching encoder, or vice versa, since each was ServiceLoader-discovered and dispatched independently. LlmContentCodecSpi makes "this content-type is fully supported" one type-enforced fact: a single contentType() key with both supplyDecoder() and supplyEncoder(), one META-INF/services registration per content-type, one LlmContentCodecFactory dispatching both directions. Pulls the internal/encode/ base package and its text/event-stream implementation (LlmContentEncoder, LlmSseContentEncoder) forward from #2571 so the collapse can happen where decode already lives, rather than forking that package ahead of its own introduction there; #2571 will need to rebase on top of this and drop its now-duplicate copies. LlmSseContentDecoder, LlmSseContentEncoder, LlmJsonContentDecoder widen from package-private to public (unchanged otherwise) since their new LlmSseContentCodecSpi/LlmJsonContentCodecSpi providers construct them from the sibling internal.codec package. Adds LlmJsonContentEncoder (new): the application/json inverse of LlmJsonContentDecoder, copying content bytes through unchanged with no framing on flush. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mw32oxEw24fLt5Ypakj6pH

Defines the pluggable-dialect contract for binding-llm, exported from the start (unlike LlmContentDecoderSpi, which stays internal): LlmDialect exposes name()/detect()/contentType() plus supplyDecoder(Kind)/ supplyEncoder(Kind) returning common-json JsonTransform stages, and LlmDialectFactorySpi is the ServiceLoader-registered entry point. HttpHeaders is a minimal read-only accessor for detect(path, headers), since no HTTP header abstraction previously existed in this codebase and pulling in jakarta.ws.rs would add a dependency never otherwise used here. Kind is nested on LlmDialect, distinguishing request/response schemas. No concrete dialect implementations yet (OpenAI/Anthropic land later) -- module-info.java exports the dialect package and declares uses without a corresponding provides. Unit-tested via a stub LlmTestDialect/ LlmTestDialectFactorySpi registered under test-scope META-INF/services, mirroring this module's existing LlmContentDecoderSpi/ LlmTestContentDecoderFactorySpi pattern. Fixes #2480 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016NcAVwRPN1Pwjzpobr75w6

…er instance contentType() previously took no parameters, so a dialect could only report one fixed content-type for its lifetime -- insufficient for an API whose response framing (event-stream vs. a single JSON document) depends on a flag in the request body, since neither contentType() nor detect(String, HttpHeaders) offered any way to inspect it. Adds HttpRequestBody, a minimal read-only scalar-member accessor mirroring HttpHeaders, and changes contentType() to contentType(Kind, HttpHeaders, HttpRequestBody): Kind lets request and response resolve independently (a dialect's request body content-type can be fixed while its response varies), and the headers/body context lets that resolution depend on the actual request rather than being fixed at dialect-instance-creation time. Both parameters are nullable for callers without that context available. LlmTestDialect now resolves text/test-event-stream for a streaming response and application/test+json otherwise, exercising the new per-Kind, per-request resolution the stub previously couldn't express. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016NcAVwRPN1Pwjzpobr75w6

…velope, not whole-body buffering Replaces LlmDialect's HttpHeaders/HttpRequestBody/contentType() with the engine's existing ModelEnvelope/ModelTransform (runtime/engine/.../model/), reusing machinery this codebase already has instead of inventing LLM-specific buffering to resolve text/event-stream vs. application/json, or a model name, by peeking one field of an otherwise-unbuffered body. detect(ModelEnvelope) folds :path/:method into the same envelope as ordinary headers -- no separate path parameter. supplyDecoder/supplyEncoder now take (Kind, ModelEnvelope) and return ModelTransform: a per-field stage that can extract a field (e.g. a model name) into the envelope while the body still flows through unchanged, mirroring KafkaExtractTransform (runtime/binding-kafka/.../cache/) -- so a caller reads that signal back off the envelope as decoding proceeds rather than buffering the whole body first to inspect it. contentType() is removed entirely: nothing in this shape needs it once the streaming/non-streaming signal is just another envelope entry a caller reads after extraction. common-json/JsonTransform is no longer used anywhere in this module now that the dialect SPI itself doesn't need it, so the dependency comes back out of module-info.java and pom.xml along with it. LlmTestDialect/LlmTestDialectFactorySpi become LlmTestConditionalDialect/ LlmTestConditionalDialectFactorySpi, since what they now demonstrate is exactly this: request detection and model-name extraction conditional on envelope contents, not a fixed per-instance answer. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016NcAVwRPN1Pwjzpobr75w6

…ken detail Extend LlmFlushEx's block-lifecycle skeleton (from #2476) with the two places OpenAI's parallel completions and per-token detail need to survive in the vocabulary, per the issue's scope: - choiceIndex (default 0) on messageStart, blockStart, blockEnd, finish, and native/raw: the (choice, block) compound index's outer half. Anthropic is always choice 0, so every existing dialect mapping is unaffected; OpenAI's n > 1 becomes one messageStart per parallel completion, distinguished by choiceIndex. usage stays choiceIndex-free since every dialect reports it aggregated across choices, never per choice. - logProbability (nullable) on LlmDataEx: the one per-delta detail the vocabulary carries directly, for dialects exposing per-token detail (OpenAI logprobs) without reopening the DATA/FLUSH split from #2476 or growing the vocabulary for the full log-probability structure — richer detail than one value per token stays behind LlmNativeFlushEx. Documents the three lossiness cases as doc comments alongside the fields they concern: choiceIndex and logProbability both drop out on any cross-dialect route to Anthropic (structurally exactly one choice, no per-token detail); message-start input token counts are resolved by decoupling inputTokens into its own deferred usage event rather than emitting a placeholder on messageStart and correcting it later, so a source that discloses tokens late (OpenAI) just emits usage late instead of needing a correction event. Fixes #2481 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AYGZDrytLjmG2AQqUN1hsu

…lects Translates between each dialect's native streaming event sequence and the canonical vocabulary from #2557, in both directions: - LlmAnthropicEventMapper: holds input_tokens from message_start until the paired usage event at message_delta; tracks the currently open block's type to know when content_block_stop needs a canonical blockEnd (tool calls only) and to route content_block_delta payloads (text_delta vs input_json_delta) on encode. - LlmOpenAiEventMapper: translates OpenAI's tool-call-only index space into the canonical (Anthropic-shaped) block index via a per-stream map, and synthesizes blockEnd lazily -- deferred until the next tool call starts or the stream finishes, since OpenAI has no explicit block-close event. Unit-tested against both worked-example tables from the issue (message role/content/tool-call cardinality changes in each direction), plus the held-usage/already-consumed and lazy-blockEnd edge cases. Fixes #2482 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019BHBkLjV2tcbNxMEgxkpSw

jfallows mentioned this pull request Sep 11, 2026

feat(binding-llm): dialect detection dispatcher and fixed-dialect config #2568

Open

claude added 3 commits September 11, 2026 19:43

jfallows force-pushed the claude/issue-2482-tpoium branch from 5a59233 to b4e4b44 Compare September 11, 2026 20:34

jfallows force-pushed the claude/issue-2482-tpoium branch 2 times, most recently from e4fe7d4 to 0ee8b63 Compare September 12, 2026 04:23

claude added 6 commits September 14, 2026 22:49

jfallows force-pushed the claude/issue-2482-tpoium branch from 0ee8b63 to ff41aa1 Compare September 14, 2026 23:36

jfallows mentioned this pull request Sep 18, 2026

binding-llm: replace LlmFlushEx union + event mapper with uniform DATA + LlmDataEx, driven directly by common-json #2596

Open

Navigation Menu

feat(binding-llm): event sequence mapper for anthropic and openai dialects - #2558

feat(binding-llm): event sequence mapper for anthropic and openai dialects#2558
jfallows wants to merge 10 commits into
developfrom
claude/issue-2482-tpoium

jfallows commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jfallows commented Sep 9, 2026

Description

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants