Conversation
Scaffold incubator/binding-llm.spec per AGENTS.md conventions and define LlmBeginEx, LlmDataEx, and the LlmFlushEx union, modelled on binding-mcp.spec's idl. LlmBeginEx carries dialect only; model routing is deferred. LlmDataEx has no fields: content flows through the DATA frame's own payload octets and INIT/FIN through its existing flags, so nothing survives in the extension once block identity moves to the FLUSH plane. LlmFlushEx is a 7-case union covering message start, block start/end, finish, usage, keepalive, and an opaque native/raw case for re-encoding events a same-dialect route doesn't recognize. Fixes #2476 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuZoMsETwEczJx3cbkb8EJ
Scaffolds incubator/binding-llm and incubator/binding-llm.conf, modelled on binding-mcp's SERVER/CLIENT BindingContext structure. LlmBindingInfo is annotated @Incubating so type: llm config loading is gated behind ZILLA_INCUBATOR_ENABLED via FeatureFilter, matching the AmqpBindingInfo/ PgsqlBindingInfo/RisingwaveBindingInfo precedent. Fixes #2477 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0142buJWS7C89AKr9uDJtSy4
…t-type Registers by content-type and hands back a per-stream LlmContentDecoder; stays in an internal, unexported package for now with no concrete implementation registered yet. Fixes #2478 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mw32oxEw24fLt5Ypakj6pH
Closes the non-streaming half of #2478's own scope: "Non-streaming application/json goes through the same abstraction as one event, rather than a special-cased branch." Only text/event-stream had an LlmContentDecoderSpi implementation; application/json requests (non-streaming dialect responses) had no decoder to dispatch to. LlmJsonContentDecoder treats the entire buffered document as a single event (one data + one flush call, no framing loop), mirroring LlmSseContentDecoder's structure and unit-test conventions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mw32oxEw24fLt5Ypakj6pH
…odecSpi LlmContentDecoderSpi and LlmContentEncoderSpi (the latter previously living downstream in #2571) let a content-type register a decoder with no matching encoder, or vice versa, since each was ServiceLoader-discovered and dispatched independently. LlmContentCodecSpi makes "this content-type is fully supported" one type-enforced fact: a single contentType() key with both supplyDecoder() and supplyEncoder(), one META-INF/services registration per content-type, one LlmContentCodecFactory dispatching both directions. Pulls the internal/encode/ base package and its text/event-stream implementation (LlmContentEncoder, LlmSseContentEncoder) forward from #2571 so the collapse can happen where decode already lives, rather than forking that package ahead of its own introduction there; #2571 will need to rebase on top of this and drop its now-duplicate copies. LlmSseContentDecoder, LlmSseContentEncoder, LlmJsonContentDecoder widen from package-private to public (unchanged otherwise) since their new LlmSseContentCodecSpi/LlmJsonContentCodecSpi providers construct them from the sibling internal.codec package. Adds LlmJsonContentEncoder (new): the application/json inverse of LlmJsonContentDecoder, copying content bytes through unchanged with no framing on flush. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mw32oxEw24fLt5Ypakj6pH
Defines the pluggable-dialect contract for binding-llm, exported from the start (unlike LlmContentDecoderSpi, which stays internal): LlmDialect exposes name()/detect()/contentType() plus supplyDecoder(Kind)/ supplyEncoder(Kind) returning common-json JsonTransform stages, and LlmDialectFactorySpi is the ServiceLoader-registered entry point. HttpHeaders is a minimal read-only accessor for detect(path, headers), since no HTTP header abstraction previously existed in this codebase and pulling in jakarta.ws.rs would add a dependency never otherwise used here. Kind is nested on LlmDialect, distinguishing request/response schemas. No concrete dialect implementations yet (OpenAI/Anthropic land later) -- module-info.java exports the dialect package and declares uses without a corresponding provides. Unit-tested via a stub LlmTestDialect/ LlmTestDialectFactorySpi registered under test-scope META-INF/services, mirroring this module's existing LlmContentDecoderSpi/ LlmTestContentDecoderFactorySpi pattern. Fixes #2480 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016NcAVwRPN1Pwjzpobr75w6
…er instance contentType() previously took no parameters, so a dialect could only report one fixed content-type for its lifetime -- insufficient for an API whose response framing (event-stream vs. a single JSON document) depends on a flag in the request body, since neither contentType() nor detect(String, HttpHeaders) offered any way to inspect it. Adds HttpRequestBody, a minimal read-only scalar-member accessor mirroring HttpHeaders, and changes contentType() to contentType(Kind, HttpHeaders, HttpRequestBody): Kind lets request and response resolve independently (a dialect's request body content-type can be fixed while its response varies), and the headers/body context lets that resolution depend on the actual request rather than being fixed at dialect-instance-creation time. Both parameters are nullable for callers without that context available. LlmTestDialect now resolves text/test-event-stream for a streaming response and application/test+json otherwise, exercising the new per-Kind, per-request resolution the stub previously couldn't express. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016NcAVwRPN1Pwjzpobr75w6
…velope, not whole-body buffering Replaces LlmDialect's HttpHeaders/HttpRequestBody/contentType() with the engine's existing ModelEnvelope/ModelTransform (runtime/engine/.../model/), reusing machinery this codebase already has instead of inventing LLM-specific buffering to resolve text/event-stream vs. application/json, or a model name, by peeking one field of an otherwise-unbuffered body. detect(ModelEnvelope) folds :path/:method into the same envelope as ordinary headers -- no separate path parameter. supplyDecoder/supplyEncoder now take (Kind, ModelEnvelope) and return ModelTransform: a per-field stage that can extract a field (e.g. a model name) into the envelope while the body still flows through unchanged, mirroring KafkaExtractTransform (runtime/binding-kafka/.../cache/) -- so a caller reads that signal back off the envelope as decoding proceeds rather than buffering the whole body first to inspect it. contentType() is removed entirely: nothing in this shape needs it once the streaming/non-streaming signal is just another envelope entry a caller reads after extraction. common-json/JsonTransform is no longer used anywhere in this module now that the dialect SPI itself doesn't need it, so the dependency comes back out of module-info.java and pom.xml along with it. LlmTestDialect/LlmTestDialectFactorySpi become LlmTestConditionalDialect/ LlmTestConditionalDialectFactorySpi, since what they now demonstrate is exactly this: request detection and model-name extraction conditional on envelope contents, not a fixed per-instance answer. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016NcAVwRPN1Pwjzpobr75w6
…ken detail Extend LlmFlushEx's block-lifecycle skeleton (from #2476) with the two places OpenAI's parallel completions and per-token detail need to survive in the vocabulary, per the issue's scope: - choiceIndex (default 0) on messageStart, blockStart, blockEnd, finish, and native/raw: the (choice, block) compound index's outer half. Anthropic is always choice 0, so every existing dialect mapping is unaffected; OpenAI's n > 1 becomes one messageStart per parallel completion, distinguished by choiceIndex. usage stays choiceIndex-free since every dialect reports it aggregated across choices, never per choice. - logProbability (nullable) on LlmDataEx: the one per-delta detail the vocabulary carries directly, for dialects exposing per-token detail (OpenAI logprobs) without reopening the DATA/FLUSH split from #2476 or growing the vocabulary for the full log-probability structure — richer detail than one value per token stays behind LlmNativeFlushEx. Documents the three lossiness cases as doc comments alongside the fields they concern: choiceIndex and logProbability both drop out on any cross-dialect route to Anthropic (structurally exactly one choice, no per-token detail); message-start input token counts are resolved by decoupling inputTokens into its own deferred usage event rather than emitting a placeholder on messageStart and correcting it later, so a source that discloses tokens late (OpenAI) just emits usage late instead of needing a correction event. Fixes #2481 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AYGZDrytLjmG2AQqUN1hsu
…lects Translates between each dialect's native streaming event sequence and the canonical vocabulary from #2557, in both directions: - LlmAnthropicEventMapper: holds input_tokens from message_start until the paired usage event at message_delta; tracks the currently open block's type to know when content_block_stop needs a canonical blockEnd (tool calls only) and to route content_block_delta payloads (text_delta vs input_json_delta) on encode. - LlmOpenAiEventMapper: translates OpenAI's tool-call-only index space into the canonical (Anthropic-shaped) block index via a per-stream map, and synthesizes blockEnd lazily -- deferred until the next tool call starts or the stream finishes, since OpenAI has no explicit block-close event. Unit-tested against both worked-example tables from the issue (message role/content/tool-call cardinality changes in each direction), plus the held-usage/already-consumed and lazy-blockEnd edge cases. Fixes #2482 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019BHBkLjV2tcbNxMEgxkpSw
LlmDialectResolver dispatches path/header detection across every LlmDialect registered via LlmDialectFactorySpi, without hardcoding any dialect's signals. A configured fixed dialect name bypasses detection entirely, including when it matches no registered dialect. When detection matches more than one dialect, or none, resolution is ambiguous and returns null so the caller rejects the request rather than guessing. LlmOptionsConfig adds the optional server-kind `dialect` option (schema, config, and adapter) used to pin a fixed dialect. Fixes #2483 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AtkteS7C66qiVLGeEDJ2FP
The sys: namespace's patch contract (Binding.system()/Exporter.system())
lets a component contribute a shared binding (e.g. http_client) but has
no pre-seeded slot for a shared catalog, so a patch adding one has
nothing to append into. Pre-seed catalogs: {} alongside the existing
bindings: {} in the base sys namespace skeleton, the same "pre-seed the
extension point" convention already used for the JSON-schema *-ext
scaffolds, so any component can share a catalog-backed resource the way
bindings are already shared.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015mHT1vrPATQGSdRcB8NNKM
…ect schemas Adds LlmDialectFactorySpi.schema(Kind), letting a dialect contribute a URL to its own JSON schema for the request or response direction without knowing anything about catalogs or patches. LlmBinding.system() enumerates every registered dialect, reads each contributed schema, and generates a sys: namespace patch adding one shared inline catalog (llm_dialects) with a <dialect>.request/<dialect>.response subject per schema a dialect contributes -- built once, at engine startup, since the dialect set is ServiceLoader-discovered off the classpath and therefore fixed for the JVM's life, the same way sys: already shares a binding (e.g. http_client) across every binding that references it. LlmDataUrlStreamHandler decodes a base64 data: URL (RFC 2397) in memory, scoped to a single URL via the URL.of(URI, URLStreamHandler) factory -- no temporary file, no globally-registered protocol handler -- used to hand the generated patch to Binding.system()'s URL-returning contract without writing it to disk. LlmBeginEx gains contentType and model fields alongside dialect, for a server-kind stream factory to stamp once it has resolved the dialect and read the request's content-type. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015mHT1vrPATQGSdRcB8NNKM
… LlmBeginEx.dialect Implements request/response framing decode-and-forward for the LLM server binding (#2484): LlmServerFactory drives the request body through the resolved dialect's ModelPipeline as bytes arrive off the wire, forwards transformed content to app0 incrementally rather than buffering the whole body, and threads the resolved dialect name onto LlmBeginEx so app0 can see which wire dialect produced the request. Flow control between the client, this binding, and app0 is enforced with dedicated decodeSlot/encodeSlot buffers on each side of the exchange (LlmServer for the network-facing leg, LlmStream for the app-facing leg), each granting credit strictly from its own local slot occupancy rather than copying a peer's sequence numbers across independent byte domains. LlmState tracks per-direction open/closing/closed transitions and end-of-stream deferral while a buffer is still draining. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015mHT1vrPATQGSdRcB8NNKM
…fferPool handles DefaultBufferPool.buffer(slot) rewraps and returns a single shared mutable field per pool instance, so holding a buffer reference across a nested call that fetches a different slot from the same pool silently repoints it. LlmServer.decodeNetwork() directly, synchronously calls LlmStream's request-relay staging method as a plain nested Java method call, so a single pool instance backing both the network-decode slot and the app0 request-relay slot would alias between them. Give LlmServerFactory two BufferPool handles instead of one: decodePool (network decode) and encodePool (app0 relay, both directions), obtained via context.bufferPool() and .duplicate() -- matching McpServerFactory's decodePool/encodePool precedent. The two relay directions sharing encodePool (the reply-direction slot on LlmServer and the request-direction slot on LlmStream) never appear in the same call stack: cross-binding accept() is ring-buffer-mediated and dispatched on a later engine tick, not a nested call, so one encodePool instance can't alias between them. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015mHT1vrPATQGSdRcB8NNKM
…nline.conf LlmSystemNamespaceGenerator emits a real "type": "inline" catalog config into the generated sys: namespace patch, serviced at actual runtime by catalog-inline's CatalogFactorySpi via ServiceLoader -- not just exercised by this module's own tests. test scope kept it off the runtime classpath entirely; provided scope wouldn't fit either, since nothing in main source compiles against catalog-inline's Java API (the reference is a plain string), so there's nothing to satisfy at compile time. Match binding-asyncapi/binding-openapi's precedent for this same generated-"type: inline"-config pattern: catalog-inline at runtime scope, plus the companion catalog-inline.conf (config-schema side) at default scope alongside the existing model-json.conf dependency. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015mHT1vrPATQGSdRcB8NNKM
LlmBindingConfig.newModelConfig() constructs a real JsonModelConfig in main source, resolved to a working ModelHandler by context.supplyModel() via a ModelFactorySpi lookup at actual runtime -- not just exercised by this module's own tests. Matches binding-asyncapi/binding-mcp-openapi/ binding-openapi, each of which also builds JsonModelConfig directly in main source and declares model-json at runtime scope. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015mHT1vrPATQGSdRcB8NNKM
Stale relative to the catalog-inline/model-json scope fixes on binding-llm (catalog-inline.conf and model-json.conf now roll up into this aggregate), plus a pre-existing gap for binding-http.spec's license entry. Regenerated via ./mvnw notice:generate -pl incubator -amd, never hand-edited, per AGENTS.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015mHT1vrPATQGSdRcB8NNKM
Add LlmServerConfig/LlmServerConfigBuilder (host/port, nested builder) following the existing *OptionsConfigAdapter pattern used by binding-kafka's options.servers, wired into LlmOptionsConfig via a new `server` field so `llm client` bindings can configure their upstream endpoint. Config adapter unit tests cover parsing and serializing options.server. Fixes #2485 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FQ4XvYNCp1NzQC4Ne4eWbK
Exercise the inject() path on LlmServerConfigBuilder so the nested server builder reaches the module's required 100% instruction coverage, mirroring the existing shouldInjectBuilder test for LlmOptionsConfigBuilder. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FQ4XvYNCp1NzQC4Ne4eWbK
…→encoder chaining Adds LlmClientFactory (kind: client), the first real integration point for LlmDialect.supplyDecoder/supplyEncoder: it compares the inbound app-declared dialect (LlmBeginEx.dialect) against the binding's own configured dialect and only chains a JsonPipeline payload transform when they differ, forwarding framing-only re-encoded content when they match. - internal/encode: LlmContentEncoder/Spi/Factory + LlmSseContentEncoder, the SSE-framing encode counterpart to internal/decode's existing SSE decoder - LlmDialectResolver.dialectNamed(String): by-name lookup for resolving the inbound dialect when it differs from the client's configured one - Schema patch: kind: client options (dialect, server); also fixes kind: server to accept options.server (previously unreachable under its additionalProperties: false, despite LlmOptionsConfig already supporting it since PR 2570) - Spec scripts (same.dialect, cross.dialect, client.opaque.fallback, client.abort) plus a new LlmClientIT, written first per this repo's test-first discipline, confirmed failing before LlmClientFactory existed Fixes #2486 Real dialect implementations (openai, anthropic), the mock backend, and the full round-trip identity test are separate, sibling issues (#2487, #2490, Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qssroe9MdjvpvQrthi15ex
…aders, body) Rebased onto #2570's latest, which changed LlmDialect.contentType() to take (Kind, headers, body) parameters, resolved per request rather than fixed once per dialect instance. The client resolves content-type separately for each direction now (REQUEST for its outbound encoder, RESPONSE for its inbound decoder) rather than a single shared call, matching the interface's own point: request and response can have different native content-types. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qssroe9MdjvpvQrthi15ex
LlmClientFactory.newStream resolved the RESPONSE content decoder eagerly at BEGIN time with body=null, so LlmDialect.contentType(Kind.RESPONSE, ...) could never actually see request-body content when deciding between a streaming and non-streaming response, defeating the per-request capability added for that method. REQUEST-side encoder resolution is unaffected since it does not depend on body content. Accumulate the request body (post cross-dialect transform, pre-framing) into a buffer-pool-backed slot as DATA arrives, and resolve the decoder at onAppEnd, once the full request is available, relying on this binding's half-duplex transmission convention to guarantee no response bytes arrive before then. Add LlmJsonRequestBody, a pure-Java HttpRequestBody view driven by common-json's one-shot parser API, plus a unit test. Add a new test-conditional dialect and two client IT/spec scenarios proving the decoder now differs based on a stream field in the request body. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qssroe9MdjvpvQrthi15ex
…line SPI LlmDialect no longer exposes contentType(Kind, HttpHeaders, HttpRequestBody); dialects now detect via ModelEnvelope and supply ModelTransform-based decoders/encoders driven through the engine's ModelHandler/ModelPipeline SPI. Content-type is read directly off real wire headers instead of being computed per-dialect, so the request-body buffering added to defer response-decoder selection (LlmJsonRequestBody) is no longer needed and is removed. LlmClientFactory is rebuilt against the new SPI: dialect resolution via LlmBindingConfig.resolveDialect/dialectNamed, a per-stream ModelEnvelope, and a ModelPipeline (ModelTransform.NONE for same-dialect) run unconditionally in both directions so downstream code always sees validated, well-formed payloads. Test dialects are renamed (test-client, test-client-sse, test-client-sse-alt) to avoid colliding with the real upstream test fixtures, and gain request/response JSON schemas so the pipeline can actually validate their payloads instead of rejecting everything for lack of a schema. Adds the missing llm:flushEx()/llm:matchFlushEx() k3po functions that the client k3po scripts already relied on. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qssroe9MdjvpvQrthi15ex
…sponse transforms Implements LlmDialect for OpenAI Chat Completions, registered via LlmDialectFactorySpi: detect() matches POST /v1/chat/completions and POST /v1/completions; contentType() is text/event-stream, the only LlmContentDecoderSpi/LlmContentEncoderSpi framing codec this module has today (a non-streaming application/json pair is a content-decoder- layer gap, not a dialect one, and is left as follow-up). supplyDecoder/supplyEncoder rename OpenAI-native request/response JSON members to the canonical vocabulary this dialect defines a synonym for -- max_tokens/maxOutputTokens, top_p/topP, n/choiceCount and friends on requests; index/choiceIndex, finish_reason/finishReason (remapping tool_calls/tool_call), logprobs/logProbability, and the usage token counts on responses -- via a depth-tracked JsonTransform that renames JSON events without DOM parsing. Everything without an established canonical synonym (id, model, messages, tools, the whole delta/tool_calls structure including streamed function.arguments fragments) forwards unchanged, at any depth, so decode -> encode round-trips with no loss. Tests cover dialect detection/registration and both Kinds of transform, including full round-trips through the canonical form. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SMybNYCLJEWTwCjDscepZm
… rename model-json's ModelTransform integration was observation-only (ModelFieldBridge), discarding REPLACED/DECLINED answers instead of writing them. JsonModelFieldTransform drives a wired ModelTransform inline as JSON streams through, computing a JSON-pointer path per scalar field (with array-index segments) at any nesting depth, and writes FIELD/REPLACED/DECLINED answers straight to the destination -- including a REPLACED substitute redirecting a field to a sibling key of the same enclosing object. Adds the missing key-write for container-valued members entering a named object member, fixes a resumed key/value write re-offering the whole text instead of the remainder (TextSource now tracks its own consumed() progress), and reuses the already-decoded scalar text/key for an unchanged FIELD answer instead of a fresh per-field allocation. Also fixes JsonModelHandlerImpl.supplyEncoder silently dropping its transform parameter. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SMybNYCLJEWTwCjDscepZm
…e SPI Rebases LlmOpenAiDialect and its request/response transforms onto the redesigned LlmDialect SPI: detect(ModelEnvelope), no more contentType() (content-type is now resolved from real upstream Content-Type headers), supplyDecoder/supplyEncoder(Kind, ModelEnvelope) returning a ModelTransform. The request/response transforms are rewritten as plain ModelTransform implementations matching each field's own full path (e.g. $.choices[0].index, $.usage.prompt_tokens) rather than tracking JSON structural depth, since the model-json adapter now computes paths itself. This drops the old JsonEvent-token depth-tracking machinery and the JsonSource/JsonController wrappers (LlmOpenAiStructuredController is no longer needed); the renamed LlmOpenAiSubstitutedSource is now a plain ModelSource. The choices[]/usage direct-member path checks defer their substring() until after confirming the path is actually a direct member, so the many deeply nested per-chunk fields that share the prefix (delta.tool_calls[].index and the like) cost no allocation. The one dropped rename from the prior implementation is logprobs/logProbability: it names a container-valued field, and this dialect only renames scalar leaves. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SMybNYCLJEWTwCjDscepZm
…openai dialect
LlmOpenAiRequestTransform renamed a fixed set of top-level fields but never
observed model, so LlmServerFactory.doAppBegin's server.envelope.get("model", 0)
read right after running the request through this transform came back empty and
LlmBeginEx.model was never stamped for real dialect: openai traffic.
Mirrors LlmTestPermissiveDialect's inline ModelExtractTransform (and
KafkaExtractTransform's own pattern): on a FIELD event at $.model, copy the
value into the envelope alongside the existing rename-or-forward decision,
without touching the RENAMES table. LlmOpenAiResponseTransform needs no
equivalent change -- nothing reads model back off a response.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SMybNYCLJEWTwCjDscepZm
…requests CI's LlmServerIT.shouldRejectRequestWithUnresolvedDialect failed: detect() matched on :method/:path alone, so it ambiguously co-matched every existing k3po server fixture -- all of which hit the same /v1/chat/completions path with a test-specific content-type (application/vnd.zilla.test-permissive+json, application/vnd.zilla.test-strict+json) to select a *different* dialect unambiguously. Only one failure surfaced in CI because failsafe stops after the first failure, but the same ambiguity affected every other fixture in that class too -- confirmed by LlmServerIT going from 6 run/1 failed/1 skipped to 8/8 passing with this fix, and LlmClientIT unaffected at 4/4. detect() now also requires content-type: application/json, the only content-type a genuine OpenAI request ever carries, so a request to the same path with a different dialect's own content-type no longer collides. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SMybNYCLJEWTwCjDscepZm
The llm.schema.patch.json allowed options.server on kind:server bindings, but LlmServerFactory never reads binding.options.server — only LlmClientFactory dials it as the upstream endpoint. Restrict the kind:server options schema to dialect (fixed-dialect mode), matching actual runtime usage and the milestone's example configs. Add LlmSchemaValidationTest exercising the full EngineConfigReader pipeline against real zilla.yaml text for both kind:server and kind:client, covering acceptance (bare server, fixed-dialect server, full client) and rejection (server option on kind:server, missing required client fields, malformed server pattern, unknown kind). Fixes #2488 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Aawd4ZFaUE3FgQnzkABj9F
…ecode LlmAnthropicResponseTransform now checks a decoded event's `$.type` field against the SSE `event:` name that framed it, rejecting when they disagree -- a well-behaved backend never sends one, so this only fires against a malformed or malicious upstream. The event name is threaded from LlmSseContentDecoder (which already tracked it) through a new LlmContentDecoderOutput.event(String) hook into the per-stream ModelEnvelope under "event", mirroring how `model` is already captured from the request. Encoding is unaffected: this dialect authors both the outgoing event: line and its type field from the same LlmFlushExFW kind, so they cannot disagree the way untrusted inbound bytes can.
…smatch Replace the hand-rolled unit tests for the anthropic response type/event mismatch check with real k3po ITs (LlmClientIT + spec-level NetworkIT/ ApplicationIT self-consistency checks), driving the actual LlmAnthropicResponseTransform decode path end to end through a live engine, per this repo's test-first discipline: no unit tests for this behavior, only ITs. Also give the anthropic dialect a minimal response JSON schema (LlmAnthropicDialectFactorySpi now returns one for Kind.RESPONSE). Without it, JsonModelDecoderPipeline can never resolve a schema id for the "anthropic.response" catalog subject (never registered, since the dialect previously contributed none), so every anthropic response -- regardless of content -- was unconditionally rejected before reaching any dialect-specific transform. This surfaced only once a real end-to-end IT exercised the response path for the first time; the prior unit tests bypassed the model pipeline entirely and could not have caught it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Lm8DhhWCAHAAUMmLeNYnx
… anthropic request LlmAnthropicDialectFactorySpi.schema(Kind.REQUEST) returned null, so LlmSystemNamespaceGenerator never registered an "anthropic.request" subject in the shared sys:llm_dialects catalog. LlmBindingConfig's model config unconditionally references dialect.name() + ".request" for every dialect regardless of whether a schema was actually registered for it, so every anthropic request -- both kind:server auto-detect and kind:client fixed-dialect -- silently hung: the request never reached the app-side stream. Add anthropic.request.schema.json (Anthropic Messages API request shape, matching openai.request.schema.json's style and required fields: model + messages only) and wire it into LlmAnthropicDialectFactorySpi the same way OpenAI's factory SPI already does for both Kind values. Confirmed with a genuine red/green cycle: LlmServerIT#shouldDetectAnthropicDialectFromPath times out identically to the reported symptom against the pre-fix code, and passes once the schema is registered. Added the matching NetworkIT/ApplicationIT self-consistency methods for the same anthropic.request fixture pair. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Lm8DhhWCAHAAUMmLeNYnx
…rver requests LlmServerFactory canonicalized every inbound request unconditionally via dialect.supplyDecoder(Kind.REQUEST, envelope) -- the wrong shape for kind: server, which has no target dialect to canonicalize toward. It only needs to detect the dialect, extract model for routing, and validate the body against that dialect's own native schema, then forward it byte-for-byte to app0. Canonical rewriting is meaningful only when bridging between two different dialects, which is kind: client's job alone -- it already gets this right via its sameDialect check and ModelTransform.NONE substitution. Add LlmDialect.supplyValidator(Kind, ModelEnvelope): validates against the same native schema supplyDecoder's model already enforces, still extracts model as a side effect, but performs no field renaming. Backed by a new shared LlmModelExtractTransform (model sits at the same top-level path in both dialects, so one dialect-neutral instance serves both LlmOpenaiDialect and LlmAnthropicDialect rather than duplicating the extraction logic per dialect). LlmServerFactory now calls supplyValidator instead of supplyDecoder; LlmClientFactory is unchanged, since it still needs real supplyDecoder/supplyEncoder composition for cross-dialect bridging. Every LlmDialect implementor needed a supplyValidator override, including the six test-only dialects in the dialect package -- each delegates to its own supplyDecoder, since none of those already do canonical renaming (supplyValidator and supplyDecoder are identical for them). Confirmed with a genuine red/green cycle: reintroduced max_tokens into the anthropic.request fixture pair and reverted the LlmServerFactory call to supplyDecoder -- shouldDetectAnthropicDialectFromPath failed with the field renamed to maxOutputTokens en route to app0, exactly the reported symptom. Restoring supplyValidator forwards it unchanged. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Lm8DhhWCAHAAUMmLeNYnx
…ipts Adds anthropic.request/streaming/nonstreaming client/server .rpt pairs on both the network side (real Anthropic Messages API wire shape: named SSE events message_start/content_block_start/content_block_delta/ content_block_stop/message_delta/message_stop, multi-block content, input_json_delta tool streaming, and the top-level non-streaming response object with usage) and the application side (llm dialect "anthropic"), mirroring the existing mock OpenAI backend fixture structure. Wires the new scripts into NetworkIT/ApplicationIT (spec self-consistency) and LlmClientIT/LlmServerIT (engine-driven, including path-based dialect detection for POST /v1/messages), matching the openai.* test methods already present. Adds client.anthropic.yaml alongside client.openai.yaml. Fixes #2493 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HXHtTkxAUq2EYMqxY8jgY8
…c mock
network/anthropic.streaming/server.rpt and network/anthropic.nonstreaming/
server.rpt were doing double duty, mirroring the same pre-existing conflict
just fixed for openai (see the sibling "give LlmClientIT its own
network-side openai mock upstream" commit): NetworkIT pairs them with the
sibling client.rpt to prove that pair is self-consistent using the real
anthropic dialect-detection path ("/v1/messages"), while LlmClientIT
separately reused the same server.rpt as the mock upstream llm:client
(fixed dialect: anthropic) connects out to -- but LlmClientFactory encodes
a fixed "/" path regardless of dialect, so the mock upstream there must
expect that literal path, not the detection path.
Split the two roles apart, following this suite's own "client.*" folder
convention: LlmClientIT now targets new network/client.anthropic.streaming
and network/client.anthropic.nonstreaming server-only fixtures (":path":
"/", matching the real encoder), leaving network/anthropic.streaming and
network/anthropic.nonstreaming's client/server pair exclusively for
NetworkIT's self-consistency check and LlmServerIT's dialect detection.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXHtTkxAUq2EYMqxY8jgY8
…r rebase Rebasing onto the upstream branch's own anthropic.request fixture (added alongside the request-schema registration fix) merged cleanly at the file level but left each of NetworkIT, ApplicationIT and LlmServerIT with the same test method declared twice -- one copy from each side of the rebase, both now pointing at the same (upstream's) fixture. Drop the redundant copy from each class. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HXHtTkxAUq2EYMqxY8jgY8
…code LlmSseContentEncoder wrote an event's data: line as soon as encodeData() was called and only wrote event: once the paired FLUSH arrived, so every emitted SSE event came out as data:...\nevent:...\n\n -- backwards from what every real SSE sender (and LlmSseContentDecoder itself) expects. Split the event-name announcement out of encodeFlush into its own encodeEventName primitive, and defer content bytes: LlmServerFactory's LlmStream (reply re-encode) and LlmClientFactory's LlmClient (request re-encode) now accumulate DATA payload into a per-stream pending buffer and only emit encodeEventName -> encodeData(pending) -> encodeFlush(id + terminator) as one ordered write when the paired FLUSH arrives. Content types with no event concept (LlmJsonContentEncoder) get a no-op encodeEventName, matching their existing no-op encodeFlush. A request whose dialect never advises an app-level FLUSH before END (e.g. a plain non-streaming JSON body) would otherwise leave its pending bytes stranded -- LlmClientFactory.onAppEnd now flushes any such leftover content before forwarding END. The mirror case doesn't occur on the reply side: every reply fixture already advises a FLUSH before closing. Updates the cross.dialect and same.dialect network fixtures, which had encoded the old, backwards field order as "correct". Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Lm8DhhWCAHAAUMmLeNYnx
…ming fixtures The anthropic.nonstreaming request/response fixtures previously carried a single text content block, unlike anthropic.streaming which already covers a text block followed by a tool_use block. Add the same text+tool_use combination to the nonstreaming response body so the round-trip identity coverage for the anthropic dialect (issue #2494) exercises multi-block content in both the streaming and non-streaming shapes, not just streaming. Covers NetworkIT/ApplicationIT (peer-to-peer script self-consistency) and LlmServerIT/LlmClientIT (engine-backed decode/re-encode fidelity), all of which already reference these fixtures. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0118QLdfK9TEA6EtYUXRjMQ3
…orms A dialect's request/response ModelTransform only inspected ModelEvent.FIELD, so a field a prior transform in the chain had already renamed (delivered as REPLACED) skipped this transform's own rename table entirely -- breaking any field two dialects both define a canonical synonym for (e.g. max_tokens) when chained via supplyDecoder(...).andThen(supplyEncoder(...)) across dialects. Widen the check to treat REPLACED the same as FIELD, matching ModelEvent's own contract that a REPLACED source is a full field view a downstream transform can still rename. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NRZevD32tHJi45aRSueikf
LlmAnthropicEventMapper.encode(DirectBuffer, ...) hardcoded index:0 on every encoded content_block_delta regardless of which block was actually open, silently corrupting Anthropic's block-position signal for any block after the first. Same-dialect traffic never exercises this encode path (it forwards native bytes unchanged), so the defect had no coverage; track the open block's own id, set from LlmBlockStartFlushEx, and report it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NRZevD32tHJi45aRSueikf
…event mapper llm:client bridging two different dialects previously translated a response solely through each dialect's field-rename ModelTransform, which forwards exactly one native document per native document in -- structurally unable to reshape event cardinality (e.g. Anthropic's message_delta carrying both a finish and a usage signal OpenAI splits across two chunks) or to translate an SSE event's own name/boundary between dialects at all, since that forwarded straight from the target dialect's wire bytes regardless of the source dialect's own vocabulary. Add a shared LlmEventMapper interface (LlmOpenaiEventMapper/ LlmAnthropicEventMapper already implemented per-event decode/encode; the non-streaming decodeMessage/encodeMessage pair is new) and a small LlmEventMapperFactory selecting one by dialect name. LlmClientFactory now routes a response through target.decode(...) -> source.encode(...) when both the source and target dialect have a registered mapper, falling back to the existing field-rename transform for any other dialect pairing (e.g. the test-only synthetic dialects). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NRZevD32tHJi45aRSueikf
Exercises the event-sequence mapper end-to-end through LlmClient against the existing mock openai/anthropic backends, in both directions and both the streaming and non-streaming shapes: an app declaring one dialect against a binding configured for the other. Reuses each direction's existing same-dialect mock-backend script and app-side request body unchanged (dialect field-rename is a no-op for the fields these requests use), so each new client.rpt asserts only the client's translated response. Streaming coverage includes the cardinality-changing translation points and the tool-call index-space mapping described in #2495: Anthropic's message_start/content_block_start(TEXT) merge into OpenAI's single first chunk; OpenAI's content-then-tool-call sequence lazily synthesizes Anthropic's content_block_stop only once the tool call starts (the one-event-lookahead case); OpenAI's finish_reason chunk splits into Anthropic's content_block_stop + message_delta; Anthropic's block-counting index (0 for the leading text block, 1 for the first tool call) round-trips through OpenAI's tool-call-only index space (0) and back. The openai-to-anthropic direction's message_delta carries usage:0 -- OpenAI sends its finish_reason chunk strictly before its usage chunk, and Anthropic's single message_delta event bundles both signals at finish time, so no output token count is available yet to bundle; this is the existing, already-unit-tested held-usage behavior (LlmAnthropicEventMapperTest shouldDropUsageAfterFinishAlreadySent), asserted here rather than changed. Fixes #2495 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NRZevD32tHJi45aRSueikf
…opic Locks in the three lossiness cases the canonical event vocabulary (llm.idl) documents but never had an explicit test: an OpenAI n>1 parallel completion's second choice is dropped at decode rather than corrupting the stream, a choiceIndex other than 0 is collapsed on Anthropic's encoded message_start, and per-delta logProbability never reaches Anthropic's content_block_delta. Also adds an OpenAI-side test proving inputTokens is deferred (not a placeholder zero) until a terminal chunk discloses it, matching LlmUsageFlushEx's documented contract. All four tests pass unmodified against the existing mapper implementation, confirming the vocabulary's documented behavior and code have not drifted. Fixes #2496 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UW6g8pWhrhNUuA2QjNJbm7
The dialect option's schema patch carried only a bare title, so nothing in the schema signaled that any registered dialect (openai, anthropic, ...) is accepted or that a client's dialect can legitimately differ from its paired server's, triggering automatic translation. Adds description text to that effect on the server/client dialect and client server properties, plus positive schema-validation coverage for an anthropic dialect and a server/client pair configured with different dialects. No new config fields were introduced by the cross-dialect translation work -- the existing dialect field already accepts any registered dialect name and translation between differing client/server dialects is fully automatic. Fixes #2497 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012NhoBsSK9W9fgX6iBcmRLw
dialect was a bare unconstrained string, so a misspelled or unregistered dialect name silently resolved to null at runtime (LlmDialectResolver) instead of failing at config-validation time. Seeds a `$defs/binding-ext/llm/dialect` scaffold in the schema patch, following the same convention already used by binding-mcp's `$defs/binding-ext/mcp`: the two dialects implemented in-tree (openai, anthropic) are enumerated there, and both the server and client dialect properties now $ref that shared enum instead of duplicating an open string type. Any future dialect -- in-tree or from another module or repository, contributed via its own BindingExtInfo schema patch -- can extend the same enum with a one-line JSON Patch append, exactly like binding-mcp-ext's index/eager type contributions in zilla-plus. Adds positive/negative schema-validation coverage: an unregistered dialect name is now rejected for both server and client, confirmed to fail against the prior open-string schema before this fix. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012NhoBsSK9W9fgX6iBcmRLw
…patch The dialect enum this branch introduced at /$defs/binding-ext/llm/dialect (openai, anthropic) rejected LlmClientIT's test-only dialect names (test-client, test-client-sse), which are registered purely by this module's own LlmDialectFactorySpi test implementations and never appear in a real deployment. LlmClientIT's client.yaml/client.opaque.yaml fixtures load through EngineConfigReader like any other config, so schema validation now rejected them before the engine could boot, breaking 4 tests (shouldForwardSameDialect, shouldForwardCrossDialect, shouldForwardClientOpaqueFallback, shouldAbortClientRequest). Adds LlmTestDialectExtInfo, a test-scope BindingExtInfo for type "llm" that appends the two test dialect names onto the same enum via a schema patch under src/test/resources, following the exact precedent already established by the engine's own type: test contribution (config/engine.conf's TestBindingInfo / TestBindingExtInfo, sourced from specs/engine.spec, documented in runtime/engine/CLAUDE.md). ServiceLoader picks it up only when binding-llm's own tests run -- engine.conf (which owns BindingExtInfo) is already a dependency, and EngineInfo.patches() applies all BindingInfo patches (which create /$defs/binding-ext/llm/dialect) before any BindingExtInfo patch, so the append always lands on an already-existing enum. The production dialect enum in binding-llm.conf stays closed to openai/anthropic for real deployments. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012NhoBsSK9W9fgX6iBcmRLw
Wires `llm server` into the options.authorization convention (guard name + credentials template), mirroring binding-mcp's server-side pattern. The dialect resolved for a request selects which header carries credentials (Authorization for openai, x-api-key for anthropic) via a new LlmDialect.credentialsHeader() extension point; a request whose header doesn't match the configured template, or whose extracted token the guard rejects, is turned away with a 401 response shaped in the resolved dialect's own JSON error envelope (LlmDialect.unauthorizedBody()). Adds LlmAuthorizationConfig/Builder (config surface), LlmBindingConfig guard resolution + authorize(), and LlmServerFactory's LlmUnauthorizedResponder for the reject path, plus config adapter tests and IT coverage (accept/reject, both dialects) mirroring binding-mcp's authorization IT pattern. Fixes #2498 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017zzRpze4nXEv4dbNjkRSVk
jacoco flagged binding-llm.conf at 0.97 instruction coverage (rule requires 1.00): LlmAuthorizationConfig.builder() (no-arg) and LlmAuthorizationConfigBuilder.thisType() were never exercised, since every existing usage goes through LlmOptionsConfigBuilder.authorization() (the mapper-taking overload). Adds the same builder()/inject() coverage pair LlmOptionsConfigTest already uses for LlmOptionsConfig and LlmServerConfig. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017zzRpze4nXEv4dbNjkRSVk
Wires `llm client` into the existing `options.authorization` convention for outbound credential handling, mirroring the server-side pattern (llm server options.authorization from the preceding commits) and binding-mcp's client- side credential forwarding (McpBindingConfig/McpClientFactory). - Extended llm.schema.patch.json with options.authorization under the kind: client branch, same patternProperties-by-guard-name shape already used on the server side. - Exposed LlmBindingConfig.guard and LlmBindingConfig.credentials (the raw template string) as public fields, and LlmBindingConfig.CREDENTIALS_PLACEHOLDER as a public constant, so client-side code can resolve outbound credentials. - LlmClientFactory resolves guard.credentials(authorization) once the stream's own authorization is a valid guard session, substitutes it into the configured template, and injects it as the dialect-selected header (target.credentialsHeader(): authorization for openai, x-api-key for anthropic) on the outbound request. When the authorization carried on the inbound application stream is itself a session already authorized by the same guard, this forwards the caller's own credential upstream unchanged (pass-through) rather than requiring Zilla to store a separate upstream secret. - Added k3po IT coverage mirroring the server-side authorization scenarios: new paired application/network scripts under incubator/binding-llm.spec for both dialects (openai.request.guarded, anthropic.request.guarded) plus new client.openai.guarded.yaml / client.anthropic.guarded.yaml fixtures using a type: test guard. - New LlmClientIT scenarios (live-engine) and ApplicationIT scenarios (spec-level self-consistency) for both dialects. Fixes #2499 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BG4hRScwRVV93odJitLfNs
…d client ITs The two new client-side forwarded-credentials scenarios (openai.request.guarded / anthropic.request.guarded) drive the caller's already-authorized session by connecting to app0 with a nonzero zilla:authorization option. LlmClientFactory forwards that same authorization value unchanged onto the outbound stream to net0. Since net0 is k3po's own "external" accept (not a real engine binding), its zilla:// transport rejects the connection with a RESET when the connect and accept sides carry mismatched authorization values -- so the network server script's accept needs the same option as the app-side connect for the scenario to reach the point where it can actually assert the injected credentials header. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BG4hRScwRVV93odJitLfNs
…skeletons Sets up the directory layout, docker-compose stack, and zilla.yaml wiring chained llm(server)+llm(client) bindings both directions (openai-facing frontend to an anthropic backend, and anthropic-facing frontend to an openai backend), with inline-guard credential pass-through. Mock backend server implementations and CI verify.sh assertions land in a follow-up commit.
…inding-llm Adds the mock OpenAI and Anthropic backend servers, the etc/test/verify.sh CI assertions (both translation directions, tool-call round trips, and both credential pass-through directions), and the example README. Also adds binding-llm to cloud/docker-image's dependencies and zpm.json.template -- it was missing from the docker image build, so the example's zilla.yaml (type: llm) would fail schema validation against the published image.
… mock responses llm(server)/llm(client) sit on top of http, the same way mcp does in examples/mcp.proxy -- they don't decode/encode HTTP/1.1 framing themselves. Insert http(server) between each tcp(server) and llm(server), and http(client) between each llm(client) and its tcp(client). Also fixes the mock backends: the llm(client)'s outbound request always targets "/" (PATH_DEFAULT in LlmClientFactory -- no per-dialect path is modeled yet), and Express's res.json() sends a "charset=utf-8" parameter that breaks the llm binding's exact-match response content-type lookup. Both mocks now listen on "/" and set the response Content-Type header explicitly without a charset.
onRequestBegin granted the request window unconditionally, before the network transport had actually connected. A request whose first DATA frame arrives before the connect completes (e.g. because the exit binding streams it eagerly right after BEGIN) could reach the network write path while the connection was still pending, which collapsed into the abort-on-pending-connect path in doNetShutdownOutput and silently dropped the request. Grant the window when the transport is already open, and otherwise defer it to flushNext(), which runs once onNetworkBegin confirms the connection is ready. This uses the existing flow-control mechanism to prevent the premature send rather than buffering the request data.
LlmClient.onAppBegin granted the application window as soon as the stream opened, without waiting for the network transport (delegated to an http client) to actually be ready. Mirrors the binding-http fix: the window is now granted once LlmHttpClient.onNetWindow reports the transport has become ready, using flow control to prevent a request from arriving before the connection can accept it.
LlmContentCodecFactory looked up decoders/encoders by exact content-type string match, which fails against a real-world Content-Type header carrying parameters (e.g. "application/json; charset=utf-8"). A missed lookup silently falls back to raw passthrough, skipping dialect translation entirely. Strip any ";"-separated parameters and match on the bare media type.
LlmServer's initialAuthorization was overwritten on every DATA frame with the raw per-frame authorization instead of keeping the guard-authorized session value captured at construction, so the exit stream never carried a valid guard session. This breaks the authorization-forwarding contract llm(client)'s options.authorization pass-through depends on: it resolves guard.credentials(authorization) only when the inbound stream's own authorization is itself a valid guard session. Threads authResult.authorization() through the LlmServer constructor as a final field instead. Add paired k3po scripts (server.openai.request.guarded, server.anthropic.request.guarded) for the exit-side stream carrying a non-zero authorization, since the existing openai.request.guarded / anthropic.request.guarded scripts are owned by the client-side scenario and expect a different request shape. Point LlmServerIT's guarded tests at the new scripts -- they previously referenced the unguarded server script, which only happened to work because the authorization value being forwarded was always 0.
The checked-in schema.expected.json fixture predates binding-llm's schema-ext registration, so the inspect.schema example's byte-for-byte comparison against the live merged JSON Schema was failing CI on this branch. Regenerated per the script's own documented procedure (docker compose exec zilla zilla inspect schema).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds
examples/llm.proxy, a docker-compose-based example demonstrating chainedllm(server)+llm(client)bindings translating between the OpenAI and Anthropic LLM API dialects, following theexamples/mcp.proxytemplate (README, compose.yaml, mock backend servers,etc/test/verify.shfor CI).The example wires two directions:
north_llm_server_openaifrontend backed entirely by a mock Anthropic API (mock-anthropic), proving request+response cross-dialect translation in both directions, including tool-call translation (tool_calls↔tool_use).options.authorizationpass-through on bothllm(server)andllm(client)), rather than requiring a separate secret configured inzilla.yaml.Building and end-to-end testing this example (via a live Docker stack, not just written config) surfaced four real defects, fixed here:
fix(binding-http):HttpClientFactory.onRequestBegingranted the request window unconditionally, before the network transport had actually connected. A request whose firstDATAframe arrives before connect completes could reach the network write path while the connection was still pending, colliding with the abort-on-pending-connect path indoNetShutdownOutputand silently dropping the request. The window grant is now deferred toflushNext(), which runs once the transport is confirmed ready — using the existing flow-control mechanism instead of buffering.fix(binding-llm)(client): the same deferral forllm(client)'s app-facing window, mirroring the binding-http fix.fix(binding-llm):LlmContentCodecFactorylooked up decoders/encoders by exact content-type string, which fails against a realContent-Type: application/json; charset=utf-8header. A missed lookup silently skipped dialect translation entirely (raw passthrough). Now strips;-separated parameters before lookup.fix(binding-llm):LlmServer'sinitialAuthorizationwas overwritten on everyDATAframe with the raw per-frame authorization instead of keeping the guard-authorized session value captured at construction, so the exit stream never carried a valid guard session — breaking the authorization pass-through contractllm(client)'soptions.authorizationdepends on. Threaded through as afinalfield instead. This exposed a pre-existing test-wiring bug inLlmServerIT's guarded scenarios (referencing an app-side script that only happened to pass because the forwarded authorization was always 0); fixed by adding correctly-authorized paired k3po scripts and a matchingApplicationITself-consistency scenario.Also fixes
binding-llmbeing missing fromcloud/docker-image's Maven dependencies andzpm.json.template— without it,type: llmfails schema validation in any real deployment built from the packaged image.Stacking
Like #2592, this branch was created from #2592's branch, so this diff includes those commits (and #2591's beneath it) until they merge to
develop, at which point this PR's diff will shrink to just its own.Test plan
examples/llm.proxy/etc/test/verify.shrun end-to-end against a live Docker stack (ghcr.io/aklivity/zilla:develop-SNAPSHOTbuilt from this branch): both translation directions, both tool-call round trips, both credential pass-through directions — all pass../mvnw verify -pl incubator/binding-llm.spec,incubator/binding-llm: 204 unit tests,NetworkIT(26/26),ApplicationIT(23/23),LlmClientIT(17/17),LlmServerIT(19/19) — all pass. Checkstyle/license clean../mvnw verify -pl runtime/binding-http: full k3po IT suite, 397 tests, 0 failures/errors. Checkstyle/license clean.Fixes #2500
🤖 Generated with Claude Code
https://claude.ai/code/session_01662aTQJ1ej8ovNf4YX4QvB
Generated by Claude Code