Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -242,8 +242,11 @@ Image-bearing messages require a qualified profile/placement before persistence
and image support from the selected Runtime before native delivery. These checks
apply to that operation only; ordinary text retains offline queueing. Initial,
prepared and active paths use the same content and preserve receipt ownership.
The qualified public profile is inline PNG/JPEG on Codex/Claude `none`; workspace
images, MiniMax images and remote URLs remain explicit implementation gaps. Core
The qualified public profile is inline PNG/JPEG on Codex/Claude `none` and
Core-managed Docker `openai_hosted`. Self-hosted/user-managed images, MiniMax
images and remote URLs remain explicit implementation gaps. Hosted images reuse
the existing preparation, active-input and workspace authority; they do not add
a downloader, a mount or a separate execution lifecycle. Core
does not fetch or transform media. See [message input coverage](contracts/agents-api/message-input.md).

User-managed onboarding creates a `self_hosted` Session first, then passes its
Expand Down Expand Up @@ -2264,8 +2267,8 @@ Idle and initial-input Session creation qualify the resolved configuration befor
persistence; saved Agent resources remain independent of engine restrictions.
Claude additionally requires medium verbosity and explicit object-root function
schemas. Function-result batches normalize through the existing shared parser. Claude accepts
text results and, on `none`, successful ordered inline PNG/JPEG results;
workspace images, failed image results and
text results and, on `none` and Core-managed Docker `openai_hosted`, successful
ordered inline PNG/JPEG results. Unqualified placements, failed image results and
invalid/remote references reject before any batch write, preserving pending calls
and retry identity. Public qualification receives the full neutral result so
success-dependent limitations remain in the profile. Image-bearing delivery alone
Expand Down
5 changes: 3 additions & 2 deletions contracts/agents-api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -418,7 +418,7 @@ operation and placement; native support is not public admission by itself.
| Engine | Qualified placements and limits |
| --- | --- |
| `codex` (default) | Qualified `none` and Docker `openai_hosted`; public functions with ordered text/image results; service-origin HTTP MCP on `none` only; verbosity follows native policy |
| `claude_sdk` | Qualified `none` and Docker `openai_hosted`; medium verbosity, object-root function schemas and text-only results; qualified anonymous/static-bearer service-origin HTTP MCP on `none` |
| `claude_sdk` | Qualified `none` and Docker `openai_hosted`; medium verbosity, object-root function schemas and text or successful inline PNG/JPEG results; qualified anonymous/static-bearer service-origin HTTP MCP on `none` |
| `mcode` | Qualified `none` text and Docker `openai_hosted`; medium verbosity; public functions/service-origin MCP, image input and complete public usage breakdown remain unsupported |

All three profiles implement user-managed `self_hosted` enrollment at `/workspace`
Expand Down Expand Up @@ -654,7 +654,8 @@ inheritance uses the same resolved tools. The bounded [deferred discovery path](
adds type-only `tool_search` for its qualified profile. Other discovery combinations, other tool kinds,
the native 64-definition cap and unique nonblank names of at most 512 bytes remain
compatibility gaps. Claude SDK additionally requires object-root schemas. It accepts text and
successful inline PNG/JPEG function results on `none`; failed/workspace images and remote references
successful inline PNG/JPEG function results on `none` and Docker `openai_hosted`;
failed images, unqualified placements and remote references
remain gaps. See [function image coverage](function-result-images.md). Codex internal Goal/Skills/user-input/discovery semantics need
upstream evidence; their presence alone does not prove a tool-set mismatch.

Expand Down
11 changes: 9 additions & 2 deletions contracts/agents-api/function-result-images.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@

The pinned official function result accepts a string or ordered text/image content,
independently of success. Our qualified Claude subset is successful inline PNG/JPEG
on `environment:none`. Error images, workspace images and remote URLs reject before
on `environment:none` and Core-managed Docker `openai_hosted`. Error images,
unqualified placements and remote URLs reject before
batch persistence, without consuming the pending call. These are implementation
gaps, not narrower official types. MiniMax public functions remain unqualified.

Expand Down Expand Up @@ -50,7 +51,13 @@ unsupported placement, operation-specific Runtime support and batch atomicity.
The bundled JPEG fixture has yellow, blue, red and green vertical bands; it contains
no metadata or credentials. PNG markers are generated with randomized band order.

The [Docker workspace workflow](message-input.md#docker-workspace-acceptance)
uses the same function-result contract alongside native file tools, public
Files/Artifacts and cold Core/Runtime continuation. It checks Claude's rejected
error/remote result directly on an outstanding call before accepting a valid
image on that same call; no mixed-message rejection substitutes for this check.

The accepted combinations and run evidence are recorded in the task board. This
batch does not qualify workspace image results, all native image limits, provider
coverage does not qualify self-hosted/user-managed image results, all native image limits, provider
parity, arbitrary managed output rewrites, crash recovery or full Agents API
compatibility. No downloader, image converter or second tool loop belongs in Core.
7 changes: 4 additions & 3 deletions contracts/agents-api/harnesses.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,17 +72,18 @@ syntactically or everything either upstream harness can theoretically perform.
| Docker hosted text execution, native local tools | Qualified | Qualified |
| Files, immutable Artifacts, cancellation, restart/history recovery | Qualified | Qualified |
| Public functions in `none` | Qualified | Qualified; object-root schemas; text or successful inline PNG/JPEG results |
| Public functions alongside hosted workspace tools | Qualified | Qualified; object-root schemas and text results |
| Public functions alongside hosted workspace tools | Qualified | Qualified; object-root schemas and text or successful inline PNG/JPEG results |
| HTTP MCP and static-bearer Vault credentials in `none` | Qualified | Qualified subset |
| Required MCP initialization | Qualified | Qualified on `none`; native readiness before initial input |
| Hosted HTTP MCP | Gap | Gap |
| Function image results | Supported subset; early acknowledgement is transport-only | Successful inline PNG/JPEG on `none`; native resizing allowed, error/workspace images rejected |
| Function image results | Supported subset; early acknowledgement is transport-only | Successful inline PNG/JPEG on `none` and Docker `openai_hosted`; native resizing allowed, error images rejected |
| Non-default verbosity | Native/model-dependent support | No equivalent qualified; medium only |
| Public detailed Usage | Supported native counters | Native raw usage retained; public breakdown gap |
| V1 `self_hosted` daemon enrollment at `/workspace` | [Qualified deployment scope](user-managed-runtime-v1.md) | [Qualified deployment scope](user-managed-runtime-v1.md) |
| Deferred function discovery | Unqualified; explicit rejection | [Single-agent text/function profile](tool-search.md) |
| Structured output | Unqualified; explicit rejection | [Qualified single-agent function profile](structured-output.md) |
| Explicit reasoning, message images | Shared service gaps | Shared service gaps |
| Message images | Inline PNG/JPEG on `none` and Docker `openai_hosted` | Inline PNG/JPEG on `none` and Docker `openai_hosted` |
| Explicit reasoning | Shared service gap | Shared service gap |
| Six Subagent reads | [Qualified scope](subagents.md) | [Qualified scope](subagents.md) |

This inventory records supported combinations, not a feature-equality checklist.
Expand Down
49 changes: 44 additions & 5 deletions contracts/agents-api/message-input.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ events share validation and atomic admission.

## Supported profile

Codex and Claude SDK support inline PNG/JPEG data URIs on `environment:none`, for
initial and active input and subsequent Turns. Use a real vision-capable model.
Codex and Claude SDK support inline PNG/JPEG data URIs on `environment:none` and
Core-managed Docker `openai_hosted`, for initial, prepared and active input. Use a real vision-capable model.
The existing 1 MiB HTTP and 512 KiB durable input limits still apply. A successful
events response acknowledges persistence, not native consumption. Active input
advances its durable receipt only after the adapter confirms application.
Expand Down Expand Up @@ -90,11 +90,50 @@ five repetitions, and shared input/dispatch race checks passed. These checks do
not qualify workspace images or additional native/provider combinations.
Native-only probes are feasibility evidence, not public qualification.

Workspace image workflows, MiniMax Code image input, remote HTTP(S) image URLs,
## Docker workspace acceptance

`tests/official_workspace_images_native.py` exposes `verify_workspace_images`
for an operator-owned standalone deployment. Supply the fixed SDK clients for
two tenants, their raw HTTP transport, a real vision model, the selected harness,
a cold Core/Runtime restart callback and a private evidence path. It creates and
deletes its own hosted Sessions; it never supplies model responses or credentials.

The common workflow covers initial text/PNG/text input, prepared image-only JPEG
followed by a separate message, active PNG input while a function waits, and a
6000x2100 PNG function result. The model must read the band order from image pixels
and write the corresponding bytes with native tools. Files listing and immutable
Artifact downloads verify those bytes. SDK and HTTP Items must retain the original
ordered input/results. The same workflow checks retry/conflict, unsupported-input
non-mutation, tenant and same-tenant Session isolation, cold history continuation,
pending cancellation and ordinary text after cancellation.

Real Kimi K3 acceptance on 2026-09-22 passed this workflow for Codex 0.153.4 and
Claude SDK 0.3.269/native 2.1.269. Evidence is retained under
`zju_a100_2:~/.parsar/remediation/20260922/workspace-images/`:
`codex/public-run-_lr5uiy_/` (152.78s) and
`claude_sdk/public-run-sb65p9e9/` (197.84s). Each run completed seven public Turns,
including one cancelled Turn, and cleaned up both owned hosted Sessions. Initial
Codex attempts exposed two acceptance-script errors: treating an earlier idle
event as the submitted Turn's completion and sending a scalar message to the
array-only events endpoint in a conflict probe. Both failures are retained;
production lifecycle behavior was not changed to obtain passing results.

Here `openai_hosted` means the Core-managed Docker deployment, with daemon, native
harness, tools and workspace in one sandbox. Image admission uses Environment type
and the common operation-specific Runtime support; it adds no provider-name branch,
media downloader, file permission or preparation lifecycle. Docker evidence does
not qualify other providers or user-managed deployment. Claude keeps its native
image-result receipt; Codex retains its documented transport-only result
acknowledgement. Neither implies crash-safe exactly-once tool effects.

## Remaining gaps

Self-hosted/user-managed image workflows, MiniMax Code image input, remote HTTP(S) image URLs,
other media types and full upstream error/default semantics remain unqualified.
MiniMax's fixed ACP advertises `image:false`; its adapter rejects images. These are
implementation gaps, not changes to the official protocol. JPEG parsing/conversion
has deterministic coverage; the recorded real visual fixtures are PNG. Empty
has deterministic coverage; the original `none` message fixtures are PNG, and
the hosted workflow also exercises JPEG. Empty
messages, local payload limits and native batch-size parity need upstream evidence.
Function-result image support is unchanged by this batch. No full protocol
Function-result image support has its own [coverage record](function-result-images.md). No full protocol
compatibility or support for arbitrary vision-model/provider combinations is claimed.
Loading
Loading