diff --git a/contracts/agents-api/README.md b/contracts/agents-api/README.md index d6843fcf5..4b5eccc27 100644 --- a/contracts/agents-api/README.md +++ b/contracts/agents-api/README.md @@ -75,6 +75,10 @@ variants work; **missing** means no implementation; **unverified** means behavio has not been shown to match upstream. Do not convert the route count into a compatibility percentage or treat a Docker result as E2B qualification. +The [execution and tools matrix](execution-tools.md) records message input, +structured output, tool configuration/results, discovery and required-action +recovery by operation, with exact qualified profiles and remaining gaps. + Paths below are SDK resource paths beneath `client.beta.agents`, except Skills and Versions under `client.skills`. Method names use the Python SDK. Vault HTTP paths start at `/vaults`, not `/agents/vaults`. @@ -396,7 +400,7 @@ and cancellation. This does not close the remaining protocol/transport gaps. | Independent deployment | Source-free Core package and separate execution PostgreSQL ownership; Docker-hosted and user-managed Runtime colocate daemon, selected harness and workspace; Core owns Docker only; no Parsar dependency | | Saved Agents and Sessions | Saved Agent routes, immutable inline/referenced Session configuration, metadata updates, root-Agent filtering and scoped cursor pagination | | Public execution | Initial/later text, active input and cancellation through Codex, Claude Code or MiniMax Code; Codex/Claude additionally support qualified public functions; see profile limits below | -| Pending function actions | Persisted calls/results/application receipts, `required_actions`, Session `requires_action`, Turn `waiting`, and live state snapshots; other interactions remain incomplete | +| Required actions | Persisted function calls/results/application receipts and pending Environment connection actions; Session reads and live snapshots expose the two pinned variants. See the [operation matrix](execution-tools.md) for qualification and unresolved timing | | Public recovery and SSE | Persisted Turn/Items queries and partial Usage; live lifecycle/Item/text events, creation streaming and the official one-Turn tool-handler helper | | Execution ownership | Immutable Session engine/device, durable input receipts and database writer fencing; uncertain claimed work fails on restart, without blind replay | | Files and Artifacts | Bounded Environment listing and inline/file_id copies into qualified V1 workspaces; source-file lifecycle and immutable output capture/download/deletion; [Files limits](environment-files.md), [source limits](source-files.md) | @@ -410,8 +414,9 @@ and cancellation. This does not close the remaining protocol/transport gaps. existing Sessions retain their immutable choice. `AGENTS_API_HARNESSES` explicitly adds installed deployment profiles without requiring a managed Provider. Model identity is independent. -All three profiles require implicit reasoning, service tier `auto` and ordinary -text output. Enabled `multi_agent` qualification is tracked separately in +All three profiles require implicit reasoning and service tier `auto`. Ordinary +text output is the baseline; [structured output](structured-output.md) has a +separately qualified Claude profile. Enabled `multi_agent` qualification is tracked separately in [Subagents](subagents.md); other profiles continue to reject unsupported execution. Optional tools/configuration are qualified per operation and placement; native support is not public admission by itself. diff --git a/contracts/agents-api/execution-tools.md b/contracts/agents-api/execution-tools.md new file mode 100644 index 000000000..f10d305fb --- /dev/null +++ b/contracts/agents-api/execution-tools.md @@ -0,0 +1,119 @@ +# Execution and tools coverage + +Assessed against Core `206474a4c10901442b1faa094281bfb5559082b5` on 2026-09-22. +The contract remains [OpenAI Python 3.13.0 at `d7c41ef`](upstream.json), Beta +`agents=v1`. This matrix records the bounded execution/tools milestone. It does not +establish complete protocol compatibility; the remaining limits below stay open. + +**Implemented** means current source and controlled checks support the operation. +**Qualified** additionally identifies a real public Core/PostgreSQL/daemon/native +model workflow in the evidence register below. **Unsupported** describes a current +admission/native limitation, not a restriction in the official contract. +**Unverified** means evidence does not establish the stated semantics or profile. +Qualification never extends automatically to another model, placement or combination. + +## Operation matrix + +All function rows refer to application-defined functions. Native workspace tools +and Environment Plugin MCP have separate inventories and qualification. + +| Operation | Implemented behavior and real qualification | Unsupported or unverified boundary | +| --- | --- | --- | +| Initial message input, `sessions.create` | String input and ordered user-message arrays share atomic admission. Codex/Claude text and inline image execution: M1/M2; ordinary MiniMax text: M1/P1. [Input contract](message-input.md), [initial parser](../../services/agents-api/internal/api/session_initial_input.go). | Empty-message behavior and local size-limit parity need upstream evidence. Images are only qualified for inline PNG/JPEG on `none` and Core-managed Docker `openai_hosted`; MiniMax, self-hosted images and remote URLs remain gaps. | +| Prepared and active messages, `sessions.events.create` | Ordered `input_text`/`input_image` arrays retain original content and distinct public user Items. HTTP 204 confirms persistence; native receipts establish application. Initial/prepared/active Docker PNG/JPEG: M2; active PNG on `none`: M1. [Shared admission](../../services/agents-api/internal/api/inputs.go). | Codex flattens native messages with blank-line separators. Claude can fold or queue native turns; it does not promise Codex's same-native-turn behavior. Public durability does not prove native consumption. | +| Structured output, `agent.text.format` | Save/inherit/freeze `{type:json_schema,schema:...}`. Claude SDK object-root, single Agent, medium verbosity, ordinary functions returning text: S1 (`none`) and S2 (Docker, including prepared/active input and Files/Artifacts). Native final text is retained unchanged. [Contract](structured-output.md), [profile](../../services/agents-api/internal/engine/claude.go). | Codex/MiniMax, other schema roots, schema numbers changed by binary64, self-hosted, Skills/Plugins, MCP, Subagent and discovery combinations reject execution. No output repair, coercion or extra model loop. Arbitrary schema dialects are unverified. | +| Function configuration, saved/inline Agents | Required name/description/schema; `defer_loading` defaults false. Saved references resolve into an immutable Session snapshot. Codex/Claude real calls: F1/F2/M2/S1/S2. [Parser](../../services/agents-api/internal/api/function_configuration.go), [saved tools](../../services/agents-api/internal/api/saved_tools.go). | MiniMax public functions reject. Claude requires object-root schemas. Local unique/nonblank name and 64-definition bounds are compatibility gaps. Saving configuration alone does not qualify execution. | +| Function-result admission, `events.create` | Required `turn_id`, `call_id`, `success`; optional nullable `error` and `output`. Output is string or ordered text/image content. Scoped atomic batches retain field presence, original content and retry identity. Same result retries are accepted; changed results conflict, including after terminal state. F1/F2/M2 plus [controlled SDK/raw checks](../../services/agents-api/tests/official_function_inputs.py). [Parser](../../services/agents-api/internal/api/function_inputs.go), [Store](../../services/agents-api/internal/store/function_results.go). | Admission is separate from application and public Item publication. Invalid or unqualified content cannot consume a pending call. Exact hosted errors, defaults and publication timing remain unverified. | +| Function text results and native application | Codex waits for a matching live root dynamic-tool completion; Claude waits for a matching live root native tool result. Text/error results, retry/conflict, cancellation and cold continuation: F1/F2. [Receipt contract](function-result-images.md). | Transport writes alone do not confirm application. Confirmation is not provider consumption, crash recovery or exactly-once external effects. No automatic result replay. | +| Function image results | Successful ordered inline PNG/JPEG with text, large PNG and image-only JPEG: Codex/Claude `none` F1/F2; Docker M2. Public Items retain submitted bytes. Codex receipt regression: F2. | Claude rejects failed images and remote references before persistence; native resizing may change its image bytes. MiniMax functions and self-hosted image workflows are unqualified. Other Codex image/error/reference combinations cannot be inferred from successful-inline evidence. | +| Deferred function discovery | Type-only `tool_search` plus mixed eager/deferred functions: Claude SDK 0.3.269/native 2.1.269, Kimi K3, single Agent, medium, `none`, text results; text and PNG input D1. Native provider observations establish lazy schema loading. [Contract](tool-search.md). | Codex/MiniMax discovery, search-only/missing-search/duplicate-search, workspace, MCP, structured-output and Subagent combinations remain gaps. Opaque native policy changes lack a reliable pre-input deferral signal. Saved tools include `tool_search`; the pinned Session response union excludes it. Exact hosted projection is unverified. | +| Explicit disabled search/PTC | Saved and inline `web_search.mode=disabled` and `programmatic_tool_calling.enabled=false`; shared native controls on initial and cold execution. All three harnesses on `none`: P1, including native inventory/control evidence. [Contract](tool-policy.md), [parser](../../services/agents-api/internal/api/disabled_tools.go). | Unsupported explicit enablement rejects. Omission retains approved native behavior, which does not establish official default-on PTC parity. Enabled search and default/error parity remain gaps; unrelated native utilities are not implicitly removed. | +| Agent service-origin MCP | Implemented Codex/Claude HTTP `none` profiles, anonymous/static bearer, scoped Vault selection and native `mcp_call` Items. [Configuration and qualification limits](../../services/agents-api/README.md#http-mcp-execution), [profiles](../../services/agents-api/internal/engine/profile.go). | This closure batch does not requalify MCP/provider combinations. MiniMax, self-hosted/hosted service-origin MCP, OAuth, nonempty inline headers/metadata and other transports/origins reject. Claude requires a static connected inventory; original MCP-envelope fidelity and continuing server health are unverified. | +| Environment Plugin MCP/native tools | Separate [Docker Plugin transport matrix](environment-templates.md#environment-origin-mcp-plugins) and [V1 deployment evidence](user-managed-runtime-v1.md). Native tools stay within the existing colocated Runtime. | Plugin MCP is not Agent service-origin MCP or a public function action. No new optional cross-product is qualified here. Native utility inventories need not be identical. | +| Required action: `function_call` | Session GET/list and Session SSE expose persisted `{arguments,call_id,name,turn_id,type}`. Actions remain pending until native application or cancellation/terminal settlement. [Projection](../../services/agents-api/internal/store/function_state.go), [confirmation](../../services/agents-api/internal/store/function_results.go); real handling F1/F2/M2/S1/S2/D1; explicit client disconnect/query/reconnect: F3. | Function-call history is not pending-state authority. Client reconnect does not replay events or redo external application effects. | +| Required action: `environment_connection` | Offline waiting input exposes `{environment_id,type}` before Turn creation. Connect the exact Session Environment through our scoped daemon enrollment. Three-harness Docker/E2B execution: E1; explicit initial pending-input/query/connection/native completion: E2; expiry: [controlled initial-input test](../../services/agents-api/tests/official_self_hosted_initial.py). | Idle offline Sessions without pending input request nothing. Registration alone is not connectivity or native readiness. Stock `exec-server`/Noise transport is outside the approved V1 route. Exact upstream registration/action-removal timing remains unverified. | +| Stream disconnect and execution loss | SSE is live-only. Reconnect, buffer events, retrieve Session/Turn/Items and deduplicate Item IDs. Worker recovery fails previously claimed work without replay; queued work can remain. [Recovery contract](README.md#public-execution-admission), [connection reconciliation](../../services/agents-api/internal/store/environment_connection_recovery.go). | Client disconnect and daemon/Core loss are different cases. F3 qualifies client-stream loss and continued pending function handling; F2 qualifies native process loss with an unapplied saved result. Neither promises replay or recovery of unknown external effects. | + +## Required-action recovery contract + +The fixed [Session union](https://github.com/openai/openai-python/blob/d7c41efee1b0802b79f3f88a678ef2052b06e9ce/src/openai/types/beta/agent_session.py) +defines both variants. The official [Functions](https://developers.openai.com/api/docs/guides/agents-api/tools/functions) +and [Manage sessions](https://developers.openai.com/api/docs/guides/agents-api/sessions/manage) +guides, checked on 2026-09-22, direct clients to retrieve `required_actions` after +restart or stream loss. A historical `function_call` Item alone cannot establish +that a result is still pending. The fixed SDK governs field/type compatibility; +current prose does not silently change that pin. + +For a pending function, use the returned Session, Turn and call identity. If the +application already performed the function, keep and submit that saved result; +do not run an external effect again merely because an action remains visible. +Core's acknowledgement boundary is native application. For an environment action, +connect its exact Environment using the existing enrollment authorization. +[Connection observations](../../services/agents-api/internal/store/environment_connections.go) +fence stale generations; transport connection can clear pre-Turn activity before +native preparation. Neither registration nor an action disappearing proves that +a model ran successfully. Query the matching Turn and Items for its outcome. + +Two timing questions remain open. Repeated `requires_action` notification order +and acknowledgement timing are local implementation choices, not proven upstream +semantics. Also, an admitted result that is cancelled before native observation +can remain internally saved without a public output Item or `item.added`. +[Item publication](../../services/agents-api/internal/store/item_projection.go) and +[the existing coverage record](README.md#public-function-configuration) preserve +this gap. Successful retry cannot manufacture the missing observation or establish +that the model consumed the result. + +## Evidence register + +Real records below use fixed SDK 3.13.0 and raw HTTP through Core, dedicated +PostgreSQL, daemon and native harness. Evidence is private under `~/.parsar/remediation/` on `zju_a100_2` and the +development host. Paths containing `validated/` or `server-evidence/` identify +downloaded local mirrors; summary files may also live only on the development +host. Locators identify retained evidence without publishing credentials or +private launch configuration. Revisions identify the accepted change, +not a claim that every historical workflow was rerun at the current baseline. + +| ID | Exact evidence and accepted scope | +| --- | --- | +| M1 | `20260922/message-image-input/public-codex-reviewed.log` (59.41s), `public-claude-reviewed.log` (79.52s), `public-mcode-network-fixed.log` (48.95s); [PR #12](https://github.com/MiniMax-AI/parsar-core/pull/12), merge `37a2923`. Codex/Claude Kimi K3 PNG initial/active inputs, receipts, retries, cancellation, cold continuation and isolation; MiniMax M2.7 ordinary text regression. | +| M2 | `20260922/workspace-images/validation-summary.json`; Codex `public-run-_lr5uiy_`, Claude `public-run-sb65p9e9`; candidate `2c295116d66ed9aafc808ec49a8cb6dcb5c674c2`, merge `1adb2489bc3415039440224e9a7905c53e68a557` ([#17](https://github.com/MiniMax-AI/parsar-core/pull/17)). Kimi K3, Codex 0.153.4, Claude SDK 0.3.269/native 2.1.269; Docker seven-Turn image/workspace workflow. Codex's receipt conclusion is superseded by F2. | +| F1 | `20260922/function-result-images/validation-summary.json`, `validated/function-image-public-2567456666/public.json`; candidate `d3cdb225bc7628b968b76c7e1b400101894ce8cd`, merge `fd23f169e1ede1b2f1c39a1d9dcce71886e3c006` ([#16](https://github.com/MiniMax-AI/parsar-core/pull/16)). Claude SDK 0.3.269/native 2.1.269 with Kimi K3 on `none`; success PNG/JPEG, failed text, retries, cancellation, history continuation. Earlier Codex transport-only evidence does not qualify current receipts. | +| F2 | `20260922/function-result-receipts/validation-summary.json`, `candidate-public-owner-sdk.log`; hosted `public-run-4fn70foy` (104.13s), `none` `function-image-public-945075091` (83.918s). Candidate `6dceb98e1c56469fcc70cd82025ce77fc870ed3d`, merge `206474a4c10901442b1faa094281bfb5559082b5` ([#20](https://github.com/MiniMax-AI/parsar-core/pull/20)); real Kimi, Codex 0.153.4. Qualifies native receipts, not completed provider consumption. | +| F3 | `20260922/execution-tools-closure/pending-actions/validation-summary.json`; Codex `codex/public-run-8gexf5a8` (65.58s), Claude `claude_sdk/public-run-5f2tr7e_` (74.33s), production source `206474a`. The same [public verifier](../../services/agents-api/tests/official_pending_actions_native.py) checks disconnected-client pending queries, success/error/cancel, original results, target isolation, retry/conflict and no implicit call replay with real Kimi on Docker. No Core restart or native-loss injection is claimed by these runs. | +| S1 | `20260922/structured-output/acceptance-summary.json`, `structured-public-1905281777/public.json`; real candidate `7f533d46c472f3cdf7c16ce9471c225a2ba7eded`, merge `8716e699dbc34904499c94b6b0b03857bbebfaad` ([#11](https://github.com/MiniMax-AI/parsar-core/pull/11)). Claude/Kimi `none` schema/function execution and cold continuation. Schema-specific active steering was not separately qualified here. | +| S2 | `20260922/hosted-structured-output/validation-summary.json`, `validated/public-inline.log`, `public-run-qvq3csf1` (186.09s); candidate `f6d2c3f2a1dc2ff5f177213bd2b20d78496011d0`, merge `aa85d8ecc60e0a8b7f2494b73caa81216ee197e2` ([#18](https://github.com/MiniMax-AI/parsar-core/pull/18)). Claude SDK 0.3.269/native 2.1.269, real Kimi; Docker saved/inline schema, prepared/active input, native files and four completed structured answers. | +| D1 | `20260922/deferred-tools/tool-search-public-2039155387/public.json`, `public-image-isolated-tests.log` (86.71s); merge `178507ae7a63d4068e1e82bef9cd256ba398ae00` ([#13](https://github.com/MiniMax-AI/parsar-core/pull/13)). Claude SDK 0.3.269/native 2.1.269/Kimi K3 `none`; text results with initial/active PNGs, cancellation and cold continuation. `claude-native-1790052750` separately records provider schema inventories and native feasibility. | +| P1 | `20260922/tool-policy/acceptance.json`, `server-evidence/tool-policy-{codex-2495445746,claude_sdk-2539346823,mcode-3747575401}/public.json`; candidate `eeb432c7495265ae3a9309e32f5b4323611c3245`, merge `4b8754a6eda6696d9b18eb8d583953ac16c6800f` ([#14](https://github.com/MiniMax-AI/parsar-core/pull/14)). Codex/Claude Kimi K3, MiniMax M2.7; four `none` Sessions per harness, native disable controls and cold continuation. | +| E1 | `20260921/self-hosted-onboarding/{source-candidate.json,source-verified.json,live,live-e2b}`; candidate `8f0cd2530d7b58cb7fb3ea124a1fc43dacecca36`. [Exact Docker/E2B run paths and limits](user-managed-runtime-v1.md#evidence-and-verification-boundaries): Codex Kimi K3 Responses, Claude Kimi K3 Anthropic-compatible API, MiniMax M2.7, immutable Linux amd64 Runtime builds. Native execution, restart/history, cancellation, Files/Artifacts and isolation; shared credential lifecycle evidence is not a separate live rotation run for every E2B profile. | +| E2 | `20260922/execution-tools-closure/environment-actions/result.json` contains Codex/Claude Kimi passes (13 checks each) and the retained MiniMax direct-network failure; `mcode-relay-retry/result.json` separately passes MiniMax M2.7 (13 checks) after correcting provider routing. Source `206474a`; same shared SDK/raw workflow, isolated Core/PostgreSQL and user-managed Docker. Covers initial creation SSE, client disconnect, exact pending Environment/no Turn or Items, scoped enrollment, one actual completed model Turn and creation retries preserving history. This is connection-action qualification, not another full lifecycle or E2B regression. | + +Controlled tests establish parser, projection, atomicity and race behavior; native +feasibility probes establish only feasibility. Neither replaces the real records. +Failed setup, provider and assertion runs remain failures with their original +scope. The unchanged gate requirements and independent review apply to this batch. + +## Milestone closure and remaining limits + +1. **Principal function recovery:** F3 directly verifies live client-stream loss + while a function is pending on Codex and Claude, authoritative queries, exact + targeting, rejection without mutation, original results, retry/conflict, + application, cancellation and isolation. F2 separately establishes truthful + native-loss settlement without replay. No active execution recovery is promised. +2. **Principal environment recovery:** E2 explicitly verifies pending + `environment_connection` recovery and single initial execution on all three + harnesses. E1 retains broader deployment/history/isolation qualification. + Registration, connection and preparation remain distinct; no exact upstream + timing or arbitrary-provider qualification is inferred. +3. **Recorded nonblocking timing:** an accepted result cancelled before native + application may lack a public result Item. The submitted data remains durable; + F2 confirms that native loss does not turn unknown delivery into Applied. The + fixed sources do not establish the publication point for this interleaving. + Keep it queued rather than inventing a public state or rewriting safe receipt + ownership. This limited timing question is not accepted protocol parity and + does not invalidate F3's principal pending-result workflow. + +Approved native capability differences (including default PTC), unsupported optional +combinations, arbitrary provider parity and low-frequency error/default edge cases +remain separate limitations. They do not authorize silently relabeling an unproved +principal workflow as deferred. OAuth, full Usage, release publication, new tool +kinds and new execution loops are outside this closure batch. diff --git a/services/agents-api/tests/official_pending_actions_native.py b/services/agents-api/tests/official_pending_actions_native.py new file mode 100644 index 000000000..3c5788781 --- /dev/null +++ b/services/agents-api/tests/official_pending_actions_native.py @@ -0,0 +1,165 @@ +"""Real hosted function-action recovery through the pinned SDK and public HTTP.""" + +import importlib.metadata +import json +from pathlib import Path +import uuid + + +def verify_pending_actions(client, foreign, http, model, evidence): + """The private runner supplies an isolated Core/native provider deployment.""" + pin = json.loads((Path(__file__).resolve().parents[3] / "contracts/agents-api/upstream.json").read_text()) + distribution = importlib.metadata.distribution("openai") + source = json.loads(distribution.read_text("direct_url.json") or "{}") + assert distribution.version == pin["sdk_version"] and source["vcs_info"]["commit_id"] == pin["commit"] + sessions = client.beta.agents.sessions + endpoint = str(client.base_url).rstrip("/") + "/agents/sessions" + headers = {"Authorization": "Bearer " + client.api_key, "OpenAI-Beta": "agents=v1"} + foreign_headers = {**headers, "Authorization": "Bearer " + foreign.api_key} + proof = {"sdk": pin, "checks": [], "rounds": [], "passed": False} + created = [] + marker = "pending-value-" + uuid.uuid4().hex + agent = {"model": model, "instructions": "Call lookup exactly once when asked. Never retry a failed tool call. Use no other tools.", "tools": [{ + "type": "function", "name": "lookup", "description": "Retrieve the requested test value.", + "parameters": {"type": "object", "properties": {"key": {"type": "string"}}, "required": ["key"], "additionalProperties": False}, + }]} + + def read(path, sdk): + response = http.get(endpoint + path, headers=headers) + assert response.status_code == 200, (path, response.status_code) + raw = response.json() + assert raw == sdk.to_dict(), path + return raw + + def items(sid): + response = http.get(endpoint + "/" + sid + "/items", headers=headers, params={"limit": 100, "order": "asc"}) + assert response.status_code == 200 + raw = response.json()["data"] + assert raw == [item.to_dict() for item in sessions.items.list(sid, limit=100, order="asc").data] + return raw + + def submit(sid, event, key, expected=204, auth=None): + response = http.post(endpoint + "/" + sid + "/events", headers={**(auth or headers), "Idempotency-Key": key}, json={"events": [event]}) + assert response.status_code == expected, (key, response.status_code, response.text) + if expected == 204: + assert response.content == b"" + else: + assert response.json()["error"]["code"] in {"not_found", "turn_conflict", "idempotency_conflict"} + return {"request": key, "status": response.status_code} + + try: + session = sessions.create(agent=agent, environment={"type": "openai_hosted"}) + created.append(session.id) + other = sessions.create(agent=agent, environment={"type": "openai_hosted"}) + created.append(other.id) + proof.update(session=session.id, other_session=other.id) + for index, mode in enumerate(("success", "error", "cancel")): + current = {"mode": mode, "before_disconnect": [], "after_reconnect": [], "refusals": []} + proof["rounds"].append(current) + prompt = "Call lookup exactly once with key " + mode + ". Then reply with its returned text. If it fails, acknowledge the error without retrying." + with sessions.events.stream(session.id, timeout=240) as stream: + sessions.events.create(session.id, events=[{"type": "agent.session.input.message", "input": [{"role": "user", "content": [{"type": "input_text", "text": prompt}]}]}], idempotency_key="pending-message-" + mode) + for event in stream: + current["before_disconnect"].append(event.to_dict()) + assert event.type not in ("agent.session.turn.failed", "agent.session.failed"), event.to_dict() + if event.type == "agent.session.requires_action": + break + else: + raise AssertionError("Stream ended before a pending action") + + # The closed stream is not the recovery authority. Query both resources. + recovered = read("/" + session.id, sessions.retrieve(session.id)) + actions = recovered["required_actions"] + assert recovered["status"] == "requires_action" and len(actions) == 1 + action = actions[0] + assert action == {"type": "function_call", "name": "lookup", "arguments": {"key": mode}, "call_id": action["call_id"], "turn_id": action["turn_id"]} + turn_path = "/" + session.id + "/turns/" + action["turn_id"] + turn = read(turn_path, sessions.turns.retrieve(action["turn_id"], session_id=session.id)) + assert turn["status"] == "waiting" + assert sessions.turns.list(session.id).data[0].id == action["turn_id"] + current.update(recovered_session=recovered, recovered_turn=turn, items_while_pending=items(session.id)) + event = {"type": "agent.session.input.tool_result", "turn_id": action["turn_id"], "call_id": action["call_id"], "success": mode != "error", "output": [{"type": "input_text", "text": marker}, {"type": "input_text", "text": "original second part"}]} + if mode == "error": + event["error"] = "original caller failure " + marker + + def pending_unchanged(): + state = sessions.retrieve(session.id) + assert state.status == "requires_action" and [a.to_dict() for a in state.required_actions] == actions + assert sessions.turns.retrieve(action["turn_id"], session_id=session.id).status == "waiting" + assert not any(item["type"] == "function_call_output" and item["call_id"] == action["call_id"] for item in items(session.id)) + assert sessions.retrieve(other.id).status == "idle" + assert sessions.turns.list(other.id).data == [] and items(other.id) == [] + + for label, target, value, auth in ( + ("wrong-tenant", session.id, event, foreign_headers), + ("wrong-session", other.id, event, headers), + ("wrong-turn", session.id, {**event, "turn_id": proof["rounds"][index - 1]["recovered_turn"]["id"] if index else str(uuid.uuid4())}, headers), + ("wrong-call", session.id, {**event, "call_id": "absent-" + uuid.uuid4().hex}, headers), + ): + current["refusals"].append(submit(target, value, label + "-" + mode, 404, auth)) + pending_unchanged() + for path in ("/" + session.id, turn_path, "/" + session.id + "/items"): + assert http.get(endpoint + path, headers=foreign_headers).status_code == 404 + proof["checks"].append(mode + "_disconnected_query_and_target_isolation") + + # Opening another live stream does not submit or replay any input. + with sessions.events.stream(session.id, timeout=240) as stream: + pending_unchanged() + submitted = {"type": "agent.session.input.cancel"} if mode == "cancel" else event + key = "pending-submit-" + mode + sessions.events.create(session.id, events=[submitted], idempotency_key=key) + current["retry"] = submit(session.id, submitted, key) + if mode != "cancel": + current["conflict"] = submit(session.id, {**event, "output": "changed"}, key, 409) + current["new_identity_conflict"] = submit(session.id, {**event, "output": "changed"}, key + "-changed", 409) + for received in stream: + current["after_reconnect"].append(received.to_dict()) + if received.type == "agent.session.idle": + break + assert received.type != "agent.session.failed", received.to_dict() + else: + raise AssertionError("Reconnected stream ended before settlement") + + terminal = sessions.turns.retrieve(action["turn_id"], session_id=session.id) + assert terminal.status == ("cancelled" if mode == "cancel" else "completed"), terminal.to_dict() + settled = read("/" + session.id, sessions.retrieve(session.id)) + assert settled["status"] == "idle" and settled["required_actions"] == [] + current["terminal"] = read(turn_path, terminal) + before_ids = {e["event_id"] for e in current["before_disconnect"]} + after_ids = {e["event_id"] for e in current["after_reconnect"]} + assert before_ids.isdisjoint(after_ids), "Reconnected live stream replayed previous events" + terminal_events = [e for e in current["after_reconnect"] if e["type"] in ("agent.session.turn.completed", "agent.session.turn.failed", "agent.session.turn.cancelled")] + assert len(terminal_events) == 1 and terminal_events[0]["turn"]["id"] == action["turn_id"] + assert terminal_events[0]["type"] == "agent.session.turn." + terminal.status + current["terminal_retry"] = submit(session.id, submitted, key) + if mode == "cancel": + current["late_result"] = submit(session.id, event, key + "-late-result", 409) + else: + current["same_result_new_identity"] = submit(session.id, event, key + "-same-result") + + durable = items(session.id) + outputs = [i for i in durable if i["type"] == "function_call_output" and i["call_id"] == action["call_id"]] + assert len(outputs) == (0 if mode == "cancel" else 1) + if outputs: + assert {k: outputs[0][k] for k in ("output", "error") if k in outputs[0]} == {k: event[k] for k in ("output", "error") if k in event} + if mode == "success": + answers = [i for i in durable if i["type"] == "message" and i["role"] == "assistant"] + assert marker in "\n".join(p.get("text", "") for p in answers[-1]["content"]) + assert len(sessions.turns.list(session.id).data) == index + 1 + current["durable_items"] = durable + proof["checks"].append(mode + "_reconnect_settlement_retry_and_original_items") + + # Reads and retries leave the three native calls and two original results intact. + durable = items(session.id) + calls = [i for i in durable if i["type"] == "function_call"] + assert len(calls) == 3 and len({i["call_id"] for i in calls}) == 3 + assert len([i for i in durable if i["type"] == "function_call_output"]) == 2 + assert len(sessions.turns.list(session.id).data) == 3 + assert sessions.retrieve(session.id).required_actions == [] + proof["checks"].append("durable_queries_no_implicit_call_replay") + proof["passed"] = True + return proof["checks"] + finally: + Path(evidence).write_text(json.dumps(proof, indent=2)) + for sid in reversed(created): + sessions.delete(sid)