Observe committed root Turn outcomes for deployment model defaults

CONTRIBUTING.md

@@ -2502,6 +2502,26 @@ replaced; do not carry obsolete compatibility code forward to satisfy this secti
   retries return committed state without replay. No Turn-level overrides or provider catalog is
   included. Public input/null semantics and examples live in
   `contracts/agents-api/model-execution.md`.
+- Deployment-default observations use a private UUID generated on every PUT,
+  including identical replacements. Read its ciphertext/revision together and freeze
+  the pair only at new Session creation; retries and historical Sessions never gain
+  or replace revision metadata. After a successful root terminal commit, one
+  independent pool operation has at most one second to update the matching current
+  revision. SQL verifies tenant, root Turn and committed outcome; only completed
+  Turns and fixed native provider failures with authoritative `engine_failed` qualify.
+  Never observe cancelled work, Core/runtime errors or input-policy classifications.
+  The metadata-only transaction sets server statement/lock timeouts within the
+  remaining client budget and issues one UPDATE, so a client timeout cannot leave
+  an indefinitely waiting statement. The update locks only the default and samples
+  DB receipt time after the lock.
+  Errors throttle for 30 seconds regardless of code; ordinary successes throttle
+  for 30 seconds, with one immediate recovery write after each accepted error.
+  An unchanged revision has at most three effective writes in any half-open
+  30-second interval of nondecreasing DB time. Clock rollback may suppress ordinary
+  writes; private recovery state permits only one recovery without clamping time.
+  Observations never change `updated_at`, readiness or execution truth. They can be
+  lost or stay stale indefinitely without another eligible Turn; there is no queue,
+  retry, probe, history backfill or provider text in the safe fields.
 - Session execution-configuration reads use a separate immutable safe projection,
   written with provenance in the same creation transaction as Session resources.
   Read no credential ciphertext and never recompute sources from current Agents

apps/docs/content/docs/api-reference/core/deployment-model-providers.mdx

@@ -18,7 +18,11 @@ _openapi:
           never returned; api_key_configured reports that one is set.
       - content: >-
           Core key only. Returns the safe view; the key is never returned. 404
-          when the harness does not exist or has no deployment default.
+          when the harness does not exist or has no deployment default. Nullable
+          last_used_at, last_error_code and last_error_at are best-effort
+          observations of committed root Turns using this exact default
+          revision; they do not establish current readiness and may remain stale
+          indefinitely.
       - content: >-
           Core key only. Idempotent; each successful request is audited.
           Sessions that already froze the default keep it. Afterwards new
@@ -35,7 +39,8 @@ _openapi:
           bundle freeze this default into their encrypted snapshot; existing
           Sessions never change. self_hosted Sessions never use it. The key is
           encrypted and never returned. Each write records an administrator
-          audit entry without the key.
+          audit entry without the key and resets last_used_at, last_error_code
+          and last_error_at to null, including identical writes.
 ---
 
 {/* This file was generated by Fumadocs. Do not edit this file directly. Any changes should be made by running the generation command again. */}

apps/docs/openapi/core-api.yaml

@@ -152,7 +152,7 @@ paths:
       tags:
         - Deployment Model Providers
     get:
-      description: Core key only. Returns the safe view; the key is never returned. 404 when the harness does not exist or has no deployment default.
+      description: Core key only. Returns the safe view; the key is never returned. 404 when the harness does not exist or has no deployment default. Nullable last_used_at, last_error_code and last_error_at are best-effort observations of committed root Turns using this exact default revision; they do not establish current readiness and may remain stale indefinitely.
       parameters:
         - schema:
             type: string
@@ -195,7 +195,7 @@ paths:
       tags:
         - Deployment Model Providers
     put:
-      description: 'Core key only. The body is the complete x_agents_core.model_provider bundle, including the write-only api_key; there is no partial update and bundles are never merged. The provider is validated for this harness: an HTTPS base_url without credentials, query or fragment, the harness''s protocol (responses for codex, anthropic for claude_sdk and mcode) and, for mcode, positive context_window and max_output_tokens. New openai_hosted and none Sessions that resolve no Session or Agent bundle freeze this default into their encrypted snapshot; existing Sessions never change. self_hosted Sessions never use it. The key is encrypted and never returned. Each write records an administrator audit entry without the key.'
+      description: 'Core key only. The body is the complete x_agents_core.model_provider bundle, including the write-only api_key; there is no partial update and bundles are never merged. The provider is validated for this harness: an HTTPS base_url without credentials, query or fragment, the harness''s protocol (responses for codex, anthropic for claude_sdk and mcode) and, for mcode, positive context_window and max_output_tokens. New openai_hosted and none Sessions that resolve no Session or Agent bundle freeze this default into their encrypted snapshot; existing Sessions never change. self_hosted Sessions never use it. The key is encrypted and never returned. Each write records an administrator audit entry without the key and resets last_used_at, last_error_code and last_error_at to null, including identical writes.'
       parameters:
         - schema:
             type: string
@@ -4454,6 +4454,30 @@ components:
             - codex
             - mcode
           type: string
+        last_error_at:
+          anyOf:
+            - format: date-time
+              type: string
+            - type: 'null'
+        last_error_code:
+          anyOf:
+            - enum:
+                - authentication_error
+                - connection_failed
+                - rate_limit_exceeded
+                - usage_limit_exceeded
+                - server_overloaded
+                - server_error
+                - resource_not_found
+                - request_timeout
+                - invalid_request
+              type: string
+            - type: 'null'
+        last_used_at:
+          anyOf:
+            - format: date-time
+              type: string
+            - type: 'null'
         max_output_tokens:
           type: integer
         object:
@@ -4471,6 +4495,9 @@ components:
         - api_key_configured
         - base_url
         - harness
+        - last_error_at
+        - last_error_code
+        - last_used_at
         - object
         - protocol
         - updated_at

apps/docs/openapi/sources.json

@@ -1,7 +1,7 @@
 {
   "sources": {
     "contracts/agents-api/openapi.yaml": "a8896e8b67d78ed0608c7cc5636077ec33f4b4895a04bd6cfb8b038afe941e01",
-    "contracts/agents-api/core.openapi.yaml": "8fe158844c170eea78d4b24e30e7caa2adbedd58ffb4a863bdd9c98ecbb50618",
+    "contracts/agents-api/core.openapi.yaml": "7801fc77259893815fe9cdcb404f045b375711b9b075eab2b2ed8c5dba55e24b",
     "contracts/agents-api/runtime.openapi.yaml": "23852205d3b45892f2de9ecf3d52518c0dd0808962c0c512f219b565aa7ccef2"
   },
   "outputs": {
@@ -22,7 +22,7 @@
     "content/docs/api-reference/meta.json": "56bf1dc145ccc8adde38c5f956f1a06f18ef18b646122250d01a294c516f8f83",
     "content/docs/api-reference/index.mdx": "9fd6b1c54149874d2999235067c101326f9e8a2e707a70b6672c81eb3ebfe63f",
     "content/docs/api-reference/core/core-administration.mdx": "5fc3ada51d2190a141f13738151e4d513390cc3bd66ce6f5013994bda68569b0",
-    "content/docs/api-reference/core/deployment-model-providers.mdx": "b2795fadcc05a1e7195b6d34c3f234cb29cc5e3f580d1521e31965453e654347",
+    "content/docs/api-reference/core/deployment-model-providers.mdx": "d2e1ed267def55e1b64fdf28fbec81f83b9c2556b1db4392ede6a19b1465b50c",
     "content/docs/api-reference/core/administrator-projects.mdx": "8fd687bb2a7119fa16e938d7c71710c2816afb9cf959eb6849ef5e6300c5419b",
     "content/docs/api-reference/core/agents.mdx": "b6c089fbea4b07bc4249e7c7ca5699d47f1512a4fc2d8ee01cb26910ae541a84",
     "content/docs/api-reference/core/environment-templates.mdx": "05907f35d7f02d66af9da8b1efaa3765badb4cdf7b94d0f6c3ab0413a2cfa1a9",
@@ -40,7 +40,7 @@
     "content/docs/api-reference/core/vaults.mdx": "12d0aac96cd4a3472ef4eaa287f7945f78862017956e79bfc3ed0ef8e772e78a",
     "content/docs/api-reference/core/credentials.mdx": "27d586b9a8fd738b9947802f32c80458195370252886bc26abedccf2125ee981",
     "content/docs/api-reference/core/sandbox-manager.mdx": "6be7c52e057365949f0247c18c710561d680847a955f78f1bbe57825d4d5f549",
-    "openapi/core-api.yaml": "4d72411e4fc9945aba575a91b9c95e7231262e2749f87796646ef17768022238",
+    "openapi/core-api.yaml": "7079694e447c0783f85aeb3b73f23af68ebb066540c67251bbae34e7e4bd8e62",
     "content/docs/api-reference/core/meta.json": "588b163ed2de2100b0f8376688274f80698a6bf6ac037eef56d17c93fa8a5062",
     "content/docs/api-reference/core/index.mdx": "ddf0a8d87a6c21761beac703a32f2a40486a1c74abb341bd726f40c44a866405",
     "content/docs/api-reference/machine/sandbox-node.mdx": "5b367a08d43cea82686345d2ba1c75cc90627a7b2970fe42a0df9d77fb3e1e80",

apps/web/e2e/fixture-console.mjs

@@ -118,7 +118,7 @@ function reset(mode = "login", fresh = false, sandbox = "configured", nodes = "d
     // The demo deployment's default harness has a default model; a fresh install has none.
     harnesses: {
       claude_sdk: { enabled: true, default: false, provider: null },
-      codex: { enabled: true, default: true, provider: fresh ? null : { object: "core.model_provider", harness: "codex", protocol: "responses", base_url: "https://model.example/v1", api_key_configured: true, updated_at: new Date((now - 86400) * 1000).toISOString().replace(/\.\d{3}Z$/, "Z") } },
+      codex: { enabled: true, default: true, provider: fresh ? null : { object: "core.model_provider", harness: "codex", protocol: "responses", base_url: "https://model.example/v1", api_key_configured: true, last_used_at: null, last_error_code: null, last_error_at: null, updated_at: new Date((now - 86400) * 1000).toISOString().replace(/\.\d{3}Z$/, "Z") } },
       mcode: { enabled: false, default: false, provider: null },
     },
     // "none": the deployment is not configured yet, so the Nodes page offers setup.
@@ -534,6 +534,7 @@ async function harnessRoute(request, response, path) {
     object: "core.model_provider", harness, protocol: input.protocol, base_url: input.base_url, api_key_configured: true,
     ...(input.context_window ? { context_window: input.context_window } : {}),
     ...(input.max_output_tokens ? { max_output_tokens: input.max_output_tokens } : {}),
+    last_used_at: null, last_error_code: null, last_error_at: null,
     updated_at: new Date().toISOString().replace(/\.\d{3}Z$/, "Z"),
   };
   return send(response, 200, entry.provider);

apps/web/src/features/overview/getting-started.test.ts

@@ -11,7 +11,7 @@ const deployment = (overrides: Partial<SandboxDeployment> = {}): SandboxDeployme
 });
 const fleet = (value: SandboxDeployment, nodes = [node("n1")]): FleetState => ({ status: "ready", snapshot: { deployment: value, nodes, allocations: [], loadedAt: 0 }, refreshing: false, error: null });
 const sandboxes = (state: FleetState) => gettingStartedSteps({ sandboxReset: false, fleet: state, localOnly: false, projects: [], sessions: 0, harnesses: [] }).sandboxes;
-const provider = { object: "core.model_provider", protocol: "responses", base_url: "https://model.example/v1", api_key_configured: true, updated_at: "2026-09-25T00:00:00Z" } as const;
+const provider = { object: "core.model_provider", protocol: "responses", base_url: "https://model.example/v1", api_key_configured: true, last_used_at: null, last_error_code: null, last_error_at: null, updated_at: "2026-09-25T00:00:00Z" } as const;
 const harness = (id: CoreHarness["id"], fields: Partial<CoreHarness> = {}): CoreHarness => ({ object: "core.harness", id, enabled: true, default: false, model_provider: null, ...fields });
 
 describe("Getting started steps", () => {

contracts/agents-api/core-errors.md

@@ -73,7 +73,12 @@ message, template name, key or unlisted-resource count is returned in details.
 
 Missing or unsettled Create receipts are uncertainty, never evidence of a different
 team or released compute. The typed client projects these codes to fixed local
-messages. It also projects `409 sandbox_configuration_error` to fixed public-URL
+messages. It preserves the three nonnull fields above only when the response
+status, code and param exactly match the table; all other params on these fixed
+errors become null. Numeric details remain limited to `current_generation` for
+`sandbox_generation_stale` and `allocations`/`pending` for `sandbox_in_use`. Unknown
+credential-bearing errors become `sandbox_configuration_unconfirmed` without replay.
+It also projects `409 sandbox_configuration_error` to fixed public-URL
 guidance, even when a PUT omitted its key; arbitrary upstream text is never echoed.
 
 ## Operation validation

contracts/agents-api/core.openapi.yaml

@@ -346,6 +346,27 @@ definitions:
           - codex
           - mcode
         type: string
+      last_error_at:
+        format: date-time
+        type: string
+        x-nullable: true
+      last_error_code:
+        enum:
+          - authentication_error
+          - connection_failed
+          - rate_limit_exceeded
+          - usage_limit_exceeded
+          - server_overloaded
+          - server_error
+          - resource_not_found
+          - request_timeout
+          - invalid_request
+        type: string
+        x-nullable: true
+      last_used_at:
+        format: date-time
+        type: string
+        x-nullable: true
       max_output_tokens:
         type: integer
       object:
@@ -363,6 +384,9 @@ definitions:
       - api_key_configured
       - base_url
       - harness
+      - last_error_at
+      - last_error_code
+      - last_used_at
       - object
       - protocol
       - updated_at
@@ -3429,7 +3453,7 @@ paths:
       tags:
         - Deployment Model Providers
     get:
-      description: Core key only. Returns the safe view; the key is never returned. 404 when the harness does not exist or has no deployment default.
+      description: Core key only. Returns the safe view; the key is never returned. 404 when the harness does not exist or has no deployment default. Nullable last_used_at, last_error_code and last_error_at are best-effort observations of committed root Turns using this exact default revision; they do not establish current readiness and may remain stale indefinitely.
       parameters:
         - description: Harness
           enum:
@@ -3467,7 +3491,7 @@ paths:
     put:
       consumes:
         - application/json
-      description: 'Core key only. The body is the complete x_agents_core.model_provider bundle, including the write-only api_key; there is no partial update and bundles are never merged. The provider is validated for this harness: an HTTPS base_url without credentials, query or fragment, the harness''s protocol (responses for codex, anthropic for claude_sdk and mcode) and, for mcode, positive context_window and max_output_tokens. New openai_hosted and none Sessions that resolve no Session or Agent bundle freeze this default into their encrypted snapshot; existing Sessions never change. self_hosted Sessions never use it. The key is encrypted and never returned. Each write records an administrator audit entry without the key.'
+      description: 'Core key only. The body is the complete x_agents_core.model_provider bundle, including the write-only api_key; there is no partial update and bundles are never merged. The provider is validated for this harness: an HTTPS base_url without credentials, query or fragment, the harness''s protocol (responses for codex, anthropic for claude_sdk and mcode) and, for mcode, positive context_window and max_output_tokens. New openai_hosted and none Sessions that resolve no Session or Agent bundle freeze this default into their encrypted snapshot; existing Sessions never change. self_hosted Sessions never use it. The key is encrypted and never returned. Each write records an administrator audit entry without the key and resets last_used_at, last_error_code and last_error_at to null, including identical writes.'
       parameters:
         - description: Harness
           enum:

contracts/agents-api/model-execution.md

@@ -182,3 +182,30 @@ Parsar manages its own workspace catalog and encrypted keys, sends this extensio
 only on the first Core Session request, and retains a private encrypted snapshot
 for uncertain creation retries. Catalog updates and deletion affect new Sessions;
 existing Sessions retain their original model, endpoint and key.
+
+### Deployment default observations
+
+The Core-only harness/default-provider reads include nullable `last_used_at`,
+`last_error_code` and `last_error_at`. PUT resets all three, even for the same bundle.
+Completed root Turns contribute use observations only when their Session froze
+that exact current default revision. Failed root Turns contribute only the fixed
+native provider codes `authentication_error`, `connection_failed`,
+`rate_limit_exceeded`, `usage_limit_exceeded`, `server_overloaded`, `server_error`,
+`resource_not_found`, `request_timeout` and `invalid_request`. Input-policy,
+Core/runtime, cancelled and waiting outcomes do not contribute. Successful use
+retains the earlier error; comparing timestamps is only a display convention.
+
+These are best-effort Core receipt times after terminal commit, not provider health
+or remote completion times. Private revision identity is independent of timestamps;
+old, explicit-provider and historical Sessions cannot update a replacement default.
+No public Session/Turn fields or retry identity change. The observation has a
+one-second budget including pool/row-lock acquisition and cannot change the committed
+Turn. A crash, failure or throttle can omit the last observation indefinitely.
+
+Errors throttle for 30 seconds regardless of code. Ordinary successful writes
+throttle for 30 seconds; the first success after an accepted error records recovery
+immediately. For an unchanged revision this allows at most three effective metadata
+writes in any half-open 30-second interval of nondecreasing DB-clock time. Equal or
+backwards clock readings are preserved and can make timestamp-based display
+ambiguous; recovery does not impose a total order. No readiness probe, automatic
+refresh, observation history or credential/raw-error read is provided.

packages/agents-client/src/admin-client.test.ts

@@ -154,7 +154,7 @@ describe("AdminClient transport boundary", () => {
   });
 
   it("manages deployment default model providers without ever reading a key", async () => {
-    const provider = { object: "core.model_provider", harness: "codex", protocol: "responses", base_url: "https://model.example/v1", api_key_configured: true, updated_at: "2026-09-26T08:00:00Z" };
+    const provider = { last_used_at: null, last_error_code: null, last_error_at: null, object: "core.model_provider", harness: "codex", protocol: "responses", base_url: "https://model.example/v1", api_key_configured: true, updated_at: "2026-09-26T08:00:00Z" };
     const harnesses = { object: "list", data: [
       { object: "core.harness", id: "claude_sdk", enabled: false, default: false, model_provider: null },
       { object: "core.harness", id: "codex", enabled: true, default: true, model_provider: provider },