From 82bcc7f8691cf083a5c367a3eb36379d8171379b Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Fri, 25 Sep 2026 12:44:54 +0000 Subject: [PATCH 1/4] Record when an allocation entered its compute phase; keep Docker's retained limit equal to its active limit - Migration 000072 adds runtime_allocations.compute_phase_changed_at (NULL for existing rows, clock_timestamp() for new ones). SetRuntimeCompute moves it only when the phase actually changes. - GET /core/v1/sandbox/nodes/{node_id}/allocations returns compute_phase_changed_at (nullable) per item; the management OpenAPI and the SandboxAllocation client type include it. - Docker never suspends, so a Docker node's max_retained always equals its max_active. Enrollment tokens and node PATCH replace any submitted value inside the deployment transaction before validating; the node list reports it that way. Provider-independent limits are still rejected before the transaction. - Capacity tests: the approved-capacity test now runs on microsandbox, which uses both limits; TestDockerRetainedLimitFollowsActive covers Docker, and TestRuntimeComputePhaseChangedAt covers the phase time. - Documentation describes the new field and the Docker retained limit. --- contracts/agents-api/sandbox-deployment.md | 19 +++++++--- .../agents-api/sandbox-manager.openapi.yaml | 6 +++ docs/configuration.md | 2 + docs/web/protocol-coverage.md | 2 +- packages/agents-client/src/sandbox-client.ts | 4 +- services/agents-api/HOSTED-SANDBOX-MANAGER.md | 4 ++ .../internal/api/sandbox_manager.go | 3 +- .../internal/api/sandbox_manager_test.go | 2 - .../internal/db/queries/runtime_nodes.sql | 2 +- .../db/queries/runtime_suspension.sql | 3 +- .../db/sqlc/initial_environment_files.sql.go | 6 ++- .../agents-api/internal/db/sqlc/models.go | 37 ++++++++++--------- .../db/sqlc/runtime_allocations.sql.go | 24 ++++++++---- .../db/sqlc/runtime_lifecycle_nodes.sql.go | 3 +- .../internal/db/sqlc/runtime_nodes.sql.go | 27 ++++++++------ .../db/sqlc/runtime_suspension.sql.go | 6 ++- .../store/runtime_node_capacity_test.go | 24 +++++++++++- .../internal/store/runtime_node_types.go | 29 ++++++++------- .../internal/store/runtime_nodes.go | 32 ++++++++++++++-- .../internal/store/runtime_suspension_test.go | 27 ++++++++++++++ .../000072_allocation_phase_time.sql | 7 ++++ 21 files changed, 195 insertions(+), 74 deletions(-) create mode 100644 services/agents-api/migrations/000072_allocation_phase_time.sql diff --git a/contracts/agents-api/sandbox-deployment.md b/contracts/agents-api/sandbox-deployment.md index 37d4bfd49..92a40495e 100644 --- a/contracts/agents-api/sandbox-deployment.md +++ b/contracts/agents-api/sandbox-deployment.md @@ -34,11 +34,20 @@ does not grant node enrollment authority. Node capacity is separate from the deployment specification. The administrator's `POST /core/v1/sandbox/enrollment-tokens` accepts optional `max_active` and -`max_retained`, defaulting to 2 and 8. Core stores that approval with the token and -copies it to the registered node. Node enrollment cannot submit capacity overrides. -The existing node configuration/identity reads expose approved limits for that -credential; node updates remain administrator operations. Local host capacity -checks may reject a deployment that cannot run safely, but never raise its limits. +`max_retained`, defaulting to 2 and 8. Microsandbox uses both limits. Docker never +suspends, so its `max_retained` always equals `max_active`; Core replaces any +submitted value, here and in `PATCH /core/v1/sandbox/nodes/{node_id}`. Core stores +that approval with the token and copies it to the registered node. Node enrollment +cannot submit capacity overrides. The existing node configuration/identity reads +expose approved limits for that credential; node updates remain administrator +operations. Local host capacity checks may reject a deployment that cannot run +safely, but never raise its limits. + +`GET /core/v1/sandbox/nodes/{node_id}/allocations` lists the node's unreleased +allocations. Each item's `compute_phase_changed_at` is the time the allocation +entered its current `compute_phase`, or null for allocations created before Core +recorded it. A suspended microsandbox allocation's age, combined with the +deployment's snapshot retention, tells roughly when Core reclaims it. ## Selection request diff --git a/contracts/agents-api/sandbox-manager.openapi.yaml b/contracts/agents-api/sandbox-manager.openapi.yaml index 2f4fd86d9..4753da7f5 100644 --- a/contracts/agents-api/sandbox-manager.openapi.yaml +++ b/contracts/agents-api/sandbox-manager.openapi.yaml @@ -295,6 +295,7 @@ definitions: max_active: type: integer max_retained: + description: Docker never suspends, so Core replaces this with max_active; microsandbox uses both limits. type: integer type: object api.SandboxMutationResponse: @@ -830,6 +831,10 @@ definitions: properties: compute_phase: type: string + compute_phase_changed_at: + description: When the allocation entered its current compute_phase; null for allocations created before it was recorded. For a suspended microsandbox allocation, this time plus the deployment's snapshot retention tells roughly when Core reclaims it. + type: string + x-nullable: true created_at: type: string diagnostic: @@ -929,6 +934,7 @@ definitions: max_active: type: integer max_retained: + description: Docker never suspends, so Core replaces this with max_active; microsandbox uses both limits. type: integer name: type: string diff --git a/docs/configuration.md b/docs/configuration.md index a73dea894..4e4bc1d16 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -131,6 +131,8 @@ approves capacity when issuing its enrollment token: defaults are **2 active** a **8 retained** sandboxes. An administrator may update capacity later through the node API. `max_retained >= max_active >= 1`; reservations and uncertain cleanup also consume capacity. Lowering a limit does not kill existing resources. +Microsandbox uses both limits. Docker never suspends, so its `max_retained` always +equals `max_active`; Core replaces any submitted value. The generated private `provider.json` includes the provider, installation identity, Core address, generation, approved specification copy and one adapter object: diff --git a/docs/web/protocol-coverage.md b/docs/web/protocol-coverage.md index a8ba98f00..70b978c55 100644 --- a/docs/web/protocol-coverage.md +++ b/docs/web/protocol-coverage.md @@ -106,7 +106,7 @@ deployment only. | Node detail | `GET /core/v1/sandbox/nodes/{node_id}?range=1h\|6h\|24h` | Sandbox metrics node dialog: the host's CPU busy share and memory from its last heartbeat, and their history over the page's range | | Allocations | `GET /core/v1/sandbox/nodes/{node_id}/allocations` | Nodes page; Sandbox metrics | | Enrollment | `POST /core/v1/sandbox/enrollment-tokens` | **Add node**: the administrator sets the node's sandbox limits (`max_active`; `max_retained` only for microsandbox, equal to `max_active` for Docker) before Core issues a single-use token inside a command that verifies the installer checksum | -| Update node | `PATCH /core/v1/sandbox/nodes/{node_id}` | **Edit node**: the name and sandbox limits together (the retained limit only for microsandbox; Docker keeps its saved one, raised to at least the active limit) | +| Update node | `PATCH /core/v1/sandbox/nodes/{node_id}` | **Edit node**: the name and sandbox limits together (the retained limit only for microsandbox; under Docker, Core sets it to the active limit) | | Remove node | `DELETE /core/v1/sandbox/nodes/{node_id}` | Confirmed node removal; the row goes only after Core acknowledges the deletion | These pages appear only when `/console/config` reports `sandbox_admin: true`. An E2B diff --git a/packages/agents-client/src/sandbox-client.ts b/packages/agents-client/src/sandbox-client.ts index 039464288..733736ffb 100644 --- a/packages/agents-client/src/sandbox-client.ts +++ b/packages/agents-client/src/sandbox-client.ts @@ -93,7 +93,7 @@ export interface SandboxNodeDetail extends SandboxNode { host: SandboxNodeHost; history: { resolution_seconds: number; points: SandboxNodeHostPoint[] }; } -/** A node's name and sandbox limits; Core takes all three, with a retained limit of at least the active one. */ +/** A node's name and sandbox limits; Core takes all three, with a retained limit of at least the active one. Under Docker, Core sets the retained limit to the active one. */ export interface SandboxNodeUpdate { name: string; max_active: number; @@ -107,6 +107,8 @@ export interface SandboxAllocation { environment_id: string; state: string; compute_phase: string; + /** When the allocation entered compute_phase; null when it predates this record. */ + compute_phase_changed_at: string | null; diagnostic: SandboxDiagnostic; initialization: string; created_at: string; diff --git a/services/agents-api/HOSTED-SANDBOX-MANAGER.md b/services/agents-api/HOSTED-SANDBOX-MANAGER.md index 32c268aef..7d8180b9f 100644 --- a/services/agents-api/HOSTED-SANDBOX-MANAGER.md +++ b/services/agents-api/HOSTED-SANDBOX-MANAGER.md @@ -57,6 +57,10 @@ readiness. To prepare the qualified E2B Runtime build, use [the E2B build guide](deploy/e2b/README.md); it is not a public Environment Template. Microsandbox suspends eligible idle Sessions after 300 seconds and retains their snapshots for 86400 seconds. Docker and E2B have no memory snapshot policy. +A node's allocation list reports `compute_phase_changed_at`, the time each +allocation entered its current `compute_phase`; it is null for allocations created +before Core recorded it. A suspended allocation's age, combined with this retention, +tells roughly when Core reclaims it. For source/manual deployments using E2B, install the packaged helper and set `AGENTS_API_E2B_PROVIDER_BIN` to its absolute executable path. Set diff --git a/services/agents-api/internal/api/sandbox_manager.go b/services/agents-api/internal/api/sandbox_manager.go index e251dea62..47b69ecbd 100644 --- a/services/agents-api/internal/api/sandbox_manager.go +++ b/services/agents-api/internal/api/sandbox_manager.go @@ -24,7 +24,8 @@ type SandboxMutationResponse struct { Updated bool `json:"updated,omitempty"` } type SandboxEnrollmentTokenRequest struct { - MaxActive *int `json:"max_active,omitempty"` + MaxActive *int `json:"max_active,omitempty"` + // Docker never suspends, so Core replaces this with max_active; microsandbox uses both limits. MaxRetained *int `json:"max_retained,omitempty"` } diff --git a/services/agents-api/internal/api/sandbox_manager_test.go b/services/agents-api/internal/api/sandbox_manager_test.go index 69126a6b3..eed1b2bd4 100644 --- a/services/agents-api/internal/api/sandbox_manager_test.go +++ b/services/agents-api/internal/api/sandbox_manager_test.go @@ -88,8 +88,6 @@ func TestSandboxEnrollmentCapacityIsAdministratorOnly(t *testing.T) { } for _, test := range []struct{ path, token, body string }{ {"/core/v1/sandbox/enrollment-tokens", "administrator", `{"max_active":0}`}, - {"/core/v1/sandbox/enrollment-tokens", "administrator", `{"max_active":9,"max_retained":8}`}, - {"/core/v1/sandbox/enrollment-tokens", "administrator", `{"max_retained":1000001}`}, {"/api/v1/sandbox-node/enroll", "one-use", `{"max_active":100}`}, {"/api/v1/sandbox-node/enroll", "one-use", `{"max_retained":100}`}, } { diff --git a/services/agents-api/internal/db/queries/runtime_nodes.sql b/services/agents-api/internal/db/queries/runtime_nodes.sql index ec0fa27df..4447bf0cb 100644 --- a/services/agents-api/internal/db/queries/runtime_nodes.sql +++ b/services/agents-api/internal/db/queries/runtime_nodes.sql @@ -90,7 +90,7 @@ UPDATE runtime_allocations SET node_id=$1,compute_activity_at=clock_timestamp() WHERE provider_key=$2 AND node_id IS NULL AND state<>'released'; -- name: ListNodeRuntimeAllocations :many -SELECT a.id,a.node_id,a.observation_error,a.state,a.compute_phase,a.initialization,a.created_at,a.environment_id,e.session_id,s.tenant_id +SELECT a.id,a.node_id,a.observation_error,a.state,a.compute_phase,a.compute_phase_changed_at,a.initialization,a.created_at,a.environment_id,e.session_id,s.tenant_id FROM runtime_allocations a JOIN environments e ON e.id=a.environment_id JOIN sessions s ON s.id=e.session_id WHERE a.node_id=$1 AND a.state<>'released' ORDER BY a.created_at,a.id LIMIT 1000; diff --git a/services/agents-api/internal/db/queries/runtime_suspension.sql b/services/agents-api/internal/db/queries/runtime_suspension.sql index fb5a32403..5c137172d 100644 --- a/services/agents-api/internal/db/queries/runtime_suspension.sql +++ b/services/agents-api/internal/db/queries/runtime_suspension.sql @@ -1,6 +1,7 @@ -- name: SetRuntimeCompute :one UPDATE runtime_allocations -SET compute_phase = sqlc.arg(phase), compute_state = sqlc.arg(state)::jsonb, +SET compute_phase_changed_at = CASE WHEN compute_phase = sqlc.arg(phase)::text THEN compute_phase_changed_at ELSE clock_timestamp() END, + compute_phase = sqlc.arg(phase), compute_state = sqlc.arg(state)::jsonb, compute_revision = compute_revision + 1, compute_retained_until = sqlc.narg(retained_until), kept_at = CASE WHEN sqlc.arg(phase)::text = 'running' THEN clock_timestamp() ELSE kept_at END diff --git a/services/agents-api/internal/db/sqlc/initial_environment_files.sql.go b/services/agents-api/internal/db/sqlc/initial_environment_files.sql.go index 2f4068192..52a9ddea9 100644 --- a/services/agents-api/internal/db/sqlc/initial_environment_files.sql.go +++ b/services/agents-api/internal/db/sqlc/initial_environment_files.sql.go @@ -15,7 +15,7 @@ const claimRuntimeInitialization = `-- name: ClaimRuntimeInitialization :one UPDATE runtime_allocations SET initialization = 'running' WHERE id = $1 AND initialization = 'pending' AND state = 'running' AND create_settled AND (node_id IS NOT NULL OR (SELECT mode FROM runtime_deployment) = 'direct' OR kept_at > clock_timestamp() - interval '1 hour') -RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error +RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error, compute_phase_changed_at ` func (q *Queries) ClaimRuntimeInitialization(ctx context.Context, id pgtype.UUID) (RuntimeAllocation, error) { @@ -40,6 +40,7 @@ func (q *Queries) ClaimRuntimeInitialization(ctx context.Context, id pgtype.UUID &i.ComputeRetainedUntil, &i.NodeID, &i.ObservationError, + &i.ComputePhaseChangedAt, ) return i, err } @@ -48,7 +49,7 @@ const completeRuntimeInitialization = `-- name: CompleteRuntimeInitialization :o UPDATE runtime_allocations SET initialization = 'complete' WHERE id = $1 AND initialization = 'running' AND state = 'running' AND create_settled AND (node_id IS NOT NULL OR (SELECT mode FROM runtime_deployment) = 'direct' OR kept_at > clock_timestamp() - interval '1 hour') -RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error +RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error, compute_phase_changed_at ` func (q *Queries) CompleteRuntimeInitialization(ctx context.Context, id pgtype.UUID) (RuntimeAllocation, error) { @@ -73,6 +74,7 @@ func (q *Queries) CompleteRuntimeInitialization(ctx context.Context, id pgtype.U &i.ComputeRetainedUntil, &i.NodeID, &i.ObservationError, + &i.ComputePhaseChangedAt, ) return i, err } diff --git a/services/agents-api/internal/db/sqlc/models.go b/services/agents-api/internal/db/sqlc/models.go index 8148710c3..54b6a59a4 100644 --- a/services/agents-api/internal/db/sqlc/models.go +++ b/services/agents-api/internal/db/sqlc/models.go @@ -210,24 +210,25 @@ type PublicExecutionTurn struct { } type RuntimeAllocation struct { - ID pgtype.UUID `json:"id"` - EnvironmentID pgtype.UUID `json:"environment_id"` - DeviceID pgtype.UUID `json:"device_id"` - ProviderKey pgtype.UUID `json:"provider_key"` - State string `json:"state"` - CreateSettled bool `json:"create_settled"` - CreatedAt pgtype.Timestamptz `json:"created_at"` - KeptAt pgtype.Timestamptz `json:"kept_at"` - ReleasedAt pgtype.Timestamptz `json:"released_at"` - Initialization string `json:"initialization"` - ComputePhase string `json:"compute_phase"` - ComputeRevision int64 `json:"compute_revision"` - ComputeState []byte `json:"compute_state"` - ComputeActivityAt pgtype.Timestamptz `json:"compute_activity_at"` - ComputeWakeRequested bool `json:"compute_wake_requested"` - ComputeRetainedUntil pgtype.Timestamptz `json:"compute_retained_until"` - NodeID pgtype.UUID `json:"node_id"` - ObservationError string `json:"observation_error"` + ID pgtype.UUID `json:"id"` + EnvironmentID pgtype.UUID `json:"environment_id"` + DeviceID pgtype.UUID `json:"device_id"` + ProviderKey pgtype.UUID `json:"provider_key"` + State string `json:"state"` + CreateSettled bool `json:"create_settled"` + CreatedAt pgtype.Timestamptz `json:"created_at"` + KeptAt pgtype.Timestamptz `json:"kept_at"` + ReleasedAt pgtype.Timestamptz `json:"released_at"` + Initialization string `json:"initialization"` + ComputePhase string `json:"compute_phase"` + ComputeRevision int64 `json:"compute_revision"` + ComputeState []byte `json:"compute_state"` + ComputeActivityAt pgtype.Timestamptz `json:"compute_activity_at"` + ComputeWakeRequested bool `json:"compute_wake_requested"` + ComputeRetainedUntil pgtype.Timestamptz `json:"compute_retained_until"` + NodeID pgtype.UUID `json:"node_id"` + ObservationError string `json:"observation_error"` + ComputePhaseChangedAt pgtype.Timestamptz `json:"compute_phase_changed_at"` } type RuntimeDeployment struct { diff --git a/services/agents-api/internal/db/sqlc/runtime_allocations.sql.go b/services/agents-api/internal/db/sqlc/runtime_allocations.sql.go index a21fd6227..2781ccf9d 100644 --- a/services/agents-api/internal/db/sqlc/runtime_allocations.sql.go +++ b/services/agents-api/internal/db/sqlc/runtime_allocations.sql.go @@ -13,7 +13,7 @@ import ( const createRuntimeAllocation = `-- name: CreateRuntimeAllocation :one INSERT INTO runtime_allocations (id, environment_id, device_id, provider_key, node_id, initialization) -VALUES ($1, $2, $3, $4, $5, CASE WHEN EXISTS (SELECT 1 FROM initial_environment_files f JOIN environments e ON e.session_id = f.session_id WHERE e.id = $2) OR EXISTS (SELECT 1 FROM environment_setups f JOIN environments e ON e.session_id = f.session_id WHERE e.id = $2) THEN 'pending' ELSE 'complete' END) RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error +VALUES ($1, $2, $3, $4, $5, CASE WHEN EXISTS (SELECT 1 FROM initial_environment_files f JOIN environments e ON e.session_id = f.session_id WHERE e.id = $2) OR EXISTS (SELECT 1 FROM environment_setups f JOIN environments e ON e.session_id = f.session_id WHERE e.id = $2) THEN 'pending' ELSE 'complete' END) RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error, compute_phase_changed_at ` type CreateRuntimeAllocationParams struct { @@ -52,12 +52,13 @@ func (q *Queries) CreateRuntimeAllocation(ctx context.Context, arg CreateRuntime &i.ComputeRetainedUntil, &i.NodeID, &i.ObservationError, + &i.ComputePhaseChangedAt, ) return i, err } const getRuntimeAllocation = `-- name: GetRuntimeAllocation :one -SELECT a.id, a.environment_id, a.device_id, a.provider_key, a.state, a.create_settled, a.created_at, a.kept_at, a.released_at, a.initialization, a.compute_phase, a.compute_revision, a.compute_state, a.compute_activity_at, a.compute_wake_requested, a.compute_retained_until, a.node_id, a.observation_error, e.session_id, s.tenant_id, s.deleted_at, (CASE WHEN a.compute_phase NOT IN ('disabled', 'running') THEN a.compute_retained_until IS NOT NULL AND a.compute_retained_until <= clock_timestamp() ELSE a.node_id IS NULL AND (SELECT mode FROM runtime_deployment) <> 'direct' AND a.kept_at <= clock_timestamp() - interval '1 hour' END)::boolean AS expired +SELECT a.id, a.environment_id, a.device_id, a.provider_key, a.state, a.create_settled, a.created_at, a.kept_at, a.released_at, a.initialization, a.compute_phase, a.compute_revision, a.compute_state, a.compute_activity_at, a.compute_wake_requested, a.compute_retained_until, a.node_id, a.observation_error, a.compute_phase_changed_at, e.session_id, s.tenant_id, s.deleted_at, (CASE WHEN a.compute_phase NOT IN ('disabled', 'running') THEN a.compute_retained_until IS NOT NULL AND a.compute_retained_until <= clock_timestamp() ELSE a.node_id IS NULL AND (SELECT mode FROM runtime_deployment) <> 'direct' AND a.kept_at <= clock_timestamp() - interval '1 hour' END)::boolean AS expired FROM runtime_allocations a JOIN environments e ON e.id = a.environment_id JOIN sessions s ON s.id = e.session_id @@ -99,6 +100,7 @@ func (q *Queries) GetRuntimeAllocation(ctx context.Context, arg GetRuntimeAlloca &i.RuntimeAllocation.ComputeRetainedUntil, &i.RuntimeAllocation.NodeID, &i.RuntimeAllocation.ObservationError, + &i.RuntimeAllocation.ComputePhaseChangedAt, &i.SessionID, &i.TenantID, &i.DeletedAt, @@ -111,7 +113,7 @@ const keepRuntimeAllocation = `-- name: KeepRuntimeAllocation :one UPDATE runtime_allocations SET kept_at = clock_timestamp() WHERE id = $1 AND state = 'running' AND (node_id IS NOT NULL OR (SELECT mode FROM runtime_deployment) = 'direct' OR kept_at > clock_timestamp() - interval '1 hour') -RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error +RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error, compute_phase_changed_at ` func (q *Queries) KeepRuntimeAllocation(ctx context.Context, id pgtype.UUID) (RuntimeAllocation, error) { @@ -136,12 +138,13 @@ func (q *Queries) KeepRuntimeAllocation(ctx context.Context, id pgtype.UUID) (Ru &i.ComputeRetainedUntil, &i.NodeID, &i.ObservationError, + &i.ComputePhaseChangedAt, ) return i, err } const listRuntimeAllocations = `-- name: ListRuntimeAllocations :many -SELECT a.id, a.environment_id, a.device_id, a.provider_key, a.state, a.create_settled, a.created_at, a.kept_at, a.released_at, a.initialization, a.compute_phase, a.compute_revision, a.compute_state, a.compute_activity_at, a.compute_wake_requested, a.compute_retained_until, a.node_id, a.observation_error, e.session_id, s.tenant_id, s.deleted_at, (CASE WHEN a.compute_phase NOT IN ('disabled', 'running') THEN a.compute_retained_until IS NOT NULL AND a.compute_retained_until <= clock_timestamp() ELSE a.node_id IS NULL AND (SELECT mode FROM runtime_deployment) <> 'direct' AND a.kept_at <= clock_timestamp() - interval '1 hour' END)::boolean AS expired +SELECT a.id, a.environment_id, a.device_id, a.provider_key, a.state, a.create_settled, a.created_at, a.kept_at, a.released_at, a.initialization, a.compute_phase, a.compute_revision, a.compute_state, a.compute_activity_at, a.compute_wake_requested, a.compute_retained_until, a.node_id, a.observation_error, a.compute_phase_changed_at, e.session_id, s.tenant_id, s.deleted_at, (CASE WHEN a.compute_phase NOT IN ('disabled', 'running') THEN a.compute_retained_until IS NOT NULL AND a.compute_retained_until <= clock_timestamp() ELSE a.node_id IS NULL AND (SELECT mode FROM runtime_deployment) <> 'direct' AND a.kept_at <= clock_timestamp() - interval '1 hour' END)::boolean AS expired FROM runtime_allocations a JOIN environments e ON e.id = a.environment_id JOIN sessions s ON s.id = e.session_id @@ -185,6 +188,7 @@ func (q *Queries) ListRuntimeAllocations(ctx context.Context, id pgtype.UUID) ([ &i.RuntimeAllocation.ComputeRetainedUntil, &i.RuntimeAllocation.NodeID, &i.RuntimeAllocation.ObservationError, + &i.RuntimeAllocation.ComputePhaseChangedAt, &i.SessionID, &i.TenantID, &i.DeletedAt, @@ -281,7 +285,7 @@ const observeRuntimeRunning = `-- name: ObserveRuntimeRunning :one UPDATE runtime_allocations SET state = 'running', create_settled = true WHERE id = $1 AND state IN ('creating', 'running') AND (node_id IS NOT NULL OR (SELECT mode FROM runtime_deployment) = 'direct' OR kept_at > clock_timestamp() - interval '1 hour') -RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error +RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error, compute_phase_changed_at ` func (q *Queries) ObserveRuntimeRunning(ctx context.Context, id pgtype.UUID) (RuntimeAllocation, error) { @@ -306,13 +310,14 @@ func (q *Queries) ObserveRuntimeRunning(ctx context.Context, id pgtype.UUID) (Ru &i.ComputeRetainedUntil, &i.NodeID, &i.ObservationError, + &i.ComputePhaseChangedAt, ) return i, err } const releaseRuntimeAllocation = `-- name: ReleaseRuntimeAllocation :one UPDATE runtime_allocations SET state = 'released', released_at = clock_timestamp() -WHERE id = $1 AND state = 'cleanup_pending' AND create_settled RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error +WHERE id = $1 AND state = 'cleanup_pending' AND create_settled RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error, compute_phase_changed_at ` func (q *Queries) ReleaseRuntimeAllocation(ctx context.Context, id pgtype.UUID) (RuntimeAllocation, error) { @@ -337,13 +342,14 @@ func (q *Queries) ReleaseRuntimeAllocation(ctx context.Context, id pgtype.UUID) &i.ComputeRetainedUntil, &i.NodeID, &i.ObservationError, + &i.ComputePhaseChangedAt, ) return i, err } const requestRuntimeCleanup = `-- name: RequestRuntimeCleanup :one UPDATE runtime_allocations SET state = 'cleanup_pending' -WHERE id = $1 AND state <> 'released' RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error +WHERE id = $1 AND state <> 'released' RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error, compute_phase_changed_at ` func (q *Queries) RequestRuntimeCleanup(ctx context.Context, id pgtype.UUID) (RuntimeAllocation, error) { @@ -368,13 +374,14 @@ func (q *Queries) RequestRuntimeCleanup(ctx context.Context, id pgtype.UUID) (Ru &i.ComputeRetainedUntil, &i.NodeID, &i.ObservationError, + &i.ComputePhaseChangedAt, ) return i, err } const settleRuntimeCreation = `-- name: SettleRuntimeCreation :one UPDATE runtime_allocations SET create_settled = true -WHERE id = $1 AND state <> 'released' RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error +WHERE id = $1 AND state <> 'released' RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error, compute_phase_changed_at ` func (q *Queries) SettleRuntimeCreation(ctx context.Context, id pgtype.UUID) (RuntimeAllocation, error) { @@ -399,6 +406,7 @@ func (q *Queries) SettleRuntimeCreation(ctx context.Context, id pgtype.UUID) (Ru &i.ComputeRetainedUntil, &i.NodeID, &i.ObservationError, + &i.ComputePhaseChangedAt, ) return i, err } diff --git a/services/agents-api/internal/db/sqlc/runtime_lifecycle_nodes.sql.go b/services/agents-api/internal/db/sqlc/runtime_lifecycle_nodes.sql.go index 2bf9d7510..a96e83b53 100644 --- a/services/agents-api/internal/db/sqlc/runtime_lifecycle_nodes.sql.go +++ b/services/agents-api/internal/db/sqlc/runtime_lifecycle_nodes.sql.go @@ -50,7 +50,7 @@ func (q *Queries) GetRuntimeLifecyclePlacement(ctx context.Context, arg GetRunti } const listRuntimeAllocationsForNode = `-- name: ListRuntimeAllocationsForNode :many -SELECT a.id, a.environment_id, a.device_id, a.provider_key, a.state, a.create_settled, a.created_at, a.kept_at, a.released_at, a.initialization, a.compute_phase, a.compute_revision, a.compute_state, a.compute_activity_at, a.compute_wake_requested, a.compute_retained_until, a.node_id, a.observation_error, e.session_id, s.tenant_id, s.deleted_at, (CASE WHEN a.compute_phase NOT IN ('disabled', 'running') THEN a.compute_retained_until IS NOT NULL AND a.compute_retained_until <= clock_timestamp() ELSE a.node_id IS NULL AND (SELECT mode FROM runtime_deployment) <> 'direct' AND a.kept_at <= clock_timestamp() - interval '1 hour' END)::boolean AS expired +SELECT a.id, a.environment_id, a.device_id, a.provider_key, a.state, a.create_settled, a.created_at, a.kept_at, a.released_at, a.initialization, a.compute_phase, a.compute_revision, a.compute_state, a.compute_activity_at, a.compute_wake_requested, a.compute_retained_until, a.node_id, a.observation_error, a.compute_phase_changed_at, e.session_id, s.tenant_id, s.deleted_at, (CASE WHEN a.compute_phase NOT IN ('disabled', 'running') THEN a.compute_retained_until IS NOT NULL AND a.compute_retained_until <= clock_timestamp() ELSE a.node_id IS NULL AND (SELECT mode FROM runtime_deployment) <> 'direct' AND a.kept_at <= clock_timestamp() - interval '1 hour' END)::boolean AS expired FROM runtime_allocations a JOIN environments e ON e.id=a.environment_id JOIN sessions s ON s.id=e.session_id @@ -100,6 +100,7 @@ func (q *Queries) ListRuntimeAllocationsForNode(ctx context.Context, arg ListRun &i.RuntimeAllocation.ComputeRetainedUntil, &i.RuntimeAllocation.NodeID, &i.RuntimeAllocation.ObservationError, + &i.RuntimeAllocation.ComputePhaseChangedAt, &i.SessionID, &i.TenantID, &i.DeletedAt, diff --git a/services/agents-api/internal/db/sqlc/runtime_nodes.sql.go b/services/agents-api/internal/db/sqlc/runtime_nodes.sql.go index d0957c530..84bd5d9d1 100644 --- a/services/agents-api/internal/db/sqlc/runtime_nodes.sql.go +++ b/services/agents-api/internal/db/sqlc/runtime_nodes.sql.go @@ -354,7 +354,7 @@ func (q *Queries) InsertRuntimeNode(ctx context.Context, arg InsertRuntimeNodePa } const listLegacyRuntimeAllocations = `-- name: ListLegacyRuntimeAllocations :many -SELECT a.id, a.environment_id, a.device_id, a.provider_key, a.state, a.create_settled, a.created_at, a.kept_at, a.released_at, a.initialization, a.compute_phase, a.compute_revision, a.compute_state, a.compute_activity_at, a.compute_wake_requested, a.compute_retained_until, a.node_id, a.observation_error, e.session_id, s.tenant_id, s.deleted_at +SELECT a.id, a.environment_id, a.device_id, a.provider_key, a.state, a.create_settled, a.created_at, a.kept_at, a.released_at, a.initialization, a.compute_phase, a.compute_revision, a.compute_state, a.compute_activity_at, a.compute_wake_requested, a.compute_retained_until, a.node_id, a.observation_error, a.compute_phase_changed_at, e.session_id, s.tenant_id, s.deleted_at FROM runtime_allocations a JOIN environments e ON e.id=a.environment_id JOIN sessions s ON s.id=e.session_id WHERE a.node_id IS NULL AND a.state<>'released' AND a.id>$1 ORDER BY a.id LIMIT 32 FOR UPDATE OF a ` @@ -394,6 +394,7 @@ func (q *Queries) ListLegacyRuntimeAllocations(ctx context.Context, id pgtype.UU &i.RuntimeAllocation.ComputeRetainedUntil, &i.RuntimeAllocation.NodeID, &i.RuntimeAllocation.ObservationError, + &i.RuntimeAllocation.ComputePhaseChangedAt, &i.SessionID, &i.TenantID, &i.DeletedAt, @@ -409,22 +410,23 @@ func (q *Queries) ListLegacyRuntimeAllocations(ctx context.Context, id pgtype.UU } const listNodeRuntimeAllocations = `-- name: ListNodeRuntimeAllocations :many -SELECT a.id,a.node_id,a.observation_error,a.state,a.compute_phase,a.initialization,a.created_at,a.environment_id,e.session_id,s.tenant_id +SELECT a.id,a.node_id,a.observation_error,a.state,a.compute_phase,a.compute_phase_changed_at,a.initialization,a.created_at,a.environment_id,e.session_id,s.tenant_id FROM runtime_allocations a JOIN environments e ON e.id=a.environment_id JOIN sessions s ON s.id=e.session_id WHERE a.node_id=$1 AND a.state<>'released' ORDER BY a.created_at,a.id LIMIT 1000 ` type ListNodeRuntimeAllocationsRow struct { - ID pgtype.UUID `json:"id"` - NodeID pgtype.UUID `json:"node_id"` - ObservationError string `json:"observation_error"` - State string `json:"state"` - ComputePhase string `json:"compute_phase"` - Initialization string `json:"initialization"` - CreatedAt pgtype.Timestamptz `json:"created_at"` - EnvironmentID pgtype.UUID `json:"environment_id"` - SessionID pgtype.UUID `json:"session_id"` - TenantID pgtype.UUID `json:"tenant_id"` + ID pgtype.UUID `json:"id"` + NodeID pgtype.UUID `json:"node_id"` + ObservationError string `json:"observation_error"` + State string `json:"state"` + ComputePhase string `json:"compute_phase"` + ComputePhaseChangedAt pgtype.Timestamptz `json:"compute_phase_changed_at"` + Initialization string `json:"initialization"` + CreatedAt pgtype.Timestamptz `json:"created_at"` + EnvironmentID pgtype.UUID `json:"environment_id"` + SessionID pgtype.UUID `json:"session_id"` + TenantID pgtype.UUID `json:"tenant_id"` } func (q *Queries) ListNodeRuntimeAllocations(ctx context.Context, nodeID pgtype.UUID) ([]ListNodeRuntimeAllocationsRow, error) { @@ -442,6 +444,7 @@ func (q *Queries) ListNodeRuntimeAllocations(ctx context.Context, nodeID pgtype. &i.ObservationError, &i.State, &i.ComputePhase, + &i.ComputePhaseChangedAt, &i.Initialization, &i.CreatedAt, &i.EnvironmentID, diff --git a/services/agents-api/internal/db/sqlc/runtime_suspension.sql.go b/services/agents-api/internal/db/sqlc/runtime_suspension.sql.go index 1b1c73b0f..f423b3d2d 100644 --- a/services/agents-api/internal/db/sqlc/runtime_suspension.sql.go +++ b/services/agents-api/internal/db/sqlc/runtime_suspension.sql.go @@ -130,7 +130,8 @@ func (q *Queries) SessionHasRuntimeNode(ctx context.Context, sessionID pgtype.UU const setRuntimeCompute = `-- name: SetRuntimeCompute :one UPDATE runtime_allocations -SET compute_phase = $1, compute_state = $2::jsonb, +SET compute_phase_changed_at = CASE WHEN compute_phase = $1::text THEN compute_phase_changed_at ELSE clock_timestamp() END, + compute_phase = $1, compute_state = $2::jsonb, compute_revision = compute_revision + 1, compute_retained_until = $3, kept_at = CASE WHEN $1::text = 'running' THEN clock_timestamp() ELSE kept_at END @@ -138,7 +139,7 @@ WHERE id = $4 AND compute_revision = $5 AND state = 'running' AND initialization = 'complete' AND ((compute_phase IN ('disabled','running') AND (node_id IS NOT NULL OR (SELECT mode FROM runtime_deployment) = 'direct' OR kept_at > clock_timestamp() - interval '1 hour')) OR (compute_phase NOT IN ('disabled','running') AND compute_retained_until > clock_timestamp())) -RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error +RETURNING id, environment_id, device_id, provider_key, state, create_settled, created_at, kept_at, released_at, initialization, compute_phase, compute_revision, compute_state, compute_activity_at, compute_wake_requested, compute_retained_until, node_id, observation_error, compute_phase_changed_at ` type SetRuntimeComputeParams struct { @@ -177,6 +178,7 @@ func (q *Queries) SetRuntimeCompute(ctx context.Context, arg SetRuntimeComputePa &i.ComputeRetainedUntil, &i.NodeID, &i.ObservationError, + &i.ComputePhaseChangedAt, ) return i, err } diff --git a/services/agents-api/internal/store/runtime_node_capacity_test.go b/services/agents-api/internal/store/runtime_node_capacity_test.go index e5052a1bb..f5f6373b1 100644 --- a/services/agents-api/internal/store/runtime_node_capacity_test.go +++ b/services/agents-api/internal/store/runtime_node_capacity_test.go @@ -9,7 +9,7 @@ import ( ) func TestRuntimeEnrollmentApprovedCapacity(t *testing.T) { - s, _, view, _ := webSpecificationFixture(t, "docker") + s, _, view, _ := webSpecificationFixture(t, "microsandbox") for _, capacity := range []RuntimeNodeCapacity{{0, 8}, {3, 2}, {1, 1000001}} { if _, _, err := s.CreateRuntimeEnrollment(t.Context(), capacity); !errors.Is(err, ErrInvalidInput) { t.Fatal("invalid capacity accepted", err) @@ -23,7 +23,7 @@ func TestRuntimeEnrollmentApprovedCapacity(t *testing.T) { if err != nil || config.MaxActive != 1 || config.MaxRetained != 3 { t.Fatal("bootstrap lost approved capacity", config, err) } - input := RuntimeNodeEnrollment{NodeID: uuid.NewString(), Name: "approved", Credential: strings.Repeat("a", 64), Provider: "docker", BackendFingerprint: strings.Repeat("b", 64), SpecificationDigest: view.SpecificationDigest, DeploymentGeneration: view.Generation} + input := RuntimeNodeEnrollment{NodeID: uuid.NewString(), Name: "approved", Credential: strings.Repeat("a", 64), Provider: "microsandbox", BackendFingerprint: strings.Repeat("b", 64), SpecificationDigest: view.SpecificationDigest, DeploymentGeneration: view.Generation} identity, err := s.EnrollRuntimeNode(t.Context(), token, input) if err != nil || identity.MaxActive != 1 || identity.MaxRetained != 3 { t.Fatal("enrollment did not apply token capacity", identity, err) @@ -43,3 +43,23 @@ func TestRuntimeEnrollmentApprovedCapacity(t *testing.T) { t.Fatal("configuration read ignored admin update", config, err) } } + +// Docker never suspends a sandbox, so its retained limit always equals its active limit. +func TestDockerRetainedLimitFollowsActive(t *testing.T) { + s, _, view, _ := webSpecificationFixture(t, "docker") + token, _, err := s.CreateRuntimeEnrollment(t.Context(), RuntimeNodeCapacity{MaxActive: 3, MaxRetained: 1}) + if err != nil { + t.Fatal(err) + } + input := RuntimeNodeEnrollment{NodeID: uuid.NewString(), Name: "Docker", Credential: strings.Repeat("d", 64), Provider: "docker", BackendFingerprint: strings.Repeat("b", 64), SpecificationDigest: view.SpecificationDigest, DeploymentGeneration: view.Generation} + if identity, err := s.EnrollRuntimeNode(t.Context(), token, input); err != nil || identity.MaxRetained != 3 { + t.Fatal("enrollment kept a separate Docker retained limit", identity, err) + } + if err := s.UpdateRuntimeNode(t.Context(), input.NodeID, RuntimeNodeUpdate{Name: "Docker", MaxActive: 5, MaxRetained: 12}); err != nil { + t.Fatal(err) + } + nodes, err := s.ListRuntimeNodes(t.Context()) + if err != nil || len(nodes) != 1 || nodes[0].MaxActive != 5 || nodes[0].MaxRetained != 5 { + t.Fatal("node update kept a separate Docker retained limit", nodes, err) + } +} diff --git a/services/agents-api/internal/store/runtime_node_types.go b/services/agents-api/internal/store/runtime_node_types.go index dc7bc53de..ba4363508 100644 --- a/services/agents-api/internal/store/runtime_node_types.go +++ b/services/agents-api/internal/store/runtime_node_types.go @@ -64,9 +64,10 @@ type RuntimeNode struct { CreatedAt time.Time `json:"created_at"` } type RuntimeNodeUpdate struct { - Name string `json:"name"` - MaxActive int `json:"max_active"` - MaxRetained int `json:"max_retained"` + Name string `json:"name"` + MaxActive int `json:"max_active"` + // Docker never suspends, so Core replaces this with max_active; microsandbox uses both limits. + MaxRetained int `json:"max_retained"` } type RuntimePlacement struct { Diagnostic string `json:"diagnostic"` @@ -129,14 +130,16 @@ type RuntimeDeploymentView struct { OwnerEpoch uint64 `json:"owner_epoch"` } type RuntimeNodeAllocation struct { - Diagnostic string `json:"diagnostic"` - ID string `json:"id"` - NodeID string `json:"node_id"` - TenantID string `json:"tenant_id"` - SessionID string `json:"session_id"` - EnvironmentID string `json:"environment_id"` - State string `json:"state"` - ComputePhase string `json:"compute_phase"` - Initialization string `json:"initialization"` - CreatedAt time.Time `json:"created_at"` + Diagnostic string `json:"diagnostic"` + ID string `json:"id"` + NodeID string `json:"node_id"` + TenantID string `json:"tenant_id"` + SessionID string `json:"session_id"` + EnvironmentID string `json:"environment_id"` + State string `json:"state"` + ComputePhase string `json:"compute_phase"` + // When the allocation entered its current compute_phase; null for allocations created before it was recorded. For a suspended microsandbox allocation, this time plus the deployment's snapshot retention tells roughly when Core reclaims it. + ComputePhaseChangedAt *time.Time `json:"compute_phase_changed_at" extensions:"x-nullable"` + Initialization string `json:"initialization"` + CreatedAt time.Time `json:"created_at"` } diff --git a/services/agents-api/internal/store/runtime_nodes.go b/services/agents-api/internal/store/runtime_nodes.go index cd5050baf..e8432834d 100644 --- a/services/agents-api/internal/store/runtime_nodes.go +++ b/services/agents-api/internal/store/runtime_nodes.go @@ -24,6 +24,15 @@ func validRuntimeDigest(value string) bool { decoded, err := hex.DecodeString(value) return err == nil && len(decoded) == 32 && hex.EncodeToString(decoded) == value } + +// retainedLimit reports the retained-sandbox limit a provider can use. Docker +// never suspends a sandbox, so its retained limit always equals its active limit. +func retainedLimit(provider string, active, retained int) int { + if provider == "docker" { + return active + } + return retained +} func validateRuntimeNode(name string, active, retained int) error { if strings.TrimSpace(name) == "" || len(name) > 128 || strings.ContainsAny(name, "\x00\r\n") || active < 1 || retained < active || retained > 1000000 { return ErrInvalidInput @@ -90,12 +99,13 @@ func runtimeNodeViews(rows []sqlc.ListRuntimeNodesRow) ([]RuntimeNode, error) { return nil, err } health.ProviderReady = n.ProviderReady - out = append(out, RuntimeNode{RuntimeNodeHealth: health, Running: n.Running, Snapshots: n.Snapshots, ID: runtimeUUID(n.ID), Name: n.Name, Provider: n.ProviderKind, Online: n.Online, LastSeenAt: seen, MaxActive: int(n.MaxActive), MaxRetained: int(n.MaxRetained), Active: n.Active, Reserved: n.Reserved, Retained: n.Retained, CleanupPending: n.CleanupPending, CreatedAt: n.CreatedAt.Time}) + out = append(out, RuntimeNode{RuntimeNodeHealth: health, Running: n.Running, Snapshots: n.Snapshots, ID: runtimeUUID(n.ID), Name: n.Name, Provider: n.ProviderKind, Online: n.Online, LastSeenAt: seen, MaxActive: int(n.MaxActive), MaxRetained: retainedLimit(n.ProviderKind, int(n.MaxActive), int(n.MaxRetained)), Active: n.Active, Reserved: n.Reserved, Retained: n.Retained, CleanupPending: n.CleanupPending, CreatedAt: n.CreatedAt.Time}) } return out, nil } func (s *Store) CreateRuntimeEnrollment(ctx context.Context, capacity RuntimeNodeCapacity) (string, time.Time, error) { - if err := validateRuntimeNode("enrollment", capacity.MaxActive, capacity.MaxRetained); err != nil { + // The retained limit depends on the provider, so the transaction checks it. + if err := validateRuntimeNode("enrollment", capacity.MaxActive, capacity.MaxActive); err != nil { return "", time.Time{}, err } var bytes [32]byte @@ -108,6 +118,10 @@ func (s *Store) CreateRuntimeEnrollment(ctx context.Context, capacity RuntimeNod if d.Mode != "nodes" || d.Maintenance || unspecifiedNodeDeployment(d) { return ErrSandboxDeploymentConflict } + capacity.MaxRetained = retainedLimit(d.ProviderKind, capacity.MaxActive, capacity.MaxRetained) + if err := validateRuntimeNode("enrollment", capacity.MaxActive, capacity.MaxRetained); err != nil { + return err + } if err := q.CreateRuntimeEnrollment(ctx, sqlc.CreateRuntimeEnrollmentParams{TokenSha256: runtimeTokenDigest(token), InstallationID: d.InstallationID, MaxActive: int32(capacity.MaxActive), MaxRetained: int32(capacity.MaxRetained)}); err != nil { return err } @@ -203,7 +217,8 @@ func (s *Store) AuthenticateRuntimeNode(ctx context.Context, nodeID, credential return nodeIdentity(n, d.ProviderKind), nil } func (s *Store) UpdateRuntimeNode(ctx context.Context, nodeID string, input RuntimeNodeUpdate) error { - if err := validateRuntimeNode(input.Name, input.MaxActive, input.MaxRetained); err != nil { + // The retained limit depends on the provider, so the transaction checks it. + if err := validateRuntimeNode(input.Name, input.MaxActive, input.MaxActive); err != nil { return err } id, err := parseConnectionGeneration(nodeID) @@ -211,6 +226,10 @@ func (s *Store) UpdateRuntimeNode(ctx context.Context, nodeID string, input Runt return err } return s.runtimeManagerTransaction(ctx, func(q *sqlc.Queries, d sqlc.RuntimeDeployment) error { + input.MaxRetained = retainedLimit(d.ProviderKind, input.MaxActive, input.MaxRetained) + if err := validateRuntimeNode(input.Name, input.MaxActive, input.MaxRetained); err != nil { + return err + } n, err := q.GetRuntimeNode(ctx, id) if errors.Is(err, pgx.ErrNoRows) { return ErrNotFound @@ -266,7 +285,12 @@ func (s *Store) ListNodeRuntimeAllocations(ctx context.Context, nodeID string) ( } out := make([]RuntimeNodeAllocation, 0, len(rows)) for _, a := range rows { - out = append(out, RuntimeNodeAllocation{Diagnostic: a.ObservationError, ID: runtimeUUID(a.ID), NodeID: runtimeUUID(a.NodeID), TenantID: runtimeUUID(a.TenantID), SessionID: runtimeUUID(a.SessionID), EnvironmentID: runtimeUUID(a.EnvironmentID), State: a.State, ComputePhase: a.ComputePhase, Initialization: a.Initialization, CreatedAt: a.CreatedAt.Time}) + var phaseChanged *time.Time + if a.ComputePhaseChangedAt.Valid { + value := a.ComputePhaseChangedAt.Time + phaseChanged = &value + } + out = append(out, RuntimeNodeAllocation{Diagnostic: a.ObservationError, ID: runtimeUUID(a.ID), NodeID: runtimeUUID(a.NodeID), TenantID: runtimeUUID(a.TenantID), SessionID: runtimeUUID(a.SessionID), EnvironmentID: runtimeUUID(a.EnvironmentID), State: a.State, ComputePhase: a.ComputePhase, ComputePhaseChangedAt: phaseChanged, Initialization: a.Initialization, CreatedAt: a.CreatedAt.Time}) } return out, nil } diff --git a/services/agents-api/internal/store/runtime_suspension_test.go b/services/agents-api/internal/store/runtime_suspension_test.go index ef40431de..7535e2f4a 100644 --- a/services/agents-api/internal/store/runtime_suspension_test.go +++ b/services/agents-api/internal/store/runtime_suspension_test.go @@ -422,3 +422,30 @@ func TestRuntimeSuspensionRechecksCompletionAgainstIdleTimeout(t *testing.T) { }) } } + +// compute_phase_changed_at moves only when an allocation enters a different phase. +func TestRuntimeComputePhaseChangedAt(t *testing.T) { + _, w, pool, owner := runtimeSuspensionFixture(t) + runtimeSuspensionCompleted(t, pool, owner) + changedAt := func() time.Time { + t.Helper() + var value time.Time + if err := pool.QueryRow(t.Context(), "SELECT compute_phase_changed_at FROM runtime_allocations WHERE id=$1", owner.ID).Scan(&value); err != nil { + t.Fatal(err) + } + return value + } + entered := changedAt() + owner, err := w.SetRuntimeCompute(t.Context(), owner, "running", json.RawMessage(`{"instance":"updated"}`), nil, 0) + if err != nil { + t.Fatal(err) + } + if !changedAt().Equal(entered) { + t.Fatal("a same-phase update moved the phase time") + } + until := time.Now().Add(time.Hour) + runtimeSuspensionStep(t, w, owner, "quiescing", &until) + if !changedAt().After(entered) { + t.Fatal("entering a new phase kept the previous phase time") + } +} diff --git a/services/agents-api/migrations/000072_allocation_phase_time.sql b/services/agents-api/migrations/000072_allocation_phase_time.sql new file mode 100644 index 000000000..caae36a2d --- /dev/null +++ b/services/agents-api/migrations/000072_allocation_phase_time.sql @@ -0,0 +1,7 @@ +-- +goose Up +-- Existing rows keep NULL: the time they entered their current phase is unknown. +ALTER TABLE runtime_allocations ADD COLUMN compute_phase_changed_at timestamptz; +ALTER TABLE runtime_allocations ALTER COLUMN compute_phase_changed_at SET DEFAULT clock_timestamp(); + +-- +goose Down +ALTER TABLE runtime_allocations DROP COLUMN compute_phase_changed_at; From 6f6b4a54308606f61033e69715c4a0cfc3a3a6bf Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Fri, 25 Sep 2026 13:07:28 +0000 Subject: [PATCH 2/4] Report Docker's retained limit as its active limit on every read; check capacity before deployment state - Node identity, node configuration (token and node credential) and enrollment apply the Docker rule, so a token or node stored before it existed also reports and stores max_retained equal to max_active. - CreateRuntimeEnrollment normalizes and validates capacity before the mode/maintenance conflict check, so invalid capacity is still a 400. - compute_phase_changed_at is described as null when unknown; an allocation that existed before Core recorded it reports null until its next phase change. - Tests: Docker reads with legacy 2/8 token and node rows; microsandbox PATCH with max_retained below max_active is rejected and writes nothing; invalid capacity under maintenance is invalid input; the node allocation list returns the phase time and null for a NULL column. --- contracts/agents-api/sandbox-deployment.md | 7 +- .../agents-api/sandbox-manager.openapi.yaml | 2 +- packages/agents-client/src/sandbox-client.ts | 2 +- services/agents-api/HOSTED-SANDBOX-MANAGER.md | 7 +- .../store/runtime_node_capacity_test.go | 65 +++++++++++++++++-- .../store/runtime_node_configuration.go | 2 +- .../internal/store/runtime_node_types.go | 2 +- .../internal/store/runtime_nodes.go | 10 +-- .../internal/store/runtime_suspension_test.go | 29 +++++++++ 9 files changed, 105 insertions(+), 21 deletions(-) diff --git a/contracts/agents-api/sandbox-deployment.md b/contracts/agents-api/sandbox-deployment.md index 92a40495e..769d31339 100644 --- a/contracts/agents-api/sandbox-deployment.md +++ b/contracts/agents-api/sandbox-deployment.md @@ -45,9 +45,10 @@ safely, but never raise its limits. `GET /core/v1/sandbox/nodes/{node_id}/allocations` lists the node's unreleased allocations. Each item's `compute_phase_changed_at` is the time the allocation -entered its current `compute_phase`, or null for allocations created before Core -recorded it. A suspended microsandbox allocation's age, combined with the -deployment's snapshot retention, tells roughly when Core reclaims it. +entered its current `compute_phase`, or null when unknown; an allocation that +existed before Core recorded it reports null until its next phase change. A +suspended microsandbox allocation's age, combined with the deployment's snapshot +retention, tells roughly when Core reclaims it. ## Selection request diff --git a/contracts/agents-api/sandbox-manager.openapi.yaml b/contracts/agents-api/sandbox-manager.openapi.yaml index 4753da7f5..bad14da4e 100644 --- a/contracts/agents-api/sandbox-manager.openapi.yaml +++ b/contracts/agents-api/sandbox-manager.openapi.yaml @@ -832,7 +832,7 @@ definitions: compute_phase: type: string compute_phase_changed_at: - description: When the allocation entered its current compute_phase; null for allocations created before it was recorded. For a suspended microsandbox allocation, this time plus the deployment's snapshot retention tells roughly when Core reclaims it. + description: The time the allocation entered its current compute_phase, or null when unknown; an allocation that existed before Core recorded it reports null until its next phase change. For a suspended microsandbox allocation, this time plus the deployment's snapshot retention tells roughly when Core reclaims it. type: string x-nullable: true created_at: diff --git a/packages/agents-client/src/sandbox-client.ts b/packages/agents-client/src/sandbox-client.ts index 733736ffb..d875b9e3a 100644 --- a/packages/agents-client/src/sandbox-client.ts +++ b/packages/agents-client/src/sandbox-client.ts @@ -107,7 +107,7 @@ export interface SandboxAllocation { environment_id: string; state: string; compute_phase: string; - /** When the allocation entered compute_phase; null when it predates this record. */ + /** When the allocation entered its current compute_phase, or null when unknown; an allocation that existed before Core recorded it reports null until its next phase change. */ compute_phase_changed_at: string | null; diagnostic: SandboxDiagnostic; initialization: string; diff --git a/services/agents-api/HOSTED-SANDBOX-MANAGER.md b/services/agents-api/HOSTED-SANDBOX-MANAGER.md index 7d8180b9f..a58523055 100644 --- a/services/agents-api/HOSTED-SANDBOX-MANAGER.md +++ b/services/agents-api/HOSTED-SANDBOX-MANAGER.md @@ -58,9 +58,10 @@ readiness. To prepare the qualified E2B Runtime build, use Microsandbox suspends eligible idle Sessions after 300 seconds and retains their snapshots for 86400 seconds. Docker and E2B have no memory snapshot policy. A node's allocation list reports `compute_phase_changed_at`, the time each -allocation entered its current `compute_phase`; it is null for allocations created -before Core recorded it. A suspended allocation's age, combined with this retention, -tells roughly when Core reclaims it. +allocation entered its current `compute_phase`, or null when unknown; an allocation +that existed before Core recorded it reports null until its next phase change. A +suspended allocation's age, combined with this retention, tells roughly when Core +reclaims it. For source/manual deployments using E2B, install the packaged helper and set `AGENTS_API_E2B_PROVIDER_BIN` to its absolute executable path. Set diff --git a/services/agents-api/internal/store/runtime_node_capacity_test.go b/services/agents-api/internal/store/runtime_node_capacity_test.go index f5f6373b1..eee23bc66 100644 --- a/services/agents-api/internal/store/runtime_node_capacity_test.go +++ b/services/agents-api/internal/store/runtime_node_capacity_test.go @@ -34,6 +34,14 @@ func TestRuntimeEnrollmentApprovedCapacity(t *testing.T) { if err := s.UpdateRuntimeNode(t.Context(), input.NodeID, RuntimeNodeUpdate{Name: "approved", MaxActive: 2, MaxRetained: 5}); err != nil { t.Fatal(err) } + // Microsandbox uses both limits, so a retained limit below the active one is rejected and nothing is written. + if err := s.UpdateRuntimeNode(t.Context(), input.NodeID, RuntimeNodeUpdate{Name: "rejected", MaxActive: 4, MaxRetained: 3}); !errors.Is(err, ErrInvalidInput) { + t.Fatal("retained limit below active accepted", err) + } + var name string + if err := s.pool.QueryRow(t.Context(), "SELECT name FROM runtime_nodes WHERE id=$1", input.NodeID).Scan(&name); err != nil || name != "approved" { + t.Fatal("rejected update was written", name, err) + } identity, err = s.AuthenticateRuntimeNode(t.Context(), input.NodeID, input.Credential) if err != nil || identity.MaxActive != 2 || identity.MaxRetained != 5 { t.Fatal("credential read ignored admin update", identity, err) @@ -42,24 +50,69 @@ func TestRuntimeEnrollmentApprovedCapacity(t *testing.T) { if err != nil || config.MaxActive != 2 || config.MaxRetained != 5 { t.Fatal("configuration read ignored admin update", config, err) } + // Invalid capacity is reported before the deployment's maintenance conflict. + if _, err := s.pool.Exec(t.Context(), "UPDATE runtime_deployment SET maintenance=true"); err != nil { + t.Fatal(err) + } + if _, _, err := s.CreateRuntimeEnrollment(t.Context(), RuntimeNodeCapacity{3, 2}); !errors.Is(err, ErrInvalidInput) { + t.Fatal("invalid capacity reported as a conflict", err) + } } -// Docker never suspends a sandbox, so its retained limit always equals its active limit. +// Docker never suspends a sandbox, so every read reports its retained limit as its +// active limit, including for a token or node stored before Core applied that rule. func TestDockerRetainedLimitFollowsActive(t *testing.T) { s, _, view, _ := webSpecificationFixture(t, "docker") + node := func(name, credential string) RuntimeNodeEnrollment { + return RuntimeNodeEnrollment{NodeID: uuid.NewString(), Name: name, Credential: strings.Repeat(credential, 64), Provider: "docker", BackendFingerprint: strings.Repeat("b", 64), SpecificationDigest: view.SpecificationDigest, DeploymentGeneration: view.Generation} + } + sql := func(query string, args ...any) { + t.Helper() + if _, err := s.pool.Exec(t.Context(), query, args...); err != nil { + t.Fatal(err) + } + } token, _, err := s.CreateRuntimeEnrollment(t.Context(), RuntimeNodeCapacity{MaxActive: 3, MaxRetained: 1}) if err != nil { t.Fatal(err) } - input := RuntimeNodeEnrollment{NodeID: uuid.NewString(), Name: "Docker", Credential: strings.Repeat("d", 64), Provider: "docker", BackendFingerprint: strings.Repeat("b", 64), SpecificationDigest: view.SpecificationDigest, DeploymentGeneration: view.Generation} - if identity, err := s.EnrollRuntimeNode(t.Context(), token, input); err != nil || identity.MaxRetained != 3 { + current := node("Docker", "d") + if identity, err := s.EnrollRuntimeNode(t.Context(), token, current); err != nil || identity.MaxRetained != 3 { t.Fatal("enrollment kept a separate Docker retained limit", identity, err) } - if err := s.UpdateRuntimeNode(t.Context(), input.NodeID, RuntimeNodeUpdate{Name: "Docker", MaxActive: 5, MaxRetained: 12}); err != nil { + legacyToken, _, err := s.CreateRuntimeEnrollment(t.Context(), RuntimeNodeCapacity{MaxActive: 2, MaxRetained: 8}) + if err != nil { + t.Fatal(err) + } + sql("UPDATE runtime_node_enrollments SET max_retained=8 WHERE token_sha256=$1", runtimeTokenDigest(legacyToken)) + if config, err := s.RuntimeNodeConfiguration(t.Context(), "", legacyToken); err != nil || config.MaxRetained != 2 { + t.Fatal("bootstrap reported a legacy Docker retained limit", config, err) + } + legacy := node("Legacy", "l") + if _, err := s.EnrollRuntimeNode(t.Context(), legacyToken, legacy); err != nil { + t.Fatal(err) + } + var stored int32 + if err := s.pool.QueryRow(t.Context(), "SELECT max_retained FROM runtime_nodes WHERE id=$1", legacy.NodeID).Scan(&stored); err != nil || stored != 2 { + t.Fatal("enrollment stored a legacy Docker retained limit", stored, err) + } + sql("UPDATE runtime_nodes SET max_retained=8 WHERE id=$1", legacy.NodeID) + if identity, err := s.AuthenticateRuntimeNode(t.Context(), legacy.NodeID, legacy.Credential); err != nil || identity.MaxRetained != 2 { + t.Fatal("node identity reported a legacy Docker retained limit", identity, err) + } + if config, err := s.RuntimeNodeConfiguration(t.Context(), legacy.NodeID, legacy.Credential); err != nil || config.MaxRetained != 2 { + t.Fatal("node configuration reported a legacy Docker retained limit", config, err) + } + if err := s.UpdateRuntimeNode(t.Context(), current.NodeID, RuntimeNodeUpdate{Name: "Docker", MaxActive: 5, MaxRetained: 12}); err != nil { t.Fatal(err) } nodes, err := s.ListRuntimeNodes(t.Context()) - if err != nil || len(nodes) != 1 || nodes[0].MaxActive != 5 || nodes[0].MaxRetained != 5 { - t.Fatal("node update kept a separate Docker retained limit", nodes, err) + if err != nil || len(nodes) != 2 { + t.Fatal(nodes, err) + } + for _, n := range nodes { + if n.MaxRetained != n.MaxActive || (n.ID == current.NodeID && n.MaxActive != 5) { + t.Fatal("node list kept a separate Docker retained limit", n) + } } } diff --git a/services/agents-api/internal/store/runtime_node_configuration.go b/services/agents-api/internal/store/runtime_node_configuration.go index 61a0cdecb..3626ce589 100644 --- a/services/agents-api/internal/store/runtime_node_configuration.go +++ b/services/agents-api/internal/store/runtime_node_configuration.go @@ -55,7 +55,7 @@ func (s *Store) RuntimeNodeConfiguration(ctx context.Context, nodeID, token stri return ErrRuntimeSpecificationMismatch } } - result = RuntimeNodeConfiguration{MaxActive: int(active), MaxRetained: int(retained), InstallationID: runtimeUUID(d.InstallationID), Provider: d.ProviderKind, CoreURL: d.CoreUrl, Generation: uint64(d.Generation), Specification: spec, SpecificationDigest: spec.Digest(d.ProviderKind)} + result = RuntimeNodeConfiguration{MaxActive: int(active), MaxRetained: retainedLimit(d.ProviderKind, int(active), int(retained)), InstallationID: runtimeUUID(d.InstallationID), Provider: d.ProviderKind, CoreURL: d.CoreUrl, Generation: uint64(d.Generation), Specification: spec, SpecificationDigest: spec.Digest(d.ProviderKind)} return nil }) return result, err diff --git a/services/agents-api/internal/store/runtime_node_types.go b/services/agents-api/internal/store/runtime_node_types.go index ba4363508..92418e068 100644 --- a/services/agents-api/internal/store/runtime_node_types.go +++ b/services/agents-api/internal/store/runtime_node_types.go @@ -138,7 +138,7 @@ type RuntimeNodeAllocation struct { EnvironmentID string `json:"environment_id"` State string `json:"state"` ComputePhase string `json:"compute_phase"` - // When the allocation entered its current compute_phase; null for allocations created before it was recorded. For a suspended microsandbox allocation, this time plus the deployment's snapshot retention tells roughly when Core reclaims it. + // The time the allocation entered its current compute_phase, or null when unknown; an allocation that existed before Core recorded it reports null until its next phase change. For a suspended microsandbox allocation, this time plus the deployment's snapshot retention tells roughly when Core reclaims it. ComputePhaseChangedAt *time.Time `json:"compute_phase_changed_at" extensions:"x-nullable"` Initialization string `json:"initialization"` CreatedAt time.Time `json:"created_at"` diff --git a/services/agents-api/internal/store/runtime_nodes.go b/services/agents-api/internal/store/runtime_nodes.go index e8432834d..56a3b34bf 100644 --- a/services/agents-api/internal/store/runtime_nodes.go +++ b/services/agents-api/internal/store/runtime_nodes.go @@ -115,13 +115,13 @@ func (s *Store) CreateRuntimeEnrollment(ctx context.Context, capacity RuntimeNod token := hex.EncodeToString(bytes[:]) var expires time.Time err := s.runtimeManagerTransaction(ctx, func(q *sqlc.Queries, d sqlc.RuntimeDeployment) error { - if d.Mode != "nodes" || d.Maintenance || unspecifiedNodeDeployment(d) { - return ErrSandboxDeploymentConflict - } capacity.MaxRetained = retainedLimit(d.ProviderKind, capacity.MaxActive, capacity.MaxRetained) if err := validateRuntimeNode("enrollment", capacity.MaxActive, capacity.MaxRetained); err != nil { return err } + if d.Mode != "nodes" || d.Maintenance || unspecifiedNodeDeployment(d) { + return ErrSandboxDeploymentConflict + } if err := q.CreateRuntimeEnrollment(ctx, sqlc.CreateRuntimeEnrollmentParams{TokenSha256: runtimeTokenDigest(token), InstallationID: d.InstallationID, MaxActive: int32(capacity.MaxActive), MaxRetained: int32(capacity.MaxRetained)}); err != nil { return err } @@ -163,7 +163,7 @@ func (s *Store) EnrollRuntimeNode(ctx context.Context, token string, input Runti } else if !errors.Is(err, pgx.ErrNoRows) { return err } - row, err := q.InsertRuntimeNode(ctx, sqlc.InsertRuntimeNodeParams{ID: id, InstallationID: d.InstallationID, Name: input.Name, BackendFingerprint: input.BackendFingerprint, CredentialSha256: runtimeTokenDigest(input.Credential), MaxActive: receipt.MaxActive, MaxRetained: receipt.MaxRetained, SpecificationDigest: input.SpecificationDigest, DeploymentGeneration: int64(input.DeploymentGeneration)}) + row, err := q.InsertRuntimeNode(ctx, sqlc.InsertRuntimeNodeParams{ID: id, InstallationID: d.InstallationID, Name: input.Name, BackendFingerprint: input.BackendFingerprint, CredentialSha256: runtimeTokenDigest(input.Credential), MaxActive: receipt.MaxActive, MaxRetained: int32(retainedLimit(d.ProviderKind, int(receipt.MaxActive), int(receipt.MaxRetained))), SpecificationDigest: input.SpecificationDigest, DeploymentGeneration: int64(input.DeploymentGeneration)}) if err != nil { return err } @@ -180,7 +180,7 @@ func (s *Store) EnrollRuntimeNode(ctx context.Context, token string, input Runti return result, err } func nodeIdentity(n sqlc.RuntimeNode, kind string) RuntimeNodeIdentity { - return RuntimeNodeIdentity{SpecificationDigest: n.SpecificationDigest, DeploymentGeneration: uint64(n.DeploymentGeneration), NodeID: runtimeUUID(n.ID), InstallationID: runtimeUUID(n.InstallationID), Provider: kind, BackendFingerprint: n.BackendFingerprint, MaxActive: int(n.MaxActive), MaxRetained: int(n.MaxRetained)} + return RuntimeNodeIdentity{SpecificationDigest: n.SpecificationDigest, DeploymentGeneration: uint64(n.DeploymentGeneration), NodeID: runtimeUUID(n.ID), InstallationID: runtimeUUID(n.InstallationID), Provider: kind, BackendFingerprint: n.BackendFingerprint, MaxActive: int(n.MaxActive), MaxRetained: retainedLimit(kind, int(n.MaxActive), int(n.MaxRetained))} } func (s *Store) AuthenticateRuntimeNode(ctx context.Context, nodeID, credential string) (RuntimeNodeIdentity, error) { id, err := parseConnectionGeneration(nodeID) diff --git a/services/agents-api/internal/store/runtime_suspension_test.go b/services/agents-api/internal/store/runtime_suspension_test.go index 7535e2f4a..1a8227a2d 100644 --- a/services/agents-api/internal/store/runtime_suspension_test.go +++ b/services/agents-api/internal/store/runtime_suspension_test.go @@ -449,3 +449,32 @@ func TestRuntimeComputePhaseChangedAt(t *testing.T) { t.Fatal("entering a new phase kept the previous phase time") } } + +// The node allocation list reports the phase time, and null when it is unknown. +func TestRuntimeComputePhaseChangedAtInNodeAllocations(t *testing.T) { + s, w, d := managerFixture(t, 1, 4) + tenant := uuid.NewString() + session, err := s.CreateSession(t.Context(), tenant, managerSessionInput("listed", d.LocalNodeID)) + if err != nil { + t.Fatal(err) + } + allocation, err := w.ReserveRuntimeAllocation(t.Context(), tenant, session.Environment.ID, d.InstallationID, device.HashCredential("runtime")) + if err != nil { + t.Fatal(err) + } + listed := func() RuntimeNodeAllocation { + t.Helper() + items, err := s.ListNodeRuntimeAllocations(t.Context(), d.LocalNodeID) + if err != nil || len(items) != 1 || items[0].ID != allocation.ID { + t.Fatal(items, err) + } + return items[0] + } + if listed().ComputePhaseChangedAt == nil { + t.Fatal("a new allocation has no phase time") + } + runtimeSuspensionSQL(t, s.pool, "UPDATE runtime_allocations SET compute_phase_changed_at=NULL WHERE id=$1", allocation.ID) + if encoded, _ := json.Marshal(listed()); !strings.Contains(string(encoded), `"compute_phase_changed_at":null`) { + t.Fatal("an unknown phase time was not null", string(encoded)) + } +} From e3a846ddbd19e7c39c922fe23526373e3c883af8 Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Fri, 25 Sep 2026 13:33:21 +0000 Subject: [PATCH 3/4] Document what each sandbox field means per provider Add a table to the deployment contract covering E2B, Docker and microsandbox: deployment specification, template build, suspension and resource counts; enrollment limits; node counts, diagnostics and host disk; allocation phases; runtime observation CPU, memory, disk and sleeping state; and runtime history CPU. Link it from the Hosted Sandbox Manager guide. --- contracts/agents-api/sandbox-deployment.md | 27 +++++++++++++++++++ services/agents-api/HOSTED-SANDBOX-MANAGER.md | 2 ++ 2 files changed, 29 insertions(+) diff --git a/contracts/agents-api/sandbox-deployment.md b/contracts/agents-api/sandbox-deployment.md index 769d31339..5f7754c2a 100644 --- a/contracts/agents-api/sandbox-deployment.md +++ b/contracts/agents-api/sandbox-deployment.md @@ -243,6 +243,33 @@ token consumption. Retained node authentication checks the same generation and digest. A changed local resource setting, Runtime or generation must fail rather than rewrite the retained identity or silently use a local default. +## What each field means per sandbox provider + +Some fields keep one name across providers but differ in meaning, or do not apply. +Deployment fields come from `GET /core/v1/sandbox/deployment`; node and allocation +fields from the administrator node routes; runtime fields from the +[Runtime observation API](runtime-observability-api.md), with `disk` only in the +administrator `GET /core/v1/admin/runtime-observations`, and the +[Runtime history API](runtime-history-api.md). + +| Field | E2B | Docker | microsandbox | +| --- | --- | --- | --- | +| Deployment `specification.resources` | `cpus` and `memory_mib`, equal to the ready template build's and taken from it when omitted; no disk fields | `cpus` and `memory_mib`; no disk quota | `cpus`, `memory_mib`, `root_disk_mib` and `environment_disk_mib` | +| Deployment `specification.runtime` | Absent; the build is selected by `e2b.template` | The full [release](#runtime-release); nodes match `image_id` or `image_manifest_digest` | The full [release](#runtime-release); nodes match `microsandbox_ref`, `runtime_sha256` and `firmware_sha256` | +| Deployment `e2b.template_build` | The build as Core read it when the selection was saved | Absent, with the whole `e2b` object | Absent, with the whole `e2b` object | +| Deployment `suspension` | `null`; Core does not suspend E2B sandboxes | `null` | `{idle_seconds, retention_seconds}` | +| Deployment `resources.allocations`, `resources.pending` | Core's unreleased E2B sandboxes, and hosted Environments waiting for one | Totals across all nodes | Totals across all nodes | +| Enrollment-token `max_active`, `max_retained` | 409 `sandbox_deployment_conflict`, after the 400 capacity checks; E2B has no nodes | `max_retained` always equals `max_active` | Both limits apply | +| Node list and detail | Empty list; detail returns 404 | Enrolled nodes | Enrolled nodes | +| Node `retained`, `snapshots`, `max_retained` | Not applicable | Docker never suspends: `retained` equals `active`, `snapshots` is 0 and `max_retained` equals `max_active` | Suspended sandboxes are `retained` minus `active` | +| Node `diagnostic` | Not applicable | `docker_unavailable`, `docker_limits_unsupported`, `runtime_image_unavailable`, `capacity_insufficient` or `provider_unavailable` | `kvm_unavailable`, `microsandbox_artifacts_unavailable`, `capacity_insufficient` or `provider_unavailable` | +| Node `host.available_disk_bytes` | Not applicable | Free space on the filesystem of the node state directory, not a container's disk | Free space on the filesystem of the node state directory; sandbox disks have their own quotas | +| Allocation `compute_phase`, `compute_phase_changed_at` | Not applicable: no node allocations | Always `disabled`, counted as running until release; the time is the allocation's creation | Includes `suspended`; its time plus `suspension.retention_seconds` tells roughly when Core reclaims the snapshot | +| Runtime observation `cpu`, `memory` | From E2B metrics: `cpu.utilization_ratio` and `capacity_cores`, memory usage and limit; no cumulative CPU time | From Docker stats: `cpu.usage_seconds_total`, CPU and memory limits, memory usage | From the VM: `cpu.usage_seconds_total`, CPU and memory limits, memory usage | +| Runtime observation `disk` | E2B `diskUsed` and `diskTotal`; `null` when the template does not report them | `null`: no disk quota | `null` for now | +| Runtime observation `lifecycle_state: sleeping` | Never | Never | While suspended | +| Runtime history CPU | Mean of the utilization ratios E2B reported in each bucket | Derived from cumulative CPU time | Derived from cumulative CPU time | + ## Failure and upgrade boundaries Malformed selections return 400; validated configuration diagnostics use diff --git a/services/agents-api/HOSTED-SANDBOX-MANAGER.md b/services/agents-api/HOSTED-SANDBOX-MANAGER.md index a58523055..3d6021d3b 100644 --- a/services/agents-api/HOSTED-SANDBOX-MANAGER.md +++ b/services/agents-api/HOSTED-SANDBOX-MANAGER.md @@ -62,6 +62,8 @@ allocation entered its current `compute_phase`, or null when unknown; an allocat that existed before Core recorded it reports null until its next phase change. A suspended allocation's age, combined with this retention, tells roughly when Core reclaims it. +See [what each field means per sandbox provider](../../contracts/agents-api/sandbox-deployment.md#what-each-field-means-per-sandbox-provider) +for fields that differ between E2B, Docker and microsandbox. For source/manual deployments using E2B, install the packaged helper and set `AGENTS_API_E2B_PROVIDER_BIN` to its absolute executable path. Set From 7dd3bafdcf1868d6e5f7279d17bb7e1cd0914642 Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Fri, 25 Sep 2026 13:46:15 +0000 Subject: [PATCH 4/4] Renumber the allocation phase time migration to 000074 000073 from #118 is already on main. Numbering this branch's migration after it lets databases that applied 000073 upgrade without goose's out-of-order migration error. No test or document references the version. --- ...allocation_phase_time.sql => 000074_allocation_phase_time.sql} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename services/agents-api/migrations/{000072_allocation_phase_time.sql => 000074_allocation_phase_time.sql} (100%) diff --git a/services/agents-api/migrations/000072_allocation_phase_time.sql b/services/agents-api/migrations/000074_allocation_phase_time.sql similarity index 100% rename from services/agents-api/migrations/000072_allocation_phase_time.sql rename to services/agents-api/migrations/000074_allocation_phase_time.sql