Skip to content

Pause idle E2B sandboxes and resume original compute - #248

Closed
sam2tom wants to merge 19 commits into
mainfrom
codex/e2b-idle-pause
Closed

sam2tom wants to merge 19 commits into
mainfrom
codex/e2b-idle-pause

Conversation

@sam2tom

@sam2tom sam2tom commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Superseded: continue in #297

All active work is consolidated in #297 (codex/e2b-unified-pause), targeting main directly. This PR is closed without merging. Historical review and evidence are retained below.

Superseded by #296 and #297

The replacement implementation is pushed as two separately reviewable PRs: #296 defines the unified suspension protocol; #297 implements E2B through that shared lifecycle. This PR remains a draft and must not be merged. The replacement head is 022875c0fb5358576058ed31668fa8d4b8bc48a5; its full Linux checks and GitHub CI pass. Live pause/resume qualification of the replacement is still pending. Historical evidence below applies only to the old implementation.

Status: architecture blocked; do not merge or roll out

Head: aaeddaa0b8db6a6e41dcb7bf5db8e9697c52fc66. Reviewed against updated origin/main at 17bbffa4a and its current AGENTS.md. The implementation does not meet the required ownership boundary. Green CI and the earlier live run do not resolve that issue.

The blocking changes are ResidentPauseProvider, runtime_compute_resident.go, managed_generations_resident.go, and Store's SetRuntimeResidentCompute / ReadyToPauseResident. They introduce a second lifecycle and provider-selected eligibility for initialized Sessions without a Turn. Generic names and capability declarations do not make that architecture acceptable. Earlier independent reviews missed this boundary violation.

Required redesign

The current shared CheckpointProvider contract requires a verified full snapshot, deletion of the source compute and restoration into a new incarnation. E2B native pause/resume retains the original sandbox. Implementing it with a fabricated SnapshotIdentity or a no-op KillCompute would violate the contract.

A unified protocol change must therefore be proposed and reviewed separately before reworking this E2B integration. The recommended scope is to replace the checkpoint-specific orchestration with one declared suspension lifecycle: Core owns idle admission, durable operation intent, capacity and authenticated daemon wake; adapters own retained resources, native pause/capture/restore, settlement evidence and native cleanup. All existing adapters and node/helper transports must change together. E2B suspension stays unsupported in that foundational change and is added in its subsequent adapter change.

Remove the resident interface, Core path and Store entry points when rebuilding this PR on the reviewed foundation. Preserve main's single idle eligibility rule initially; supporting initialized Sessions without a Turn requires a shared lifecycle change for every supported provider. Preserve the original 3600-second E2B native timeout. Do not deploy a protocol/persisted-state change without its reviewed upgrade or drain contract.

CI evidence for aaeddaa

GitHub currently reports SUCCESS for check, backend, tooling, web, both web-acceptance shards, official-client, and Linux/macOS/Windows platform jobs. These validate the existing implementation, not the proposed redesign. Local macOS make check was not a complete pass; Linux-specific gates require Linux.

Live evidence: prior commit 31f5a5b only

The following is the recorded real-provider acceptance of the earlier implementation. It is not acceptance of aaeddaa or of a future unified protocol.

  • Endpoint: https://sandbox.sandbase.ai; Codex with gpt-5.5 through the configured SandBase model endpoint.
  • Template: openagentcore-codex-31f5a5b9, tpl_661ea8869eca4687b7957836c7eae7c5:db2ecf2b-dea1-4c18-b7bf-f7297c2c0d05.
  • Session: eff37a81-a9bc-4d98-aa63-75ee3a31b356.
  • Initial Turn: 2885e928-078a-43f5-8aff-2b4531177955, completed, 15,081 tokens. A tool wrote and read a workspace marker.
  • Last activity recorded at 2026-09-30 10:03:21.521797 UTC; Core suspended at 10:08:27.136032 UTC (approximately 306 seconds). The SDK observed paused state.
  • Resume retained sandbox sbx4570cf44a3841567f23d90a3cc781204ef11 and the existing workspace file.
  • Follow-up Turn: d6ebbba7-9ab2-4129-b11a-2662faffdf4f, completed, 15,401 tokens. A tool read the existing marker without recreating it.
  • A read-only recheck during this review returned HTTP 200 for the retained Turns and items, confirmed both completed statuses and the marker in the six retained items. It did not repeat the pause/resume cycle.
  • Session/history were preserved. The 24-hour retention expiry was not qualified.

Narrower template evidence for aaeddaa

openagentcore-all-aaeddaa0 (tpl_0043dba4f2e742c1a2d3405a76f22556:efa1c425-c2cf-49c5-9cc5-07efab13e920) contains Codex 0.153.4, Claude SDK 0.3.269 / native 2.1.269, and MiniMax Code 0.4.12. Recorded native probes exited zero for all three and found their activation paths. This proves template contents/native readiness only. The template was not selected in Core; three managed real-model conversations and pause/resume on this template are not qualified.

Protocol proposal review status

Independent design review accepts the direction of a separate unified lifecycle proposal, not an implementation-ready contract. The exact protocol review must resolve Create/Kill settlement, revision/fence precedence, admission during retained-artifact finalization, durable adapter fencing across restart/retirement, capacity persistence after unknown resume, and the state upgrade/drain contract. No implementation changes were made during this review; head remains aaeddaa. The rejected resident paths are still present and remain blocking.

Remaining merge gates

  • Review and implement the separate unified provider protocol and its upgrade semantics.
  • Remove parallel resident paths and implement E2B against the shared lifecycle inside its adapter.
  • Fresh independent architectural review of the complete final diff against current main.
  • Canonical make check and adapter/native contract checks on the final revision.
  • Exact-revision live E2B pause/resume, same native ID, workspace and native conversation continuity, daemon authentication and completed real-model Turns.
  • Repeated cycles, retention cleanup and microsandbox checkpoint regression acceptance.

This PR remains blocked until the architecture and final-revision evidence meet those gates.

@sam2tom
sam2tom force-pushed the codex/e2b-idle-pause branch 2 times, most recently from 67ec174 to f305018 Compare September 30, 2026 01:52
@sam2tom
sam2tom force-pushed the codex/e2b-idle-pause branch from f305018 to 234ed63 Compare September 30, 2026 01:53
@blacksmith-sh

This comment has been minimized.

@SaladDay

Copy link
Copy Markdown
Collaborator

Our design principle is that provider-specific complexity stays inside the adapter. A “thin” integration means replacing a Provider does not require changing the common execution flow; it does not mean the adapter must contain little code.

Core can own shared scheduling, persistence and recovery through capability contracts such as ResidentPauseProvider. E2B SDK calls and native behavior belong in the E2B adapter. Please also replace the newly added if input.Provider == "e2b" in sandbox_deployment_mutations.go with a provider-neutral comparison of the registered policy, so the Store does not need vendor-specific behavior.

@blacksmith-sh

This comment has been minimized.

@sam2tom
sam2tom marked this pull request as ready for review September 30, 2026 09:24
@SaladDay

SaladDay commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

I do not recommend merging this PR yet. I updated main and reviewed commit aaeddaa against the latest AGENTS.md.

The requirement is to keep E2B pause/resume behavior and its complexity inside the adapter. The current changes cross that boundary:

  • Core execution gains a resident branch and runtime_compute_resident.go to manage pause, resume, and uncertain outcomes.
  • Store gains SetRuntimeResidentCompute / ReadyToPauseResident and changes idle eligibility for Sessions that have never received a Turn based on the resident path.
  • ResidentPauseProvider adds a separate extension interface. The latest AGENTS.md explicitly prohibits a vendor-only pause interface and a dedicated Core path. Generic naming or capability checks do not resolve this boundary issue.

Please move E2B-specific behavior and recovery logic into the adapter and reuse the shared lifecycle. If the existing protocol cannot express the required behavior, propose and review a unified protocol change separately before integrating E2B, rather than adding parallel Core/Store paths.

The current CI checks are green, but the architecture issue still blocks merging. The PR description also lists live E2B pause/resume acceptance as outstanding; please provide that evidence and update the description.

@sam2tom sam2tom closed this Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants