fix(gateway): delete finalized ephemeral sandboxes while connected

[johntmyers]

🏗️ build-from-issue-agent

Summary

Detached --no-keep sandboxes could remain in Completed or Error indefinitely because their supervisor kept its control session connected after the canonical process exited. Start driver deletion when the terminal result is finalized, so cleanup completes while that session is still connected.

Related Issue

Closes #3938

Changes

• Schedule ephemeral deletion after durable main-process finalization and session finalization, including the race where the session disconnects between those steps.
• Preserve the disconnect fallback and existing retained, restart-policy, and provisioning-timeout behavior. Delete by sandbox ID to avoid targeting a reused name.
• Add unit coverage for cleanup while connected and retained/restarting cases.
• Add shared detached success/failure e2e tests that verify both gateway record deletion and driver resource removal. Wire them into Docker, Kubernetes, Podman, and VM suites.
• Update the Gator skill's cleanup description. Published docs already describe the intended lifecycle; configuration is unchanged.

Testing

• mise run pre-commit passes
• Unit tests added/updated; three focused cleanup tests pass
• E2E tests added/updated
• OPENSHELL_E2E_DOCKER_TEST=ephemeral_cleanup mise run e2e:docker: both cleanup tests and all six conformance scenarios pass
• New e2e target compiles and passes Clippy
• mise run ci passes, including the full unit suite

Earlier local test attempts timed out in proxy::tests::mediated_connect_keeps_workload_bytes_read_with_the_synthesized_header in openshell-supervisor-network; that test passed in the final full CI run. This PR does not change that crate. Live Podman, Kubernetes, and VM runs were not performed locally.

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)

[johntmyers]

🏗️ build-from-issue-agent

E2E Test Attestation

Local Docker e2e passed for the code committed in 3c14bb866494851a99d1d377fadfd6f77cf839c5. The test ran before committing; the subsequent change only corrected the Gator skill's cleanup description.

Command: OPENSHELL_E2E_DOCKER_TEST=ephemeral_cleanup mise run e2e:docker

Gateway mode: Docker, with gateway and runtime images built from this branch.

Result: 2 cleanup tests passed, 0 failed, 0 skipped (4.96 seconds); all six prerequisite conformance scenarios passed.

Tests executed:

• detached_ephemeral_success_removes_sandbox_and_driver_resources — passed
• detached_ephemeral_failure_removes_sandbox_and_driver_resources — passed
• Conformance smoke — passed
• Conformance sandbox-lifecycle — passed
• Conformance file-transfer — passed
• Conformance mechanistic-proposal — passed
• Conformance new-hostname-proposal — passed
• Conformance policy-local — passed

Both cleanup tests observed driver resources before releasing the canonical process, then verified that the gateway record and driver resources disappeared without a test-issued delete.

The final mise run ci and pre-commit checks also passed. Live Podman, Kubernetes, and VM lanes were not run locally.

[github-actions]

Label test:e2e applied for 3c14bb8. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

[drew]

gator-agent

PR Review Status

PR #3984 is project-valid through accepted issue #3938. The independent initial review found no blocking findings, and the existing documentation already describes the restored ephemeral-cleanup behavior.

Action required: A maintainer must open E2E run 36762320399 and choose Re-run all jobs so the required current-head E2E suite executes with test:e2e applied.

Blocking findings:

• No blocking findings remain

Carried findings:

• None

Non-blocking suggestions:

• None

Gator metadata

• Validation: Accepted issue #3938 covers this focused gateway lifecycle fix; the PR author is a repository admin.
• Docs: Not needed because this restores the published --detach --no-keep lifecycle without changing the user contract.
• Checks: Branch Checks, Helm Lint, Trivy Changes, DCO, and vouch passed on the current head; required E2E dispatch remains outstanding.
• E2E: test:e2e applied; E2E Label Help requires Re-run all jobs on run 36762320399.
• Head SHA: 3c14bb866494851a99d1d377fadfd6f77cf839c5
• Base SHA: 7caff12d3c9013e7063d22391a76f775f6ce93b5
• Merge base SHA: 7caff12d3c9013e7063d22391a76f775f6ce93b5
• Patch ID: 284e7c9ae4b350c725eb10a8a8c81f08bcb800fa
• Gator payload: 10
• Review mode: initial
• Previous reviewed SHA: none
• Review budget exhausted: no
• Maintainer decision required: no
• Next state: gator:blocked
• Blocked reason: test_dispatch_required

[drew]

/ok to test 3c14bb8

[drew]

gator-agent

Monitoring Complete

Monitoring is complete because this PR has merged.

Final status: The reviewed head was approved, required checks including Core E2E passed, and the PR merged successfully. I removed the active gator:* label because there is nothing left for gator to monitor on this PR.

Gator metadata

• Head SHA: 3c14bb866494851a99d1d377fadfd6f77cf839c5
• Gator payload: 10
• Previous state: gator:merge-ready