Skip to content

fix(e2e): stop sandbox leaks from async Drop cleanup - #3750

Open
ericcurtin wants to merge 2 commits into
NVIDIA:mainfrom
ericcurtin:fix/2922-sandbox-guard-drop-cleanup
Open

ericcurtin wants to merge 2 commits into
NVIDIA:mainfrom
ericcurtin:fix/2922-sandbox-guard-drop-cleanup

Conversation

@ericcurtin

Copy link
Copy Markdown
Contributor

🏗️ build-from-issue-agent

Summary

SandboxGuard::Drop cleaned up via a detached thread that got killed with the test process before the delete finished. Switched to a blocking delete, like ManagedCleanup already does. Also wrapped two tests' manual cleanup in RAII guards.

Related Issue

Closes #2922

Changes

  • e2e/rust/src/harness/sandbox.rs: synchronous delete in Drop.
  • e2e/rust/tests/sandbox_labels.rs: use SandboxGuard for the four test sandboxes.
  • e2e/rust/tests/sandbox_lifecycle.rs: use SandboxGuard in both canonical-main-exit tests.

Testing

  • cargo test --lib in e2e/rust
  • mise run pre-commit

Checklist

  • Conventional Commits
  • Signed off (DCO)

Note: issue #2922 has state:accepted but no agent:implementation-requested label. This is a direct build request, so I proceeded without adding it.

Closes NVIDIA#2922

SandboxGuard::Drop spawned a detached thread to delete the sandbox.
The thread got killed with the test process before the delete
finished. Switch to a blocking command in Drop, like ManagedCleanup
already does. Also wrap two tests' manual cleanup in RAII guards so
a panic does not leak a sandbox.

Signed-off-by: Eric Curtin <eric.curtin@docker.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 27, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@elezar elezar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The guards should be armed before sandbox creation, using an explicit unique name.

Currently, each guard is constructed only after the create helper returns or after the initial assertions pass. If creation partially succeeds but returns an unexpected status or output, or if name parsing fails, the test panics before installing the guard and still leaks the sandbox.

Please generate the sandbox name first, construct SandboxGuard::manage_existing(name.clone()), and pass that name to sandbox create --name. Since deletion uses allow_missing, dropping the guard when creation never produced a sandbox is a safe no-op.

For example:

let sandbox_name = unique_sandbox_name("canonical-zero");
let _cleanup = SandboxGuard::manage_existing(sandbox_name.clone());

let output = openshell_tty_cmd(&[
    "sandbox",
    "create",
    "--name",
    &sandbox_name,
    "--",
    "echo",
    "OK",
])
.output()
.await
.expect("spawn openshell sandbox create");

Please apply the same ordering to the label test: its names are already known before create_sandbox_with_labels, so each guard can be installed before its create call. This ensures cleanup covers the assertion and parsing failures that these tests are intended to diagnose.

Address review: install guards with explicit names first.

Signed-off-by: Eric Curtin <eric.curtin@docker.com>
@ericcurtin

Copy link
Copy Markdown
Contributor Author

Addressed: guards are now armed before create with explicit unique names, in both lifecycle tests and the labels test.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(e2e): sandbox pods not cleaned up during test runs — SandboxGuard::Drop uses detached threads

2 participants