test(oci): share OCI USER and WORKDIR checks across Docker and Podman by matthewgrossman · Pull Request #3931 · NVIDIA/OpenShell

matthewgrossman · 2026-09-30T06:01:31Z

Summary

An image's USER and WORKDIR should set the sandbox's identity and workspace the same way whether a gateway uses Docker or Podman. This PR moves the existing Docker-only image checks into a shared test suite that runs through the installed OpenShell CLI, and runs it on Docker rootful, Podman rootful, and Podman rootless in CI.

This is the last of four stacked PRs split out of #3801: #3979 (remove unreachable sandbox code) ← #3981 (Podman workspace volume ownership) ← #3982 (Podman OCI WORKDIR) ← this PR. The fixes come first so each PR passes its own tests. The suite checks behavior that #3981 and #3982 add on Podman, and the WorkspaceValidationFailed reason that #3982 adds.

Why this matters

Until now, a Docker-only e2e test checked what happens when an image chooses a user and a working directory. It could not catch a Podman regression in the same behavior, and it did not exercise the installed release artifacts. The new suite builds test images beside the gateway and checks that the sandbox starts as the right user, can write in its workspace, preserves image files, transfers files to the right place, and refuses a directory the user cannot write.

Related Issue

Related to #2526 (fixed by #3982). This responds to review point 3 on #3801 (run shared OCI checks against installed artifacts on each runtime) and to review feedback on this PR.

Changes

New tests/suites/features/oci-image crate with four scenarios that run through the candidate CLI:
- a named USER with a custom WORKDIR: identity, working directory, root-owned image content left unchanged, writes from the main process and sandbox exec, upload/download relative to the workspace, and a directory upload that merges into an existing directory
- a numeric USER whose WORKDIR parents are private (0700) to that user. The scenario supplies a policy with no process section, so the image USER is the only source of the sandbox identity (moved here from podman_oci_identity).
- no WORKDIR and no /sandbox in the image: falls back to the managed /sandbox workspace
- a WORKDIR the image user can't write: sandbox creation fails with WorkspaceValidationFailed before the command runs
tmachine wiring: oci-image testsuite in tests/config.nix, ociImageArchive / build-oci-image-test-archive in tests/artifacts.nix, and the tests/ansible/playbooks/features/oci-image.yaml playbook. The playbook picks docker, podman, or sudo -n podman so images are built into the store the gateway reads.
Add oci-image on ubuntu-docker-rootful, fedora-podman-rootful, and fedora-podman-rootless to the feature-specific matrix in branch-e2e.yml, release-dev.yml, and release-tag.yml.
Delete e2e/rust/tests/custom_image.rs and its [[test]] entry; the suite covers everything it checked, including the directory-merge upload.
Trim podman_oci_identity to the checks only Podman can make: the pinned image ID and the isolated workload/supervisor container pair (users, capabilities, networking, mounts). The shared suite now covers the identity that the main process and sandbox exec see, on both runtimes.
Document the suite in TESTING.md.

Testing

mise run pre-commit passes
cargo fmt --check and cargo clippy --all-targets -D warnings pass for openshell-test-feature-oci-image; openshell-e2e still builds after removing custom_image
tests/artifacts.nix and tests/config.nix parse (nix-instantiate --parse). I couldn't evaluate the flake locally because fetching flake inputs failed TLS verification in this environment.
The suite passes 4/4 against a local rootful Podman gateway (e2e/with-podman-gateway.sh), including the WorkspaceValidationFailed assertion. The trimmed podman_oci_identity passes.
Docker run of the suite: there's no local Docker daemon on the machine I used, so the ubuntu-docker-rootful / oci-image CI lane is the Docker check.

Checklist

Follows Conventional Commits
Commits are signed off (DCO)
Architecture docs updated (if applicable) — TESTING.md

copy-pr-bot · 2026-09-30T06:01:34Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

github-actions · 2026-09-30T06:02:11Z

🌿 Preview your docs: https://nvidia-preview-pr-3931.docs.buildwithfern.com/openshell

matthewgrossman · 2026-09-30T06:12:43Z

/ok to test 2fed032

github-actions · 2026-09-30T06:16:33Z

Label test:e2e applied for 2fed032. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

elezar · 2026-09-30T07:13:07Z

This is great! Thanks.

One expectation I had (related to #3712) would be that at least some of the tests in podman_oci_identity.rs would also move / be replaced by the new feature-specific tests. My agent seems to think that there may still be some tests there that are driver-agnostic (at least for Docker and Podman) in their intent -- even though they may rely on runtime specifics to verify behaviour.

I don't think that that file should be removed entirely because it also includes tests for other behaviour.

elezar · 2026-09-30T07:22:07Z

+            ])
+            .await
+            .map_err(|error| error.to_string())?;
+        if create.success() || create.stdout().contains("should-not-run") {


Any sandbox-creation failure currently passes this scenario, so an unrelated provisioning regression could produce a false success. Require the failure to identify workspace validation—preferably through a structured error, or otherwise through a stable WorkspaceValidationFailed/WorkingDir diagnostic.

elezar · 2026-10-01T14:42:57Z

The shared Docker/Podman OCI suite would also be a useful place to pin the image identity contract discussed in #4030. Could we extend the scenarios to cover:

Explicit USER root, with no policy identity override: sandbox creation fails with IdentityResolutionFailed before the workload command runs.
No image USER and no policy identity: the workload runs as UID/GID 1000:1000.
Explicit USER root with a valid non-root policy user/group override: the workload runs as the policy-selected identity. Use a compatible workspace so this tests identity resolution independently of workspace permissions.

Each case should run on the existing Docker rootful, Podman rootful, and Podman rootless lanes. These assertions can cover the current behavior without waiting for #4030's diagnostic changes. Detailed rejection provenance belongs in resolver tests, and once-only interactive failure output needs a real PTY test alongside the #4030 fix.

Add an oci-image feature testsuite that runs against installed artifacts. It covers custom WORKDIR placement, image content and ownership, workspace writes from the main process and exec, file transfer including directory-merge upload, OCI user identity, the managed /sandbox fallback, and rejection of an unwritable WORKDIR. CI runs the suite on the Docker rootful, Podman rootful, and Podman rootless tmachine guests. Replace the Docker-only custom_image e2e with the shared suite. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

…lback The unwritable WORKDIR scenario now requires the WorkspaceValidationFailed reason, so unrelated provisioning failures cannot pass it. The numeric USER scenario supplies a policy without a process section, so the image USER is the only source of the sandbox identity. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

The oci-image feature suite covers the sandbox identity seen by the main process and sandbox exec on Docker and Podman. The Podman e2e keeps the checks only it can make: the pinned image ID and the isolated workload/supervisor container pair. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

matthewgrossman · 2026-10-01T19:15:57Z

/ok to test 7658c9f

This was referenced Sep 30, 2026

fix(podman): create managed workspace volumes owned by the workload identity #3932

Merged

feat(podman): honor OCI image working directories #3933

Merged

matthewgrossman added this pull request to stack #3936 September 30, 2026 06:06

matthewgrossman mentioned this pull request Sep 30, 2026

bug: rootful Podman managed workspace is unwritable for non-root workloads #3937

Open

4 tasks

elezar mentioned this pull request Sep 30, 2026

test(podman): run existing Podman behavioral e2e coverage in CI #3712

Open

9 tasks

matthewgrossman added the test:e2e Requires end-to-end coverage label Sep 30, 2026

elezar reviewed Sep 30, 2026

View reviewed changes

matthewgrossman changed the title ~~test(oci): share OCI image checks across Docker and Podman~~ fix(oci): share OCI image checks across runtimes and report unusable WORKDIRs Sep 30, 2026

matthewgrossman force-pushed the podman-workdir/a-oci-image-suite branch from 33f5e2d to 9bf8773 Compare September 30, 2026 18:33

matthewgrossman mentioned this pull request Sep 30, 2026

refactor(sandbox): remove unreachable root-side identity and workspace code #3979

Merged

7 of 8 tasks

matthewgrossman removed this pull request from stack #3936 September 30, 2026 18:46

matthewgrossman changed the base branch from main to podman-workdir/0-remove-dead-workspace-code September 30, 2026 18:52

This was referenced Sep 30, 2026

fix(podman): create managed workspace volumes owned by the workload identity #3981

Open

feat(podman): honor OCI image working directories #3982

Draft

matthewgrossman changed the base branch from podman-workdir/0-remove-dead-workspace-code to podman-workdir/c-oci-workdir September 30, 2026 18:53

matthewgrossman added this pull request to stack #3983 September 30, 2026 18:53

matthewgrossman changed the title ~~fix(oci): share OCI image checks across runtimes and report unusable WORKDIRs~~ test(oci): share OCI USER and WORKDIR checks across Docker and Podman Sep 30, 2026

matthewgrossman force-pushed the podman-workdir/a-oci-image-suite branch from 9bf8773 to e65a902 Compare September 30, 2026 19:54

matthewgrossman force-pushed the podman-workdir/a-oci-image-suite branch from e65a902 to f184d63 Compare September 30, 2026 23:20

elezar mentioned this pull request Oct 1, 2026

bug: sandbox from a USER root image fails with an unexplained "UID or GID zero" error #4030

Open

6 tasks

matthewgrossman added 2 commits October 1, 2026 12:14

matthewgrossman force-pushed the podman-workdir/a-oci-image-suite branch from f184d63 to 7658c9f Compare October 1, 2026 19:15

Navigation Menu

test(oci): share OCI USER and WORKDIR checks across Docker and Podman - #3931

test(oci): share OCI USER and WORKDIR checks across Docker and Podman#3931
matthewgrossman wants to merge 3 commits into
podman-workdir/c-oci-workdirfrom
podman-workdir/a-oci-image-suite

matthewgrossman commented Sep 30, 2026 •

edited

Loading

Uh oh!

copy-pr-bot Bot commented Sep 30, 2026

Uh oh!

github-actions Bot commented Sep 30, 2026

Uh oh!

matthewgrossman commented Sep 30, 2026

Uh oh!

github-actions Bot commented Sep 30, 2026

Uh oh!

elezar commented Sep 30, 2026

Uh oh!

elezar Sep 30, 2026 •

edited

Loading

Uh oh!

elezar commented Oct 1, 2026

Uh oh!

matthewgrossman commented Oct 1, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

matthewgrossman commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why this matters

Related Issue

Changes

Testing

Checklist

Uh oh!

copy-pr-bot Bot commented Sep 30, 2026

Uh oh!

github-actions Bot commented Sep 30, 2026

Uh oh!

matthewgrossman commented Sep 30, 2026

Uh oh!

github-actions Bot commented Sep 30, 2026

Uh oh!

elezar commented Sep 30, 2026

Uh oh!

elezar Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elezar commented Oct 1, 2026

Uh oh!

matthewgrossman commented Oct 1, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

matthewgrossman commented Sep 30, 2026 •

edited

Loading

elezar Sep 30, 2026 •

edited

Loading