Skip to content

Commit befdfeb

Browse files
refactor(podman): trim OCI workdir change set
Drop image-volume admission (split to a follow-up), the Docker and core mount-validation refactor, redundant per-arm Podman mount checks, and unrelated doc and test churn. Keep image VOLUME masking validation for custom workdirs. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
1 parent ba9645e commit befdfeb

11 files changed

Lines changed: 93 additions & 282 deletions

File tree

‎architecture/compute-runtimes.md‎

Lines changed: 20 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,10 @@ Podman provisions a paired workload and supervisor container using its native
99
libpod API. The workload uses `network=none`; the external supervisor alone joins
1010
the configured network. A per-sandbox named volume carries their mutually
1111
authenticated gRPC Unix socket, with supervisor credentials kept in its separate
12-
filesystem. The supervisor and final sandbox runtime run as the resolved non-root
13-
identity with all capabilities dropped. Only the managed `/sandbox` fallback uses
14-
a trusted root bootstrap to prepare its driver-owned workspace before dropping
15-
irreversibly to that identity. The containers do not share PID, mount, or network
16-
namespaces. Podman owns paired lifecycle and health; the common protocol owns
17-
process, identity, TCP, DNS, and forwarding semantics.
12+
filesystem. Both containers run as the resolved non-root identity with all
13+
capabilities dropped. They share only a user namespace for volume ownership,
14+
not PID, mount, or network namespaces. Podman owns paired lifecycle and health;
15+
the common protocol owns process, identity, TCP, DNS, and forwarding semantics.
1816

1917
## Driver Contract
2018

@@ -307,7 +305,7 @@ delete, reconciliation removes the row; otherwise it can remain `Deleting`.
307305
| Runtime | Best fit | Sandbox boundary | Notes |
308306
|---|---|---|---|
309307
| Docker | Local development with Docker available. | Capability-free workload container. | Uses `network_mode=none`; a separate capability-free supervisor container mediates egress and access over a private daemon-local Unix socket volume. |
310-
| Podman | Local development with Podman available. | Capability-free workload container. | Uses `network=none`; a separate capability-free supervisor container mediates egress and access over a private Unix socket volume. |
308+
| Podman | Existing rootless driver. | Container. | Not converted by this isolation stack. |
311309
| Kubernetes | Cluster deployment through Helm. | Capability-free sandbox Pod. | Always creates a namespace-wide empty-egress workload NetworkPolicy and a separate capability-free supervisor Pod over mutually authenticated TLS. It requires an enforcing CNI and trusted sandbox namespace; the Kubernetes API does not attest policy enforcement. |
312310
| VM | Experimental microVM isolation. | Per-sandbox libkrun or QEMU VM. | The NIC-less guest runs `openshell-sandbox` as PID 1; host `openshell-supervisor` owns gateway networking and reaches the guest over vsock. |
313311
| Extension | Out-of-tree drivers operated alongside the gateway. | Whatever boundary the driver implements. | Selected by a custom `compute_drivers = ["<name>"]` entry with `[openshell.drivers.<name>].socket_path`, or at launch time by pairing `--drivers <name>` with `--compute-driver-socket=<path>`. A launch-time endpoint may use a canonical built-in name to preserve its driver-config key while replacing in-process construction. The gateway connects to an operator-provisioned UDS, snapshots `GetCapabilities`, and dispatches all sandbox lifecycle calls through `compute_driver.proto`. The driver process and socket lifecycle are operator-owned; the gateway does not spawn, supervise, or remove unmanaged extension drivers. The trust boundary is the socket's filesystem permissions: the operator must ensure only the gateway uid can read/write it. |
@@ -392,7 +390,7 @@ Drivers deliver the two binaries to separate trust domains:
392390
| Runtime | Delivery model |
393391
|---|---|
394392
| Docker | A digest-pinned daemon-local volume supplies `openshell-sandbox`; the companion image runs `openshell-supervisor`. |
395-
| Podman | The driver pins `sandbox_runtime_image` and `supervisor_image` to image IDs. The former supplies `openshell-sandbox`; the latter is passed as the companion container's image and runs `openshell-supervisor`. |
393+
| Podman | Existing driver behavior; not converted by this stack. |
396394
| Kubernetes | A non-root init container stages `openshell-sandbox` into a memory volume; a directly managed Pod runs `openshell-supervisor`. |
397395
| VM | `openshell-sandbox` is embedded in the guest rootfs; a separately digest-checked native `openshell-supervisor` runs on the host. |
398396
| Extension | Defined by the out-of-tree driver. |
@@ -408,33 +406,24 @@ The gateway preserves whether each policy process field was omitted and passes
408406
the admitted selectors to the driver. The driver resolves one exact UID, GID,
409407
and supplementary-group set before creating the immutable workload:
410408

411-
- Docker pins the image ID, resolves policy selectors against the image's
412-
`/etc/passwd` and `/etc/group`, and validates its OCI working directory.
413-
- Podman pins the image ID, resolves policy selectors against the image's
414-
`/etc/passwd` and `/etc/group`, and validates its OCI working directory.
409+
- Docker and Podman pin the image ID, resolve policy selectors against the
410+
image's `/etc/passwd` and `/etc/group`, and validate its OCI working
411+
directory.
415412
- Kubernetes uses platform-resolved numeric values, including OpenShift
416413
namespace ranges.
417414
- VM uses the configured numeric guest identity.
418415

419-
UID/GID zero and `u32::MAX` are invalid. Agent commands run as the resolved
420-
non-root user. For Podman's managed `/sandbox` workspace, trusted setup briefly
421-
starts as root to prepare the workspace, then switches to that user before
422-
reading bootstrap material or accepting commands. Identity-changing policy
423-
updates require sandbox recreation, while other policy updates remain live.
424-
425-
Docker uses an absolute OCI working directory as the workspace. Empty, root,
426-
and explicit `/sandbox` values select `/sandbox`; other paths must already
427-
exist without symlink or reserved-mount collisions and must be usable by the
428-
resolved identity.
429-
430-
Podman reads the image user and working directory from one pinned image. Empty,
431-
`/`, and explicit `/sandbox` values use the managed `/sandbox` workspace. A
432-
custom path must be absolute, normalized, and outside system and OpenShell
433-
reserved paths. Image and driver mounts cannot cover the workspace. Custom
434-
paths use the image's container filesystem directly; the final non-root user
435-
must be able to write the existing directory. The managed `/sandbox` fallback
436-
uses a driver-owned workspace volume.
437-
Kubernetes and VM use `/sandbox`.
416+
UID/GID zero and `u32::MAX` are invalid. The sandbox and every child start with
417+
the resolved identity and zero capability masks; neither process performs an
418+
in-workload UID transition. Identity-changing policy updates require sandbox
419+
recreation, while other policy updates remain live.
420+
421+
Docker and Podman use an absolute OCI working directory as the workspace.
422+
Empty, root, and explicit `/sandbox` values select `/sandbox`; other paths must
423+
already exist without symlink or reserved-mount collisions and must be usable
424+
by the resolved identity. Podman keeps a custom workspace in the container
425+
filesystem and uses a driver-owned volume only for `/sandbox`. Kubernetes and
426+
VM use `/sandbox`.
438427

439428
### Executable Identity Binding
440429

‎crates/openshell-core/src/driver_mounts.rs‎

Lines changed: 0 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -98,18 +98,6 @@ pub fn validate_container_mount_target(target: &str) -> Result<(), String> {
9898
Ok(())
9999
}
100100

101-
/// Validate a mount target against shared and driver-specific control paths.
102-
pub fn validate_container_mount_target_with_control_paths(
103-
target: &str,
104-
control_paths: &[&str],
105-
) -> Result<(), String> {
106-
validate_container_mount_target(target)?;
107-
for control_path in control_paths {
108-
validate_mount_control_path(target, control_path)?;
109-
}
110-
Ok(())
111-
}
112-
113101
/// Resolve an OCI image working directory to the internal workspace root used
114102
/// by local container drivers.
115103
///
@@ -372,27 +360,6 @@ mod tests {
372360
validate_mount_control_path("/custom-other", "/custom/ssh.sock").unwrap();
373361
}
374362

375-
#[test]
376-
fn container_target_checks_shared_and_driver_control_paths() {
377-
let control_paths = &["/.openshell/channel"];
378-
assert!(
379-
validate_container_mount_target_with_control_paths(
380-
"/etc/openshell/tls/client",
381-
control_paths,
382-
)
383-
.is_err()
384-
);
385-
assert!(
386-
validate_container_mount_target_with_control_paths(
387-
"/.openshell/channel/sandbox",
388-
control_paths,
389-
)
390-
.is_err()
391-
);
392-
validate_container_mount_target_with_control_paths("/workspace/cache", control_paths)
393-
.unwrap();
394-
}
395-
396363
#[test]
397364
fn workspace_rejects_malformed_runtime_control_paths() {
398365
for control_path in [

‎crates/openshell-core/src/resource_admission.rs‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,6 @@ impl std::str::FromStr for DriverAdmissionConfig {
9393
/// Reserved driver-owned runtime metadata; caller labels must never override it.
9494
pub const CONFIG_USED_LABEL: &str = "openshell.ai/caller-driver-config-used";
9595
pub const IDENTITIES_LABEL: &str = "openshell.ai/resource-admission-identities";
96-
pub const PRIVATE_IMAGE_VOLUME_TARGETS_LABEL: &str = "openshell.ai/private-image-volume-targets";
9796

9897
pub fn check_config_provenance(allowed: bool, recorded: Option<&str>) -> Result<(), tonic::Status> {
9998
match recorded {

‎crates/openshell-driver-docker/src/lib.rs‎

Lines changed: 7 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1215,7 +1215,7 @@ impl DockerComputeDriver {
12151215
.ok_or_else(|| Status::failed_precondition("sandbox lacks resource identity record"))?;
12161216
let mut actual = std::collections::BTreeMap::new();
12171217
let anonymous_targets: Vec<String> = labels
1218-
.get(openshell_core::resource_admission::PRIVATE_IMAGE_VOLUME_TARGETS_LABEL)
1218+
.get("openshell.ai/private-image-volume-targets")
12191219
.and_then(|value| serde_json::from_str(value).ok())
12201220
.unwrap_or_default();
12211221
for mount in container.mounts.as_deref().unwrap_or_default() {
@@ -5657,11 +5657,7 @@ fn build_container_create_body_for_image(
56575657
driver_mounts::validate_workspace_control_path(&workspace_root, BOUNDARY_MOUNT_PATH)
56585658
.map_err(Status::failed_precondition)?;
56595659
for volume in &image.volumes {
5660-
driver_mounts::validate_container_mount_target_with_control_paths(
5661-
volume,
5662-
&[BOUNDARY_MOUNT_PATH],
5663-
)
5664-
.map_err(|error| {
5660+
driver_mounts::validate_container_mount_target(volume).map_err(|error| {
56655661
Status::failed_precondition(format!(
56665662
"invalid image-declared volume '{volume}': {error}"
56675663
))
@@ -5671,6 +5667,8 @@ fn build_container_create_body_for_image(
56715667
"image-declared volume '{volume}' masks OCI WorkingDir '{workspace_root}' before workspace validation"
56725668
))
56735669
})?;
5670+
driver_mounts::validate_mount_control_path(volume, BOUNDARY_MOUNT_PATH)
5671+
.map_err(Status::failed_precondition)?;
56745672
}
56755673
for mount in &driver_config.mounts {
56765674
let target = match mount {
@@ -5681,11 +5679,8 @@ fn build_container_create_body_for_image(
56815679
};
56825680
driver_mounts::validate_workspace_mount_target(target, &workspace_root)
56835681
.map_err(Status::failed_precondition)?;
5684-
driver_mounts::validate_container_mount_target_with_control_paths(
5685-
target,
5686-
&[BOUNDARY_MOUNT_PATH],
5687-
)
5688-
.map_err(Status::failed_precondition)?;
5682+
driver_mounts::validate_mount_control_path(target, BOUNDARY_MOUNT_PATH)
5683+
.map_err(Status::failed_precondition)?;
56895684
}
56905685
let mut user_mounts = docker_driver_mounts(driver_config)?;
56915686
user_mounts.push(Mount {
@@ -5709,7 +5704,7 @@ fn build_container_create_body_for_image(
57095704
});
57105705
let mut labels = template.labels.clone();
57115706
labels.insert(
5712-
openshell_core::resource_admission::PRIVATE_IMAGE_VOLUME_TARGETS_LABEL.into(),
5707+
"openshell.ai/private-image-volume-targets".into(),
57135708
serde_json::to_string(&image.volumes)
57145709
.map_err(|error| Status::internal(error.to_string()))?,
57155710
);

‎crates/openshell-driver-podman/README.md‎

Lines changed: 17 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -37,12 +37,13 @@ The supervisor joins the workload's **user namespace only** to preserve UID/GID
3737
mapping for shared-volume access. PID, mount, and network namespaces remain
3838
separate. The channel volume uses shared SELinux relabeling (`:z`).
3939

40-
Before starting either container, the driver uploads bootstrap files to the
41-
channel volume. For managed `/sandbox`, it also prepares the workspace volume;
42-
custom image workspaces need no upload. Restart restores only the channel
43-
bootstrap, preserving the workspace. The workload starts before the supervisor
44-
so its user namespace exists when the supervisor joins it; a stopped supervisor
45-
resolves that namespace again on its next start.
40+
Before starting either container, the driver uploads volume-relative archives
41+
directly to the channel and workspace volume destinations. A rootfs upload on a
42+
stopped Podman container does not populate nested named volumes. Restart restores
43+
only the channel bootstrap into the existing channel volume, preserving the
44+
workspace. The workload starts before the supervisor so its user namespace exists
45+
when the supervisor joins it; a stopped supervisor resolves that namespace again
46+
on its next start. Custom image workspaces have no workspace volume or upload.
4647

4748
The runtime must pass the sandbox's unprivileged enforcement probe, including
4849
nested seccomp notification and Landlock. Unsupported runtime defaults fail
@@ -92,22 +93,16 @@ environment belong to agent children, never the supervisor process.
9293
## OCI working directory
9394

9495
OpenShell reads `WORKDIR` from the workload image. If it is unset, `/`, or
95-
`/sandbox`, OpenShell uses its managed `/sandbox` workspace. A custom path must
96-
be absolute, with no `.` or `..` segments. It cannot overlap `/proc`, `/sys`,
96+
`/sandbox`, OpenShell uses its managed `/sandbox` workspace volume. A custom
97+
path must be absolute and normalized, and cannot overlap `/proc`, `/sys`,
9798
`/dev`, OpenShell-reserved paths, or the workload's private control and CA
98-
mounts. For a custom path, image volumes and driver mounts cannot cover the
99-
workspace or one of its parents; mounts nested below it remain valid. Podman
100-
creates a private volume for each image-declared path nested below it.
99+
mounts. Image volumes and driver mounts cannot cover it; mounts nested below it
100+
remain valid.
101101

102-
For a custom path, Podman uses the image's container filesystem directly.
103-
OpenShell keeps the image directory's ownership and permissions, starts as the
104-
final non-root user, and rejects the image if that user cannot reach and write
105-
the directory. Agent commands use the path as their working directory;
106-
`filesystem.include_workdir` grants access to it when enabled.
107-
108-
For `/sandbox`, OpenShell creates a workspace volume and prepares it before
109-
switching to the non-root user. The separate supervisor receives the path but
110-
does not mount the workspace.
102+
A custom path stays in the image's container filesystem with its ownership and
103+
permissions. The workload starts as the final non-root user, which must be able
104+
to reach and write the directory. Agent commands use the path as their working
105+
directory.
111106

112107
## Lifecycle and readiness
113108

@@ -135,8 +130,8 @@ User `bind`, `volume`, `tmpfs`, and `image` mounts and CDI GPU selection remain
135130
native Podman features and apply only to the workload. Bind mounts require the
136131
operator's `enable_bind_mounts` opt-in and disabled label admission. Supplemental
137132
image mounts also require disabled admission. Driver JSON requires
138-
`allow_driver_config = true`. The workload's private mounts cannot be
139-
replaced. User-owned volumes are never created or deleted.
133+
`allow_driver_config = true`. Reserved control paths and the workspace
134+
root cannot be replaced. User-owned volumes are never created or deleted.
140135

141136
See [gateway configuration](../../docs/how-it-works/gateways/configuration.mdx) for
142137
operator settings and [NETWORKING.md](NETWORKING.md) for supervisor networking.

0 commit comments

Comments
 (0)