diff --git a/architecture/compute-runtimes.md b/architecture/compute-runtimes.md index ec6fff115a..95736400bc 100644 --- a/architecture/compute-runtimes.md +++ b/architecture/compute-runtimes.md @@ -9,10 +9,12 @@ Podman provisions a paired workload and supervisor container using its native libpod API. The workload uses `network=none`; the external supervisor alone joins the configured network. A per-sandbox named volume carries their mutually authenticated gRPC Unix socket, with supervisor credentials kept in its separate -filesystem. Both containers run as the resolved non-root identity with all -capabilities dropped. They share only a user namespace for volume ownership, -not PID, mount, or network namespaces. Podman owns paired lifecycle and health; -the common protocol owns process, identity, TCP, DNS, and forwarding semantics. +filesystem. The supervisor and final sandbox runtime run as the resolved non-root +identity with all capabilities dropped. Only the managed `/sandbox` fallback uses +a trusted root bootstrap to prepare its driver-owned workspace before dropping +irreversibly to that identity. The containers do not share PID, mount, or network +namespaces. Podman owns paired lifecycle and health; the common protocol owns +process, identity, TCP, DNS, and forwarding semantics. ## Driver Contract @@ -305,7 +307,7 @@ delete, reconciliation removes the row; otherwise it can remain `Deleting`. | Runtime | Best fit | Sandbox boundary | Notes | |---|---|---|---| | Docker | Local development with Docker available. | Capability-free workload container. | Uses `network_mode=none`; a separate capability-free supervisor container mediates egress and access over a private daemon-local Unix socket volume. | -| Podman | Existing rootless driver. | Container. | Not converted by this isolation stack. | +| Podman | Local development with Podman available. | Capability-free workload container. | Uses `network=none`; a separate capability-free supervisor container mediates egress and access over a private Unix socket volume. | | Kubernetes | Cluster deployment through Helm. | Capability-free sandbox Pod. | Always creates a namespace-wide empty-egress workload NetworkPolicy and a separate capability-free supervisor Pod over mutually authenticated TLS. It requires an enforcing CNI and trusted sandbox namespace; the Kubernetes API does not attest policy enforcement. | | VM | Experimental microVM isolation. | Per-sandbox libkrun or QEMU VM. | The NIC-less guest runs `openshell-sandbox` as PID 1; host `openshell-supervisor` owns gateway networking and reaches the guest over vsock. | | Extension | Out-of-tree drivers operated alongside the gateway. | Whatever boundary the driver implements. | Selected by a custom `compute_drivers = [""]` entry with `[openshell.drivers.].socket_path`, or at launch time by pairing `--drivers ` with `--compute-driver-socket=`. A launch-time endpoint may use a canonical built-in name to preserve its driver-config key while replacing in-process construction. The gateway connects to an operator-provisioned UDS, snapshots `GetCapabilities`, and dispatches all sandbox lifecycle calls through `compute_driver.proto`. The driver process and socket lifecycle are operator-owned; the gateway does not spawn, supervise, or remove unmanaged extension drivers. The trust boundary is the socket's filesystem permissions: the operator must ensure only the gateway uid can read/write it. | @@ -390,7 +392,7 @@ Drivers deliver the two binaries to separate trust domains: | Runtime | Delivery model | |---|---| | Docker | A digest-pinned daemon-local volume supplies `openshell-sandbox`; the companion image runs `openshell-supervisor`. | -| Podman | Existing driver behavior; not converted by this stack. | +| Podman | The driver pins `sandbox_runtime_image` and `supervisor_image` to image IDs. The former supplies `openshell-sandbox`; the latter is passed as the companion container's image and runs `openshell-supervisor`. | | Kubernetes | A non-root init container stages `openshell-sandbox` into a memory volume; a directly managed Pod runs `openshell-supervisor`. | | VM | `openshell-sandbox` is embedded in the guest rootfs; a separately digest-checked native `openshell-supervisor` runs on the host. | | Extension | Defined by the out-of-tree driver. | @@ -408,19 +410,31 @@ and supplementary-group set before creating the immutable workload: - Docker pins the image ID, resolves policy selectors against the image's `/etc/passwd` and `/etc/group`, and validates its OCI working directory. +- Podman pins the image ID, resolves policy selectors against the image's + `/etc/passwd` and `/etc/group`, and validates its OCI working directory. - Kubernetes uses platform-resolved numeric values, including OpenShift namespace ranges. - VM uses the configured numeric guest identity. -UID/GID zero and `u32::MAX` are invalid. The sandbox and every child start with -the resolved identity and zero capability masks; neither process performs an -in-workload UID transition. Identity-changing policy updates require sandbox -recreation, while other policy updates remain live. +UID/GID zero and `u32::MAX` are invalid. Agent commands run as the resolved +non-root user. For Podman's managed `/sandbox` workspace, trusted setup briefly +starts as root to prepare the workspace, then switches to that user before +reading bootstrap material or accepting commands. Identity-changing policy +updates require sandbox recreation, while other policy updates remain live. Docker uses an absolute OCI working directory as the workspace. Empty, root, and explicit `/sandbox` values select `/sandbox`; other paths must already exist without symlink or reserved-mount collisions and must be usable by the -resolved identity. Kubernetes and VM use `/sandbox`. +resolved identity. + +Podman reads the image user and working directory from one pinned image. Empty, +`/`, and explicit `/sandbox` values use the managed `/sandbox` workspace. A +custom path must be absolute, normalized, and outside system and OpenShell +reserved paths. Image and driver mounts cannot cover the workspace. Podman +mounts the persistent workspace volume there and copies any existing image +files into it on first use. OpenShell preserves their ownership and permissions; +the final non-root user must be able to write the resulting workspace. +Kubernetes and VM use `/sandbox`. ### Executable Identity Binding diff --git a/crates/openshell-core/src/driver_mounts.rs b/crates/openshell-core/src/driver_mounts.rs index b1a3049882..2cce47c63f 100644 --- a/crates/openshell-core/src/driver_mounts.rs +++ b/crates/openshell-core/src/driver_mounts.rs @@ -98,6 +98,18 @@ pub fn validate_container_mount_target(target: &str) -> Result<(), String> { Ok(()) } +/// Validate a mount target against shared and driver-specific control paths. +pub fn validate_container_mount_target_with_control_paths( + target: &str, + control_paths: &[&str], +) -> Result<(), String> { + validate_container_mount_target(target)?; + for control_path in control_paths { + validate_mount_control_path(target, control_path)?; + } + Ok(()) +} + /// Resolve an OCI image working directory to the internal workspace root used /// by local container drivers. /// @@ -360,6 +372,27 @@ mod tests { validate_mount_control_path("/custom-other", "/custom/ssh.sock").unwrap(); } + #[test] + fn container_target_checks_shared_and_driver_control_paths() { + let control_paths = &["/.openshell/channel"]; + assert!( + validate_container_mount_target_with_control_paths( + "/etc/openshell/tls/client", + control_paths, + ) + .is_err() + ); + assert!( + validate_container_mount_target_with_control_paths( + "/.openshell/channel/sandbox", + control_paths, + ) + .is_err() + ); + validate_container_mount_target_with_control_paths("/workspace/cache", control_paths) + .unwrap(); + } + #[test] fn workspace_rejects_malformed_runtime_control_paths() { for control_path in [ diff --git a/crates/openshell-core/src/resource_admission.rs b/crates/openshell-core/src/resource_admission.rs index 7d20c405f4..b0229e0142 100644 --- a/crates/openshell-core/src/resource_admission.rs +++ b/crates/openshell-core/src/resource_admission.rs @@ -93,6 +93,7 @@ impl std::str::FromStr for DriverAdmissionConfig { /// Reserved driver-owned runtime metadata; caller labels must never override it. pub const CONFIG_USED_LABEL: &str = "openshell.ai/caller-driver-config-used"; pub const IDENTITIES_LABEL: &str = "openshell.ai/resource-admission-identities"; +pub const PRIVATE_IMAGE_VOLUME_TARGETS_LABEL: &str = "openshell.ai/private-image-volume-targets"; pub fn check_config_provenance(allowed: bool, recorded: Option<&str>) -> Result<(), tonic::Status> { match recorded { diff --git a/crates/openshell-driver-docker/src/lib.rs b/crates/openshell-driver-docker/src/lib.rs index 1f4c91f5e4..831acd3f3f 100644 --- a/crates/openshell-driver-docker/src/lib.rs +++ b/crates/openshell-driver-docker/src/lib.rs @@ -1215,7 +1215,7 @@ impl DockerComputeDriver { .ok_or_else(|| Status::failed_precondition("sandbox lacks resource identity record"))?; let mut actual = std::collections::BTreeMap::new(); let anonymous_targets: Vec = labels - .get("openshell.ai/private-image-volume-targets") + .get(openshell_core::resource_admission::PRIVATE_IMAGE_VOLUME_TARGETS_LABEL) .and_then(|value| serde_json::from_str(value).ok()) .unwrap_or_default(); for mount in container.mounts.as_deref().unwrap_or_default() { @@ -5657,7 +5657,11 @@ fn build_container_create_body_for_image( driver_mounts::validate_workspace_control_path(&workspace_root, BOUNDARY_MOUNT_PATH) .map_err(Status::failed_precondition)?; for volume in &image.volumes { - driver_mounts::validate_container_mount_target(volume).map_err(|error| { + driver_mounts::validate_container_mount_target_with_control_paths( + volume, + &[BOUNDARY_MOUNT_PATH], + ) + .map_err(|error| { Status::failed_precondition(format!( "invalid image-declared volume '{volume}': {error}" )) @@ -5667,8 +5671,6 @@ fn build_container_create_body_for_image( "image-declared volume '{volume}' masks OCI WorkingDir '{workspace_root}' before workspace validation" )) })?; - driver_mounts::validate_mount_control_path(volume, BOUNDARY_MOUNT_PATH) - .map_err(Status::failed_precondition)?; } for mount in &driver_config.mounts { let target = match mount { @@ -5679,8 +5681,11 @@ fn build_container_create_body_for_image( }; driver_mounts::validate_workspace_mount_target(target, &workspace_root) .map_err(Status::failed_precondition)?; - driver_mounts::validate_mount_control_path(target, BOUNDARY_MOUNT_PATH) - .map_err(Status::failed_precondition)?; + driver_mounts::validate_container_mount_target_with_control_paths( + target, + &[BOUNDARY_MOUNT_PATH], + ) + .map_err(Status::failed_precondition)?; } let mut user_mounts = docker_driver_mounts(driver_config)?; user_mounts.push(Mount { @@ -5704,7 +5709,7 @@ fn build_container_create_body_for_image( }); let mut labels = template.labels.clone(); labels.insert( - "openshell.ai/private-image-volume-targets".into(), + openshell_core::resource_admission::PRIVATE_IMAGE_VOLUME_TARGETS_LABEL.into(), serde_json::to_string(&image.volumes) .map_err(|error| Status::internal(error.to_string()))?, ); diff --git a/crates/openshell-driver-podman/README.md b/crates/openshell-driver-podman/README.md index 133326a0ec..ffd4044944 100644 --- a/crates/openshell-driver-podman/README.md +++ b/crates/openshell-driver-podman/README.md @@ -90,6 +90,27 @@ binary extraction path. `supervisor_image` supplies the dynamically linked glibc `/openshell-supervisor` binary outside the workload. Image and request environment belong to agent children, never the supervisor process. +## OCI working directory + +OpenShell reads `WORKDIR` from the workload image. If it is unset, `/`, or +`/sandbox`, OpenShell uses its managed `/sandbox` workspace. A custom path must +be absolute, with no `.` or `..` segments. It cannot overlap `/proc`, `/sys`, +`/dev`, OpenShell-reserved paths, or the workload's private control and CA +mounts. For a custom path, image volumes and driver mounts cannot cover the +workspace or one of its parents; mounts nested below it remain valid. Podman +creates a private volume for each image-declared path nested below it. + +For a custom path, Podman mounts a persistent workspace volume there. When the +volume is first created, Podman copies any files already in that image directory +into it. OpenShell keeps their ownership and permissions, starts as the final +non-root user, and rejects the image if that user cannot reach and write the +directory. Agent commands use the path as their working directory and `HOME`; +`filesystem.include_workdir` grants access to it when enabled. + +For `/sandbox`, OpenShell prepares the managed workspace before switching to +the non-root user. The separate supervisor receives the path but does not mount +the workspace. Test custom images with the Podman configuration you will use. + ## Lifecycle and readiness Create builds both stopped containers and stages the private archives before @@ -116,8 +137,8 @@ User `bind`, `volume`, `tmpfs`, and `image` mounts and CDI GPU selection remain native Podman features and apply only to the workload. Bind mounts require the operator's `enable_bind_mounts` opt-in and disabled label admission. Supplemental image mounts also require disabled admission. Driver JSON requires -`allow_driver_config = true`. Reserved control paths and the workspace -root cannot be replaced. User-owned volumes are never created or deleted. +`allow_driver_config = true`. The workload's private mounts cannot be +replaced. User-owned volumes are never created or deleted. See [gateway configuration](../../docs/how-it-works/gateways/configuration.mdx) for operator settings and [NETWORKING.md](NETWORKING.md) for supervisor networking. diff --git a/crates/openshell-driver-podman/src/client.rs b/crates/openshell-driver-podman/src/client.rs index 80e24513a3..84301f85d1 100644 --- a/crates/openshell-driver-podman/src/client.rs +++ b/crates/openshell-driver-podman/src/client.rs @@ -185,6 +185,10 @@ pub struct ImageConfig { pub user: String, #[serde(default)] pub env: Vec, + #[serde(default)] + pub working_dir: String, + #[serde(default)] + pub volumes: Option>, } /// A container summary returned by the list API. @@ -1158,12 +1162,12 @@ mod tests { } #[tokio::test] - async fn inspect_image_reads_immutable_id_and_oci_user() { + async fn inspect_image_reads_immutable_id_and_oci_config() { let (socket_path, request_log, handle) = spawn_podman_stub( "inspect-image", vec![StubResponse::new( StatusCode::OK, - r#"{"Id":"sha256:immutable","Config":{"User":"app:staff"}}"#, + r#"{"Id":"sha256:immutable","Config":{"User":"app:staff","Env":["A=one"],"WorkingDir":"/workspace/project","Volumes":{"/workspace/project/cache":{}}}}"#, )], ); let client = PodmanClient::new(socket_path.clone()); @@ -1178,6 +1182,20 @@ mod tests { image.config.as_ref().map(|config| config.user.as_str()), Some("app:staff") ); + assert_eq!( + image + .config + .as_ref() + .map(|config| config.working_dir.as_str()), + Some("/workspace/project") + ); + assert!( + image + .config + .as_ref() + .and_then(|config| config.volumes.as_ref()) + .is_some_and(|volumes| volumes.contains_key("/workspace/project/cache")) + ); handle.await.expect("stub task should finish"); assert_eq!( request_log diff --git a/crates/openshell-driver-podman/src/container.rs b/crates/openshell-driver-podman/src/container.rs index bb4ee88c9f..106331c311 100644 --- a/crates/openshell-driver-podman/src/container.rs +++ b/crates/openshell-driver-podman/src/container.rs @@ -3,6 +3,7 @@ //! Container spec construction for the Podman driver. +use crate::client::ImageInspect; use crate::config::PodmanComputeConfig; use openshell_core::ComputeDriverError; use openshell_core::driver_mounts::SelinuxLabel; @@ -49,6 +50,10 @@ const CONTAINER_PREFIX: &str = "openshell-"; /// Volume name prefix. const VOLUME_PREFIX: &str = "openshell-sandbox-"; +const PODMAN_WORKLOAD_CONTROL_PATHS: &[&str] = &[ + "/.openshell", + openshell_sandbox_backend::SUPERVISOR_CA_RUNTIME_ROOT, +]; /// Secret name prefix for per-sandbox gateway JWTs. const TOKEN_SECRET_PREFIX: &str = "openshell-token-"; @@ -217,6 +222,72 @@ pub fn short_id(id: &str) -> String { id.chars().take(12).collect() } +/// Immutable OCI image metadata normalized once for final launch. +#[derive(Debug, Clone)] +pub struct ResolvedPodmanImage { + pub(crate) id: String, + pub(crate) oci_user: String, + pub(crate) environment: Vec, + pub(crate) workspace_root: String, + image_volume_targets: Vec, +} + +impl ResolvedPodmanImage { + /// Resolve the image metadata and reject: + /// - a malformed or reserved working directory; + /// - for a custom working directory, an image volume with an invalid or + /// reserved target, or one covering the resolved workspace. + pub fn from_inspect(inspected: &ImageInspect) -> Result { + let image_config = inspected.config.as_ref(); + let workspace_root = driver_mounts::resolve_oci_workspace_root( + image_config.map_or("", |config| config.working_dir.as_str()), + ) + .map_err(ComputeDriverError::Precondition)?; + for control_path in PODMAN_WORKLOAD_CONTROL_PATHS { + driver_mounts::validate_workspace_control_path(&workspace_root, control_path) + .map_err(ComputeDriverError::Precondition)?; + } + let mut image_volume_targets = Vec::new(); + if workspace_root != driver_mounts::DEFAULT_WORKSPACE_ROOT + && let Some(volumes) = image_config.and_then(|config| config.volumes.as_ref()) + { + for volume in volumes.keys() { + driver_mounts::validate_container_mount_target_with_control_paths( + volume, + PODMAN_WORKLOAD_CONTROL_PATHS, + ) + .map_err(|error| { + ComputeDriverError::Precondition(format!( + "invalid image-declared volume '{volume}': {error}" + )) + })?; + driver_mounts::validate_workspace_mount_target(volume, &workspace_root).map_err( + |_| { + ComputeDriverError::Precondition(format!( + "image-declared volume '{volume}' masks OCI WorkingDir '{workspace_root}' before workspace validation" + )) + }, + )?; + image_volume_targets.push(volume.clone()); + } + image_volume_targets.sort(); + } + Ok(Self { + id: inspected.id.clone(), + oci_user: image_config + .map_or("", |config| config.user.as_str()) + .to_string(), + environment: image_config.map_or_else(Vec::new, |config| config.env.clone()), + workspace_root, + image_volume_targets, + }) + } + + pub(crate) fn uses_managed_workspace(&self) -> bool { + self.workspace_root == driver_mounts::DEFAULT_WORKSPACE_ROOT + } +} + // --------------------------------------------------------------------------- // Typed container spec structs for the Podman libpod create API. // --------------------------------------------------------------------------- @@ -230,10 +301,11 @@ pub struct ContainerSpec { volumes: Vec, image_volumes: Vec, hostname: String, + /// Start trusted runtime binaries independently of the image-selected + /// workspace. The resolved workspace is applied to untrusted children. + work_dir: String, /// Overrides the image's ENTRYPOINT. In Podman's libpod API, `command` - /// only overrides CMD (appended as args to the entrypoint). We must set - /// `entrypoint` explicitly so the supervisor binary runs directly, - /// regardless of what ENTRYPOINT the sandbox image defines. + /// only overrides CMD (appended as args to the entrypoint). entrypoint: Vec, command: Vec, user: String, @@ -804,7 +876,10 @@ fn podman_user_mounts( None => {} } driver_mounts::validate_absolute_mount_source(&source, "bind source")?; - driver_mounts::validate_container_mount_target(&target)?; + driver_mounts::validate_container_mount_target_with_control_paths( + &target, + PODMAN_WORKLOAD_CONTROL_PATHS, + )?; result.mounts.push(Mount { kind: "bind".into(), source, @@ -820,7 +895,10 @@ fn podman_user_mounts( } => { reject_subpath(subpath.as_deref(), "podman volume mounts")?; driver_mounts::validate_mount_source(&source, "volume source")?; - driver_mounts::validate_container_mount_target(&target)?; + driver_mounts::validate_container_mount_target_with_control_paths( + &target, + PODMAN_WORKLOAD_CONTROL_PATHS, + )?; result.volumes.push(NamedVolume { name: source, dest: target, @@ -846,7 +924,10 @@ fn podman_user_mounts( { options.push(format!("mode={mode:o}")); } - driver_mounts::validate_container_mount_target(&target)?; + driver_mounts::validate_container_mount_target_with_control_paths( + &target, + PODMAN_WORKLOAD_CONTROL_PATHS, + )?; result.mounts.push(Mount { kind: "tmpfs".into(), source: "tmpfs".into(), @@ -862,7 +943,10 @@ fn podman_user_mounts( } => { reject_subpath(subpath.as_deref(), "podman image mounts")?; driver_mounts::validate_mount_source(&source, "image source")?; - driver_mounts::validate_container_mount_target(&target)?; + driver_mounts::validate_container_mount_target_with_control_paths( + &target, + PODMAN_WORKLOAD_CONTROL_PATHS, + )?; result.image_volumes.push(ImageVolume { source, destination: target, @@ -937,8 +1021,10 @@ fn validate_podman_driver_mounts( target } }; - driver_mounts::validate_container_mount_target(target)?; - driver_mounts::validate_mount_control_path(target, "/.openshell")?; + driver_mounts::validate_container_mount_target_with_control_paths( + target, + PODMAN_WORKLOAD_CONTROL_PATHS, + )?; let normalized_target = driver_mounts::normalize_mount_target(target); if !targets.insert(normalized_target.clone()) { return Err(format!( @@ -1062,14 +1148,17 @@ pub fn build_container_spec_with_token_and_gpu_devices( gpu_device_ids: Option<&[String]>, ) -> Result { let image = resolve_image(sandbox, config); + let resolved_image = ResolvedPodmanImage::from_inspect(&ImageInspect { + id: image.to_string(), + config: None, + })?; build_container_spec_for_image( sandbox, config, token_secret_name, gpu_device_ids, image, - image, - "", + &resolved_image, None, None, ) @@ -1083,8 +1172,7 @@ pub fn build_container_spec_for_image( token_secret_name: Option<&str>, gpu_device_ids: Option<&[String]>, requested_image: &str, - image_id: &str, - oci_user: &str, + image: &ResolvedPodmanImage, supervisor_bin_path: Option<&Path>, tls_secret_names: Option<&[String; 3]>, ) -> Result { @@ -1094,8 +1182,7 @@ pub fn build_container_spec_for_image( token_secret_name, gpu_device_ids, requested_image, - image_id, - oci_user, + image, supervisor_bin_path, tls_secret_names, )?) @@ -1109,15 +1196,14 @@ fn build_base_spec( token_secret_name: Option<&str>, gpu_device_ids: Option<&[String]>, requested_image: &str, - image_id: &str, - oci_user: &str, + image: &ResolvedPodmanImage, supervisor_bin_path: Option<&Path>, tls_secret_names: Option<&[String; 3]>, ) -> Result { let name = container_name(&sandbox.workspace, &sandbox.name, &sandbox.id); let vol = volume_name(&sandbox.id); - let env = build_env(sandbox, config, requested_image, oci_user)?; + let env = build_env(sandbox, config, requested_image, &image.oci_user)?; let mut labels = build_labels(sandbox); labels.insert( "openshell.ai/runtime-binary-source".into(), @@ -1128,6 +1214,26 @@ fn build_base_spec( let resource_limits = build_resource_limits(sandbox, config); let user_mounts = podman_user_mounts(sandbox, config.enable_bind_mounts) .map_err(ComputeDriverError::InvalidArgument)?; + for target in user_mounts + .mounts + .iter() + .map(|mount| mount.destination.as_str()) + .chain( + user_mounts + .volumes + .iter() + .map(|volume| volume.dest.as_str()), + ) + .chain( + user_mounts + .image_volumes + .iter() + .map(|volume| volume.destination.as_str()), + ) + { + driver_mounts::validate_workspace_mount_target(target, &image.workspace_root) + .map_err(ComputeDriverError::Precondition)?; + } if sandbox .spec .as_ref() @@ -1153,7 +1259,7 @@ fn build_base_spec( let mut volumes = vec![NamedVolume { name: vol, - dest: "/sandbox".into(), + dest: image.workspace_root.clone(), options: vec!["rw".into()], }]; volumes.extend(user_mounts.volumes); @@ -1168,15 +1274,12 @@ fn build_base_spec( }] }; image_volumes.extend(user_mounts.image_volumes); - let mut command = vec![ - "--workdir".to_string(), - driver_mounts::DEFAULT_WORKSPACE_ROOT.to_string(), - ]; + let mut command = vec!["--workdir".to_string(), image.workspace_root.clone()]; command.extend(upstream_proxy_cli_args(config)); let container_spec = ContainerSpec { name, - image: image_id.to_string(), + image: image.id.clone(), labels, env, volumes, @@ -1187,6 +1290,7 @@ fn build_base_spec( // /openshell-sandbox, so it appears at /opt/openshell/bin/openshell-sandbox. image_volumes, hostname: format!("sandbox-{}", sandbox.name), + work_dir: "/".into(), // Override the image's ENTRYPOINT so the supervisor binary runs // directly. Workload images can set // ENTRYPOINT ["/bin/bash"], and Podman's `command` field only @@ -1194,10 +1298,9 @@ fn build_base_spec( // Without this, the container would run the entrypoint binary with // the supervisor path as an argument instead of executing it directly. entrypoint: vec![SUPERVISOR_BINARY_PATH.into()], - // Keep Podman's existing /sandbox workspace contract explicit while - // the supervisor supports driver-selected workdirs. Operator-owned - // corporate proxy flags follow it; the workload command comes from - // the reserved environment variable. + // Pass the resolved workspace to the logical supervisor. Operator-owned + // corporate proxy flags follow it; the workload command comes from the + // reserved environment variable. command, // The paired builder supplies the immutable non-root identity. user: String::new(), @@ -1439,9 +1542,7 @@ pub struct IsolationSpecInput<'a> { pub resolver_secret: &'a str, pub gpu_devices: Option<&'a [String]>, pub requested_image: &'a str, - pub image_id: &'a str, - pub image_user: &'a str, - pub image_env: &'a [String], + pub image: &'a ResolvedPodmanImage, pub supervisor_bin: Option<&'a Path>, pub tls_secrets: Option<&'a [String; 3]>, pub identity: &'a openshell_isolation_interface::contract::ResolvedWorkloadIdentity, @@ -1478,8 +1579,7 @@ pub fn build_isolation_specs( input.token_secret, input.gpu_devices, input.requested_image, - input.image_id, - input.image_user, + input.image, input.supervisor_bin, input.tls_secrets, ) @@ -1492,13 +1592,21 @@ pub fn build_isolation_specs( workload .labels .insert(crate::isolation::LABEL_ROLE.into(), "sandbox".into()); + workload.labels.insert( + openshell_core::resource_admission::PRIVATE_IMAGE_VOLUME_TARGETS_LABEL.into(), + serde_json::to_string(&input.image.image_volume_targets) + .map_err(|error| ComputeDriverError::Message(error.to_string()))?, + ); workload.env = BTreeMap::new(); workload.unsetenv = input - .image_env + .image + .environment .iter() .filter_map(|entry| entry.split_once('=').map(|(key, _)| key.to_string())) .collect(); - if input.rootless || input.identity.source == "default" { + if input.image.uses_managed_workspace() + && (input.rootless || input.identity.source == "default") + { // Podman's archive endpoint leaves named-volume contents owned by // container root for rootless services and for a rootful USER-less // image's newly-created workspace. Start the trusted runtime as root @@ -1510,7 +1618,7 @@ pub fn build_isolation_specs( input.identity.uid.to_string(), input.identity.gid.to_string(), crate::isolation::BOOTSTRAP_PATH.into(), - driver_mounts::DEFAULT_WORKSPACE_ROOT.into(), + input.image.workspace_root.clone(), ]; workload.user = "0:0".into(); workload.groups.clear(); @@ -1791,11 +1899,28 @@ fn parse_memory_to_bytes(quantity: &str) -> Option { #[cfg(test)] mod tests { use super::*; + use crate::client::ImageConfig; use openshell_core::proto::compute::v1::{GpuResourceRequirements, ResourceRequirements}; static ENV_LOCK: std::sync::LazyLock> = std::sync::LazyLock::new(|| std::sync::Mutex::new(())); + fn resolved_image(id: &str, user: &str, working_dir: &str) -> ResolvedPodmanImage { + ResolvedPodmanImage::from_inspect(&ImageInspect { + id: id.to_string(), + config: Some(ImageConfig { + user: user.to_string(), + env: vec![ + "LD_PRELOAD=/hostile.so".into(), + "HTTP_PROXY=http://bypass".into(), + ], + working_dir: working_dir.to_string(), + volumes: None, + }), + }) + .unwrap() + } + #[test] fn isolated_pair_keeps_privileges_network_and_secrets_out_of_workload() { let sandbox = DriverSandbox { @@ -1816,10 +1941,7 @@ mod tests { "sha256:image".into(), ) .unwrap(); - let env = vec![ - "LD_PRELOAD=/hostile.so".into(), - "HTTP_PROXY=http://bypass".into(), - ]; + let image = resolved_image("sha256:image", "1000:1001", ""); let specs = build_isolation_specs(IsolationSpecInput { sandbox: &sandbox, config: &config, @@ -1827,9 +1949,7 @@ mod tests { resolver_secret: "resolver", gpu_devices: None, requested_image: "image:latest", - image_id: "sha256:image", - image_user: "1000:1001", - image_env: &env, + image: &image, supervisor_bin: None, tls_secrets: None, identity: &identity, @@ -1891,9 +2011,7 @@ mod tests { resolver_secret: "resolver", gpu_devices: None, requested_image: "image:latest", - image_id: "sha256:image", - image_user: "", - image_env: &env, + image: &resolved_image("sha256:image", "", ""), supervisor_bin: None, tls_secrets: None, identity: &default_identity, @@ -1911,6 +2029,38 @@ mod tests { driver_mounts::DEFAULT_WORKSPACE_ROOT, ] ); + let mut custom_image = resolved_image("sha256:image", "1000:1001", "/workspace/project"); + custom_image + .image_volume_targets + .push("/workspace/project/cache".into()); + let custom_specs = build_isolation_specs(IsolationSpecInput { + sandbox: &sandbox, + config: &config, + token_secret: Some("jwt"), + resolver_secret: "resolver", + gpu_devices: None, + requested_image: "image:latest", + image: &custom_image, + supervisor_bin: None, + tls_secrets: None, + identity: &identity, + rootless: true, + }) + .unwrap(); + assert_eq!(custom_specs.workload.user, "1000:1001"); + assert_eq!( + custom_specs + .workload + .labels + .get(openshell_core::resource_admission::PRIVATE_IMAGE_VOLUME_TARGETS_LABEL) + .map(String::as_str), + Some("[\"/workspace/project/cache\"]") + ); + assert!(custom_specs.workload.cap_add.is_empty()); + assert_eq!( + custom_specs.workload.command, + vec!["--bootstrap", crate::isolation::BOOTSTRAP_PATH] + ); let workload_json = serde_json::to_string(&specs.workload).unwrap(); assert!(workload_json.contains("\"apparmor_profile\":\"openshell-sandbox\"")); assert_eq!(specs.supervisor.healthconfig.test, vec!["NONE"]); @@ -1989,6 +2139,33 @@ mod tests { ); } + #[test] + fn resolved_image_rejects_volumes_covering_working_dir() { + let inspect = |volume: &str| ImageInspect { + id: "sha256:image".into(), + config: Some(ImageConfig { + working_dir: "/workspace/project".into(), + volumes: Some(std::collections::HashMap::from([( + volume.into(), + Value::Null, + )])), + ..Default::default() + }), + }; + + assert!(ResolvedPodmanImage::from_inspect(&inspect("/workspace")).is_err()); + let image = ResolvedPodmanImage::from_inspect(&inspect("/workspace/project/cache")) + .expect("image volumes nested below the workspace remain valid"); + assert_eq!(image.workspace_root, "/workspace/project"); + assert_eq!(image.image_volume_targets, vec!["/workspace/project/cache"]); + + let mut fallback = inspect("/etc/openshell"); + fallback.config.as_mut().unwrap().working_dir = "/sandbox".into(); + let fallback = ResolvedPodmanImage::from_inspect(&fallback) + .expect("existing /sandbox images keep their image-volume behavior"); + assert!(fallback.image_volume_targets.is_empty()); + } + fn json_struct(value: Value) -> prost_types::Struct { let Value::Object(object) = value else { panic!("expected JSON object"); @@ -2098,14 +2275,14 @@ mod tests { spec.environment.insert(key.to_string(), value.to_string()); } + let image = resolved_image("sha256:immutable", "app:staff", "/workspace/project"); let container = build_container_spec_for_image( &sandbox, &test_config(), None, None, "registry.example/app:latest", - "sha256:immutable", - "app:staff", + &image, None, None, ) @@ -2121,6 +2298,17 @@ mod tests { assert_eq!(container["image_pull_policy"].as_str(), Some("never")); assert_eq!(container["dns_search"], serde_json::json!([])); assert_eq!(container["dns_option"], serde_json::json!([])); + assert_eq!(container["work_dir"].as_str(), Some("/")); + assert_eq!( + container["command"], + serde_json::json!(["--workdir", "/workspace/project"]) + ); + assert!(container["volumes"].as_array().is_some_and(|volumes| { + volumes.iter().any(|volume| { + volume["name"].as_str() == Some("openshell-sandbox-test-id-workspace") + && volume["dest"].as_str() == Some("/workspace/project") + }) + })); assert_eq!( container["env"][openshell_core::sandbox_env::OCI_IMAGE_USER].as_str(), Some("app:staff") @@ -2133,10 +2321,6 @@ mod tests { container["env"][openshell_core::sandbox_env::SANDBOX_GID].as_str(), Some("") ); - assert_eq!( - container["command"], - serde_json::json!(["--workdir", "/sandbox"]) - ); } #[test] @@ -2360,18 +2544,9 @@ mod tests { fn container_spec_defaults_drop_capabilities_and_keep_runtime_seccomp() { let sandbox = test_sandbox("test-id", "test-name"); let config = test_config(); - let spec = build_base_spec( - &sandbox, - &config, - None, - None, - "image", - "sha256:image", - "", - None, - None, - ) - .unwrap(); + let image = resolved_image("sha256:image", "", ""); + let spec = + build_base_spec(&sandbox, &config, None, None, "image", &image, None, None).unwrap(); assert_eq!(spec.cap_drop, vec!["ALL"]); assert!(spec.cap_add.is_empty()); assert!(spec.seccomp_profile_path.is_empty()); @@ -3081,6 +3256,39 @@ mod tests { ); } + #[test] + fn resolved_workspace_rejects_covering_driver_mount() { + use openshell_core::proto::compute::v1::{DriverSandboxSpec, DriverSandboxTemplate}; + + let image = resolved_image("sha256:immutable", "1000:1000", "/workspace/project"); + let mut sandbox = test_sandbox("test-id", "test-name"); + sandbox.spec = Some(DriverSandboxSpec { + template: Some(DriverSandboxTemplate { + driver_config: Some(json_struct(serde_json::json!({ + "mounts": [{"type": "tmpfs", "target": "/workspace"}] + }))), + ..Default::default() + }), + ..Default::default() + }); + let error = build_container_spec_for_image( + &sandbox, + &test_config(), + None, + None, + "image:latest", + &image, + None, + None, + ) + .unwrap_err(); + assert!( + error + .to_string() + .contains("reserved for the OpenShell workspace") + ); + } + #[test] fn driver_config_rejects_bind_mounts_unless_enabled() { use openshell_core::proto::compute::v1::{DriverSandboxSpec, DriverSandboxTemplate}; @@ -3303,7 +3511,7 @@ mod tests { "mounts": [{ "type": "volume", "source": "work-nfs", - "target": "/etc/openshell/tls/client" + "target": "/opt/openshell/bin" }] }))), ..Default::default() @@ -3315,6 +3523,19 @@ mod tests { let err = try_build_container_spec_with_token(&sandbox, &config, None).unwrap_err(); assert!(err.to_string().contains("reserved OpenShell path")); + + sandbox + .spec + .as_mut() + .unwrap() + .template + .as_mut() + .unwrap() + .driver_config = Some(json_struct(serde_json::json!({ + "mounts": [{"type": "volume", "source": "work-nfs", "target": "/etc/openshell/tls/client"}] + }))); + let err = try_build_container_spec_with_token(&sandbox, &config, None).unwrap_err(); + assert!(err.to_string().contains("reserved OpenShell path")); } #[test] @@ -3755,14 +3976,14 @@ mod tests { let sandbox = test_sandbox("bind-sv-id", "bind-sv-name"); let config = test_config(); let image = resolve_image(&sandbox, &config); + let resolved = resolved_image(image, "", ""); let spec = build_container_spec_for_image( &sandbox, &config, None, None, image, - image, - "", + &resolved, Some(Path::new("/host/cache/openshell-sandbox")), None, ) diff --git a/crates/openshell-driver-podman/src/driver.rs b/crates/openshell-driver-podman/src/driver.rs index 3788c756f4..20b7d90c37 100644 --- a/crates/openshell-driver-podman/src/driver.rs +++ b/crates/openshell-driver-podman/src/driver.rs @@ -761,6 +761,10 @@ impl PodmanComputeDriver { .get(openshell_core::resource_admission::IDENTITIES_LABEL) .and_then(|value| serde_json::from_str(value).ok()) .ok_or_else(missing)?; + let anonymous_targets: Vec = labels + .get(openshell_core::resource_admission::PRIVATE_IMAGE_VOLUME_TARGETS_LABEL) + .and_then(|value| serde_json::from_str(value).ok()) + .unwrap_or_default(); let mut actual = std::collections::BTreeMap::new(); for mount in mounts { match mount["Type"].as_str() { @@ -789,6 +793,16 @@ impl PodmanComputeDriver { if !owned || volume.driver != "local" || !volume.options.is_empty() { return Err(missing()); } + } else if !expected.contains_key(name) + && mount["Destination"].as_str().is_some_and(|destination| { + anonymous_targets.iter().any(|target| target == destination) + }) + { + if volume.driver != "local" || !volume.options.is_empty() { + return Err(ComputeDriverError::Precondition( + "image-private volume backing changed".into(), + )); + } } else { actual.insert(name.to_string(), volume.admission_identity()); self.config @@ -905,7 +919,7 @@ impl PodmanComputeDriver { "Creating sandbox container" ); - let (image, immutable_image_id, image_user, image_env) = async { + let (image, resolved_image) = async { let phase_status = openshell_otel::ErrorStatusGuard::current(); let result = async { // The sandbox runtime is shipped in a standalone OCI image. @@ -963,10 +977,8 @@ impl PodmanComputeDriver { "podman image '{image}' inspection did not return an immutable image ID" ))); } - let image_user = inspected_image - .config - .as_ref() - .map_or_else(String::new, |config| config.user.clone()); + let resolved_image = + container::ResolvedPodmanImage::from_inspect(&inspected_image)?; for mount_image in container::podman_driver_image_mount_sources( sandbox, @@ -981,8 +993,7 @@ impl PodmanComputeDriver { .map_err(ComputeDriverError::from)?; } - let image_env = inspected_image.config.as_ref().map_or_else(Vec::new, |config| config.env.clone()); - Ok((image.to_string(), inspected_image.id, image_user, image_env)) + Ok((image.to_string(), resolved_image)) } .await; phase_status.finish(result) @@ -1006,7 +1017,7 @@ impl PodmanComputeDriver { .map_err(ComputeDriverError::from)?; let identity = self - .resolve_workload_identity(sandbox, &immutable_image_id, &image_user) + .resolve_workload_identity(sandbox, &resolved_image.id, &resolved_image.oci_user) .await?; let channel_volume = crate::isolation::channel_volume_name(&sandbox.id); let mut runtime_config = self.config.clone(); @@ -1158,9 +1169,7 @@ impl PodmanComputeDriver { resolver_secret: &resolver_secret_name, gpu_devices: gpu_devices.as_deref(), requested_image: &image, - image_id: &immutable_image_id, - image_user: &image_user, - image_env: &image_env, + image: &resolved_image, supervisor_bin: supervisor_bin_path.as_deref(), tls_secrets: tls_secret_names.as_ref(), identity: &identity, @@ -1186,7 +1195,7 @@ impl PodmanComputeDriver { created_workload = Some(workload_id.clone()); self.client.verify_isolation_fence(&workload_id).await?; self.admit_container_resources(&workload_id).await?; - let child_env = podman_child_environment(sandbox, &image_env); + let child_env = podman_child_environment(sandbox, &resolved_image.environment); let launch_authentication = sandbox .spec .as_ref() @@ -1222,9 +1231,15 @@ impl PodmanComputeDriver { archives.channel, ) .await?; - self.client - .copy_to_container(&workload_id, "/sandbox", archives.workspace) - .await?; + if resolved_image.uses_managed_workspace() { + self.client + .copy_to_container( + &workload_id, + &resolved_image.workspace_root, + archives.workspace, + ) + .await?; + } let supervisor_id = self .client .create_typed_container(&specs.supervisor) @@ -3097,6 +3112,55 @@ mod tests { } } + #[tokio::test] + async fn admission_accepts_image_declared_volume_below_custom_workdir() { + let (socket, requests, handle) = spawn_podman_stub( + "image-volume-admission", + vec![ + StubResponse::new( + StatusCode::OK, + serde_json::json!({ + "Id": "workload-1", + "Name": "workload-1", + "State": {"Status": "created", "Running": false}, + "Config": {"Labels": { + "openshell.ai/sandbox-workspace": "team-a", + "openshell.ai/sandbox-id": "sandbox-1", + "openshell.ai/caller-driver-config-used": "false", + "openshell.ai/resource-admission-identities": "{}", + "openshell.ai/private-image-volume-targets": "[\"/home/app/project/cache\"]" + }}, + "Mounts": [{ + "Type": "volume", + "Name": "anonymous-1", + "Destination": "/home/app/project/cache" + }] + }) + .to_string(), + ), + StubResponse::new( + StatusCode::OK, + serde_json::json!({ + "Name": "anonymous-1", "Driver": "local", "Options": {}, "Labels": {} + }) + .to_string(), + ), + ], + ); + let driver = PodmanComputeDriver::for_tests(PodmanComputeConfig { + socket_path: Some(socket.clone()), + ..Default::default() + }); + + driver + .admit_container_resources("workload-1") + .await + .expect("a local image-declared volume is private to the workload"); + handle.await.unwrap(); + assert_eq!(requests.lock().unwrap().len(), 2); + let _ = fs::remove_file(socket); + } + #[tokio::test] async fn admission_driver_config_denial_does_not_contact_podman() { for enabled in [true, false] { diff --git a/docs/how-it-works/sandboxes/runtimes.mdx b/docs/how-it-works/sandboxes/runtimes.mdx index cd3d576b52..ead8b55179 100644 --- a/docs/how-it-works/sandboxes/runtimes.mdx +++ b/docs/how-it-works/sandboxes/runtimes.mdx @@ -267,4 +267,13 @@ Set `process.run_as_user` and `process.run_as_group` in the sandbox policy to ch | Kubernetes | OpenShift SCC namespace annotations, otherwise `1000`. Override with `sandbox_uid` and `sandbox_gid`. | | MicroVM | The image's `sandbox` account, otherwise `1000`. Override with `sandbox_uid` and `sandbox_gid`. | -On Docker, the image's `WORKDIR` becomes the workspace. Images with no `WORKDIR`, `/`, or `/sandbox` use `/sandbox`. Any other `WORKDIR` must exist in the image and be writable by the sandbox user. Podman, Kubernetes, and MicroVM always use `/sandbox`. +On Docker, the image's `WORKDIR` becomes the workspace. Images with no `WORKDIR`, `/`, or `/sandbox` use `/sandbox`. Any other `WORKDIR` must exist in the image and be writable by the sandbox user. + +Podman also uses the image's `WORKDIR` as the workspace, falling back to +`/sandbox` for an empty, `/`, or `/sandbox` value. A custom path must be an +absolute, normalized path outside system and OpenShell-reserved paths. +Image volumes and driver mounts cannot cover it. Podman mounts the persistent +workspace volume there and copies existing image-directory files into a new +volume. OpenShell preserves their permissions and requires the final non-root +user to be able to write there. Kubernetes and MicroVM continue to use +`/sandbox`. diff --git a/e2e/rust/e2e-podman.sh b/e2e/rust/e2e-podman.sh index bbbcfe6fa9..4d23fd831f 100755 --- a/e2e/rust/e2e-podman.sh +++ b/e2e/rust/e2e-podman.sh @@ -38,6 +38,7 @@ PODMAN_CI_TESTS=( podman_corporate_proxy podman_gateway_start podman_host_gateway + podman_oci_identity provider_token_exchange ) diff --git a/e2e/rust/tests/podman_oci_identity.rs b/e2e/rust/tests/podman_oci_identity.rs index 3405d730c9..53725e9b4f 100644 --- a/e2e/rust/tests/podman_oci_identity.rs +++ b/e2e/rust/tests/podman_oci_identity.rs @@ -3,14 +3,13 @@ #![cfg(feature = "e2e-podman")] -//! Podman-specific E2E coverage for OCI identity inspection and immutable-image -//! launch. +//! Podman-specific E2E coverage for OCI identity/workspace inspection, +//! workspace-volume copy-up, and immutable-image launch. //! //! The test builds an image through the selected Podman engine, creates a -//! sandbox from its mutable tag, and verifies both the child identity and the -//! image ID recorded on the real sandbox container. This exercises the Podman -//! API inspect → protected metadata → create path rather than only its unit -//! serialization boundaries. +//! sandbox from its mutable tag, and verifies the child identity, copied image +//! content, workspace placement, and image ID recorded on the real sandbox +//! container. use std::process::Stdio; @@ -54,7 +53,18 @@ impl ImageGuard { let containerfile = context.path().join("Containerfile"); std::fs::write( &containerfile, - format!("FROM {BASE_IMAGE}\nUSER {OCI_UID}:{OCI_GID}\n"), + format!( + r"FROM {BASE_IMAGE} +USER 0:0 +RUN mkdir -p /home/app/project/cache && \ + chown {OCI_UID}:{OCI_GID} /home/app /home/app/project /home/app/project/cache && \ + chmod 0700 /home/app /home/app/project /home/app/project/cache +WORKDIR /home/app/project +RUN printf root-owned > root-owned.txt && chown {OCI_UID}:{OCI_GID} . +VOLUME /home/app/project/cache +USER {OCI_UID}:{OCI_GID} +" + ), ) .map_err(|err| format!("write Containerfile: {err}"))?; @@ -89,7 +99,6 @@ impl ImageGuard { "Podman-built image has OCI user '{user}', expected {OCI_UID}:{OCI_GID}" )); } - Ok(Self { engine, tag, id }) } } @@ -175,7 +184,7 @@ fn normalized_image_id(image_id: &str) -> &str { } #[tokio::test] -async fn podman_uses_oci_identity_and_inspected_image_id() { +async fn podman_uses_oci_identity_workspace_copy_up_and_inspected_image_id() { if !is_e2e_driver("podman") { eprintln!("Skipping Podman OCI identity test: e2e driver is not podman"); return; @@ -189,17 +198,18 @@ async fn podman_uses_oci_identity_and_inspected_image_id() { std::fs::write(policy.path(), OCI_FALLBACK_POLICY).expect("write OCI fallback policy"); let policy_path = policy.path().to_str().expect("policy path is UTF-8"); let mut sandbox = SandboxGuard::create_keep_with_args( - &[ - "--from", - &image.tag, - "--policy", - policy_path, - "--no-tty", - ], + &["--from", &image.tag, "--policy", policy_path, "--no-tty"], &[ "sh", "-c", - "set -eu; printf 'direct-identity=%s:%s\n' \"$(id -u)\" \"$(id -g)\"; echo podman-oci-identity-ready; sleep infinity", + "set -eu; \ + test \"$(pwd -P)\" = /home/app/project; \ + test \"$HOME\" = /home/app/project; \ + test \"$(cat root-owned.txt)\" = root-owned; \ + touch direct-workspace-write; \ + touch cache/from-create; \ + printf 'direct-identity=%s:%s\n' \"$(id -u)\" \"$(id -g)\"; \ + echo podman-oci-identity-ready; sleep infinity", ], READY_MARKER, ) @@ -216,7 +226,14 @@ async fn podman_uses_oci_identity_and_inspected_image_id() { .exec(&[ "sh", "-c", - "test \"$(id -u):$(id -g)\" = 2345:2346; echo podman-ssh-identity-ok", + "set -eu; \ + test \"$(id -u):$(id -g)\" = 2345:2346; \ + test \"$(pwd -P)\" = /home/app/project; \ + test -f direct-workspace-write; \ + test -f cache/from-create; \ + touch ssh-workspace-write; \ + touch cache/from-ssh; \ + echo podman-ssh-identity-ok", ]) .await .expect("SSH child should use Podman OCI identity"); @@ -258,8 +275,9 @@ async fn assert_isolated_pair(image: &ImageGuard, sandbox: &SandboxGuard, contai ) .unwrap(); assert_eq!( - workload_user, "0:0", - "the trusted rootless boundary starts as container root before dropping to the OCI identity" + workload_user, + format!("{OCI_UID}:{OCI_GID}"), + "a custom OCI workspace must start directly as the final identity" ); let supervisor_user = run_engine( &image.engine, @@ -299,6 +317,9 @@ async fn assert_isolated_pair(image: &ImageGuard, sandbox: &SandboxGuard, contai .unwrap(); assert!(!mounts.contains("/etc/openshell/tls")); assert!(!mounts.contains("/.openshell/supervisor")); + assert!(mounts.lines().any(|path| path == "/home/app/project")); + assert!(mounts.lines().any(|path| path == "/home/app/project/cache")); + assert!(!mounts.lines().any(|path| path == "/sandbox")); let posture = sandbox.exec(&["sh", "-c", "set -eu; awk '/^CapEff:|^CapBnd:|^NoNewPrivs:/ {print}' /proc/self/status; test ! -r /.openshell/channel/sandbox/server.key; test ! -r /.openshell/supervisor/runtime-descriptor.json"]).await.expect("workload cannot read either control credential set"); assert!(posture.contains("0000000000000000")); } diff --git a/skills/debug-openshell-cluster/SKILL.md b/skills/debug-openshell-cluster/SKILL.md index 85360a77ad..f18f1bb8b1 100644 --- a/skills/debug-openshell-cluster/SKILL.md +++ b/skills/debug-openshell-cluster/SKILL.md @@ -281,7 +281,8 @@ Common findings: - Sandbox fails before readiness with an identity-resolution error: inspect the image's OCI `USER` and matching `/etc/passwd` and `/etc/group` entries, or explicitly set both process identity fields in policy. Numeric workload identities `1` through `4294967294` are accepted; root, the invalid identity sentinel, and missing identities are rejected. - Sandbox fails before readiness with an OCI workspace validation error: inspect the image's `WorkingDir` using the immutable image ID reported by the gateway. Empty, `/`, and explicit `/sandbox` use the managed `/sandbox` compatibility workspace. Any other workdir must be an absolute normalized directory with no symlink components; the final policy UID, primary GID, and supplementary groups must pass the kernel's effective traverse/write checks, including POSIX ACL and LSM decisions. OpenShell does not create, chown, or chmod a non-default image workdir. - Docker also rejects an image `VOLUME` that covers the workdir or one of its parents because the runtime would mask the immutable path before validation. Move the `VOLUME` below the workspace or remove the declaration. -- A workdir rejected as a special filesystem or OpenShell control-path collision cannot be made valid with permissions. Move the image workdir away from kernel-backed mounts and the concrete supervisor, TLS, token, runtime, and socket paths named in the error. +- A workdir rejected as a special filesystem or OpenShell control-path collision cannot be made valid with permissions. Move the image workdir away from kernel-backed mounts and the reserved paths named in the error. +- Podman also rejects an image volume or driver mount that covers the workdir. The workspace volume keeps files copied from the image without changing their permissions; ensure the sandbox user can write the resulting directory. - Local Docker gateway setup cannot copy `openshell-sandbox` after exporting a supervisor image: the sandbox runtime and supervisor are separate artifacts. The runtime image must provide `/openshell-sandbox`; the supervisor image provides `/openshell-supervisor`. - Docker driver cannot initialize because it cannot find `openshell-sandbox`: verify the sibling binary next to `openshell-gateway`, or that the configured `sandbox_runtime_image` contains `/openshell-sandbox`. - Sandbox never registers: check gateway logs and the supervisor's gateway endpoint. diff --git a/skills/openshell-cli/SKILL.md b/skills/openshell-cli/SKILL.md index 818d12a269..c77a6e487c 100644 --- a/skills/openshell-cli/SKILL.md +++ b/skills/openshell-cli/SKILL.md @@ -678,6 +678,11 @@ Explicit numeric fields may use any UID/GID from `1` through Warn users that low IDs can inherit permissions from matching accounts, image files, mounted volumes, or devices. +Podman gateways use a normalized absolute OCI `WORKDIR` as the workspace. +Empty, `/`, and explicit `/sandbox` declarations use the managed `/sandbox` +fallback. For a custom path, the final identity must be able to traverse and +write the directory in the persistent volume. + ### Forward ports ```bash