fix(docker): validate explicit CDI device inventory

Signed-off-by: Evan Lezar <elezar@nvidia.com>

Author: elezar

crates/openshell-driver-docker/README.md

@@ -85,7 +85,7 @@ contract:
 | `apparmor=unconfined` | Avoids Docker's default profile blocking required mount operations. |
 | `restart_policy = unless-stopped` | Keeps managed sandboxes resumable across daemon or gateway restarts. |
 | `PidsLimit` | Enforces the sandbox PID budget at the Docker cgroup layer. Set `[openshell.drivers.docker].sandbox_pids_limit = 0` to inherit the Docker/runtime default. |
-| CDI GPU request | Uses opaque `driver_config.cdi_devices` values when set; otherwise selects the requested count of NVIDIA CDI GPUs in round-robin order when daemon CDI support is detected. Docker daemon `/info` can permit `nvidia.com/gpu=all` as a WSL2 all-only compatibility fallback, where it counts as one selectable device. Exact CDI device lists must not contain duplicates and must match the effective GPU count. |
+| CDI GPU request | Validates `driver_config.cdi_devices` against Docker's current NVIDIA CDI inventory when set; otherwise selects the requested count in round-robin order when daemon CDI support is detected. Docker daemon `/info` can permit `nvidia.com/gpu=all` as a WSL2 all-only compatibility fallback, where it counts as one selectable device. Exact CDI device lists must not contain duplicates and must match the effective GPU count. |
 
 The agent child process does not retain these supervisor privileges.
 

crates/openshell-driver-docker/src/lib.rs

@@ -654,6 +654,10 @@ impl DockerComputeDriver {
                 "driver_config.cdi_devices",
             )
             .map_err(Status::invalid_argument)?;
+
+            self.refresh_gpu_inventory().await?;
+            validate_discovered_cdi_gpu_devices(cdi_devices, &self.gpu_selector.device_ids())?;
+
             return Ok(Some(cdi_devices.to_vec()));
         }
 
@@ -2543,6 +2547,30 @@ fn docker_gpu_selection_status(err: CdiGpuSelectionError) -> Status {
     Status::failed_precondition(err.to_string())
 }
 
+fn validate_discovered_cdi_gpu_devices(
+    requested: &[String],
+    discovered: &[String],
+) -> Result<(), Status> {
+    let unavailable = requested
+        .iter()
+        .filter(|device| !discovered.iter().any(|known| known == *device))
+        .cloned()
+        .collect::<Vec<_>>();
+    if unavailable.is_empty() {
+        return Ok(());
+    }
+
+    let available = if discovered.is_empty() {
+        "none".to_string()
+    } else {
+        discovered.join(", ")
+    };
+    Err(Status::failed_precondition(format!(
+        "requested Docker CDI GPU device(s) were not discovered: {}; available devices: {available}",
+        unavailable.join(", "),
+    )))
+}
+
 #[cfg(test)]
 fn build_container_create_body(
     sandbox: &DriverSandbox,

crates/openshell-driver-docker/src/tests.rs

@@ -1666,6 +1666,19 @@ fn validate_sandbox_accepts_single_cdi_device_without_gpu_count() {
         .expect("single exact CDI device should be compatible with a default GPU request");
 }
 
+#[test]
+fn explicit_cdi_devices_must_be_in_docker_inventory() {
+    let err = validate_discovered_cdi_gpu_devices(
+        &["nvidia.com/gpu=invalid".to_string()],
+        &["nvidia.com/gpu=0".to_string()],
+    )
+    .unwrap_err();
+
+    assert_eq!(err.code(), tonic::Code::FailedPrecondition);
+    assert!(err.message().contains("nvidia.com/gpu=invalid"));
+    assert!(err.message().contains("nvidia.com/gpu=0"));
+}
+
 #[test]
 fn validate_sandbox_rejects_multiple_cdi_devices_without_gpu_count() {
     let mut config = runtime_config();

docs/reference/sandbox-compute-drivers.mdx

@@ -98,12 +98,13 @@ creates. On WSL2 all-only runtimes, Docker or Podman can use
 selectable device.
 
 Exact GPU device selection remains driver-owned and requires `--gpu`. Docker
-and Podman accept `cdi_devices` as opaque CDI device names; replace the
-top-level `docker` key with `podman` when using the Podman driver, for example
-`{"docker":{"cdi_devices":["nvidia.com/gpu=0"]}}`. Explicit CDI device lists
-must not contain duplicates, and their length must match the effective GPU
-count. A single exact CDI device is compatible with the default `--gpu`
-request. The VM driver accepts `gpu_device_ids`, for example
+validates `cdi_devices` against its current NVIDIA CDI inventory before it
+creates a container. Podman accepts CDI device names as driver-owned values.
+Replace the top-level `docker` key with `podman` when using the Podman driver.
+For example: `{"docker":{"cdi_devices":["nvidia.com/gpu=0"]}}`. Explicit
+CDI device lists must not contain duplicates, and their length must match the
+effective GPU count. A single exact CDI device is compatible with the default
+`--gpu` request. The VM driver accepts `gpu_device_ids`, for example
 `{"vm":{"gpu_device_ids":["0000:2d:00.0"]}}`; the current VM implementation
 accepts at most one entry and allows either `--gpu` or `--gpu 1` when
 `gpu_device_ids` is set.

docs/sandboxes/manage-sandboxes.mdx

@@ -100,9 +100,10 @@ device.
 
 Exact GPU device selection is driver-specific and still requires `--gpu`. For
 Docker or Podman, pass CDI IDs through `cdi_devices`. The top-level key must
-match the active driver; replace `docker` with `podman` when using Podman. CDI
-IDs are treated as opaque strings. The list must not contain duplicate IDs, and
-its length must match the effective GPU count:
+match the active driver; replace `docker` with `podman` when using Podman.
+Docker validates each ID against its current NVIDIA CDI inventory before it
+creates the sandbox container. The list must not contain duplicate IDs, and its
+length must match the effective GPU count:
 
 ```shell
 openshell sandbox create \