[Bug]: CUDA initialization fails in Docker containers after upgrading to libnvidia-container/nvidia-container-toolkit 1.19.1 (works on 1.19.0)

[cacauisadog]

Summary

After upgrading from:

• libnvidia-container 1.19.0-1
• nvidia-container-toolkit 1.19.0-1

to:

• libnvidia-container 1.19.1-1
• nvidia-container-toolkit 1.19.1-1

CUDA initialization inside Docker containers fails with:

cuInit(0) failed -> CUDA_ERROR_UNKNOWN

Downgrading both packages back to 1.19.0-1 fixes the issue immediately.

Environment

OS:

• Manjaro Linux

GPU:

• NVIDIA GeForce GTX 970 (Maxwell, Compute Capability 5.2)

Driver:

• 580.173.02

Kernel:

• 6.18.39-1-MANJARO

Docker:

• cgroup v2
• systemd cgroup driver

Host test

This succeeds:

ffmpeg \
  -hide_banner \
  -init_hw_device cuda=0 \
  -f lavfi -i testsrc \
  -frames:v 1 \
  -f null -

Container reproducer

docker run --rm --gpus all \
  linuxserver/ffmpeg:latest \
  -hide_banner \
  -init_hw_device cuda=0 \
  -f lavfi -i testsrc \
  -frames:v 1 \
  -f null -

Expected

FFmpeg initializes CUDA and processes one frame.

Actual on 1.19.1

[CUDA @ ...] cu->cuInit(0) failed -> CUDA_ERROR_UNKNOWN: unknown error
Device creation failed

Actual after downgrading to 1.19.0-1

The exact same command succeeds.

Additional information

• nvidia-smi works on the host.
• nvidia-smi works inside containers.
• The failure only affects CUDA initialization inside containers.
• Host CUDA works correctly.
• Jellyfin hardware transcoding also fails on 1.19.1 and works again after downgrading, but the issue can be reproduced independently using the FFmpeg command above.

[henry118]

@cacauisadog i am not able to repro locally. can you please share

1. the output of docker info in your environment?
2. the content of the CDI spec at /var/run/cdi/nvidia.yaml?

thanks