Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@
.DS_Store
*.swp

# Local agent tooling
.claude/

# C extensions
*.so

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
ARG DTK_VERSION=26.04
ARG VLLM_VERSION=0.18.1

FROM gpustack/runner:dtk${DTK_VERSION}-vllm${VLLM_VERSION} AS vllm
SHELL ["/bin/bash", "-eo", "pipefail", "-c"]

ARG TARGETPLATFORM
ARG TARGETOS
ARG TARGETARCH

## Restore NumPy 1.x ABI compatibility

# The DTK base image currently ships a Torch build compiled against the NumPy
# 1.x ABI. NumPy 2.x cannot initialize in that Torch build, which causes
# distributed initialization to fail when it calls Tensor.numpy().
#
# Install without dependencies to avoid changing the prebuilt DTK Python
# environment beyond the NumPy compatibility pin.
RUN <<EOF
uv pip install --no-deps "numpy==1.26.4"

python - <<'PYEOF'
import numpy

assert numpy.__version__ == "1.26.4", \
f"expected numpy 1.26.4, got {numpy.__version__}"
PYEOF
EOF

## Remove CUDA-only NIXL EP packages

# The prebuilt DTK vLLM base image can inherit NIXL Expert Parallel packages
# from its Python environment. NIXL EP provides NVIDIA CUDA backends which
# require libcuda.so.1 and cannot run on Hygon DCU systems.
#
# vLLM detects NIXL EP through importlib.util.find_spec("nixl_ep"). Remove
# the distributions entirely, rather than leaving an ImportError stub, so
# vLLM selects its non-NIXL path. Use uv to resolve the active Python
# environment and package paths instead of relying on a fixed site-packages
# location.
RUN <<EOF
# The NIXL EP modules are owned by the nixl, nixl-cu12, and nixl-cu13
# distributions in the DTK base image. Do not fail if a future base image
# no longer includes one of them; the import check below remains required.
uv pip uninstall nixl nixl-cu12 nixl-cu13 || true

python - <<'PYEOF'
import importlib.util

for module_name in ("nixl_ep", "nixl_ep_cu12", "nixl_ep_cu13"):
assert importlib.util.find_spec(module_name) is None, \
f"{module_name} is still discoverable after DTK image cleanup"
PYEOF
EOF

RUN <<EOF
# Review
uv pip tree \
--package numpy \
--package torch \
--package vllm
EOF

## Entrypoint

WORKDIR /
ENTRYPOINT [ "tini", "--" ]
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
rules:

#
# Hygon DTK
#

## Packed Hygon DTK 26.04.
##
- backend: "dtk"
services:
- "vllm"
platforms:
- "linux/amd64"
args:
- "DTK_VERSION=26.04"
- "VLLM_VERSION=0.18.1"
1 change: 1 addition & 0 deletions pack/.post_operation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,4 @@ We leverage the matrix expansion feature of GPUStack Runner to achieve this, and
- [ ] 2026-02-14: Patch SGLang 0.5.8.post1 of CANN/CUDA/ROCm released images to reduce Z-Image loading memory occupation.
- [x] 2026-02-28: Reinstall `vllm-omni` packages for vLLM 0.16.0 of CUDA released images.
- [x] 2026-03-03: Fix malformed ARM64 image for vLLM 0.15.1 of CUDA released images.
- [ ] 2026-09-01: Pin `numpy` to 1.26.4 and remove CUDA-only NIXL EP packages for vLLM 0.18.1 of DTK 26.04 released images.
Loading