vllm-dgx-spark-gb10

vLLM for NVIDIA DGX Spark / GB10 Blackwell (SM121)

vLLM container for DGX Spark - 2.5x faster than llama.cpp

Keywords: vLLM DGX Spark, NVIDIA GB10, Blackwell GPU, SM121, ARM64 LLM, CUDA 13.1, Qwen3 inference

🏆 Performance Results

📦 Quick Start

Option 1: Docker Hub (Recommended)

Pull and run directly from Docker Hub:

# Pull the image (7.3GB compressed)
docker pull hellohal2064/vllm-dgx-spark-gb10:latest

# Run with your model
docker run --rm --gpus all \
  -v /path/to/models:/models \
  -p 8000:8000 \
  -e MODEL_PATH=/models/your-model \
  -e MAX_MODEL_LEN=131072 \
  -e GPU_MEMORY_UTIL=0.90 \
  hellohal2064/vllm-dgx-spark-gb10:latest

Option 2: HuggingFace Download

Download the pre-built image tarball from this repository:

# Download image tarball from this repo
wget https://huggingface.co/Hellohal2064/vllm-dgx-spark-gb10/resolve/main/docker/vllm-gb10-latest.tar.gz

# Load the image
gunzip -c vllm-gb10-latest.tar.gz | docker load

# Run with your model
docker run --rm --gpus all \
  -v /path/to/models:/models \
  -p 8000:8000 \
  -e MODEL_PATH=/models/your-model \
  -e MAX_MODEL_LEN=131072 \
  -e GPU_MEMORY_UTIL=0.90 \
  vllm-gb10:latest

Option 3: Build from Source

Clone this repository and build locally:

# Clone and build
git clone https://huggingface.co/Hellohal2064/vllm-dgx-spark-gb10
cd vllm-dgx-spark-gb10
docker build -t vllm-gb10:latest .

🔧 Environment Variables

📏 Context Window Configuration

The MAX_MODEL_LEN environment variable controls the maximum context window size. The DGX Spark's 119GB unified memory allows for very large context windows:

Example with 256K context window:

docker run --rm --gpus all \
  -v /data/models:/models \
  -p 8000:8000 \
  -e MODEL_PATH=/models/Qwen3-Next-80B-A3B-FP8 \
  -e MAX_MODEL_LEN=262144 \
  -e GPU_MEMORY_UTIL=0.95 \
  hellohal2064/vllm-dgx-spark-gb10:latest

Trade-offs:

• Larger context = more GPU memory for KV cache
• Larger context = fewer concurrent requests possible
• With 119GB on DGX Spark, you can run 256K+ context with 80B models

🛠️ Technical Details

Build Environment

• Base: nvcr.io/nvidia/cuda:13.1.0-devel-ubuntu24.04
• PyTorch: 2.11.0.dev20260103+cu130 (nightly with SM121 support)
• vLLM: 0.1.dev1+g0d4044edd (patched for GB10)
• Architecture: ARM64 (aarch64)

Hardware Compatibility

• Primary Target: NVIDIA DGX Spark with GB10 Blackwell GPU
• CUDA Compute: SM121 (Blackwell)
• Memory: Supports unified memory architecture (119GB on DGX Spark)

Key Optimizations

• Native SM121 CUDA kernels (not emulated)
• TRITON_ATTN backend (FlashInfer doesn't support SM121 yet)
• MoE expert configurations for GB10
• CPU parallelization (OMP_NUM_THREADS=20)
• Full CUDA graph optimization (no --enforce-eager)
• Chunked prefill enabled for long context efficiency

📁 Repository Contents

• Dockerfile - Build instructions
• docker-compose.yml - Production deployment config
• entrypoint.sh - Flexible startup script
• docker/vllm-gb10-latest.tar.gz - Pre-built image (7.24GB compressed)

🧪 Tested Models

🤝 Contributing

Issues and PRs welcome! This is a community effort to bring vLLM to the DGX Spark platform.

📄 License

Apache 2.0