qwen3-coder-next

gdubicki/Qwen3-Coder-Next-NVFP4-GB10 on DGX Spark (GB10)

Runs gdubicki/Qwen3-Coder-Next-NVFP4-GB10 (quantized by saricles) via vLLM with an OpenAI-compatible API endpoint. Tested on DGX Spark (GB10 Blackwell, SM12.1, 128 GB unified LPDDR5X).

Model overview

Model features

Performance

Measured on DGX Spark (GB10, SM12.1, 128 GB unified LPDDR5X):

Comparison across models on GB10:

Qwen3-Coder-Next is 80B total but 3B active — same throughput as Nemotron-3-Nano with full 262K context.

Quick start

# Required — model is gated on Hugging Face (accept license at gdubicki/Qwen3-Coder-Next-NVFP4-GB10 first):
export HF_TOKEN=hf_xxxx

bash start-qwen3-coder-next.sh

The script will:

1. Stop and remove any existing qwen3-coder-next-vllm container
2. Flush the system page cache (frees unified memory before vLLM starts)
3. Start the container in detached mode
4. Poll http://localhost:8000/health until the API is ready (~45 GB download on first run)

Test the API

bash test-api.sh              # localhost
bash test-api.sh 192.168.x.x  # remote host

Reasoning (chain-of-thought)

Reasoning is ON by default. Toggle per request:

# Reasoning OFF
curl -s -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gdubicki/Qwen3-Coder-Next-NVFP4-GB10","messages":[{"role":"user","content":"What is the capital of France?"}],"max_tokens":60,"chat_template_kwargs":{"enable_thinking":false}}'

# Reasoning ON (default)
curl -s -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gdubicki/Qwen3-Coder-Next-NVFP4-GB10","messages":[{"role":"user","content":"Write a binary search implementation in Python."}],"max_tokens":2000,"chat_template_kwargs":{"enable_thinking":true}}'

Cline configuration

1. Open Cline settings (sidebar icon → gear icon)
2. Fill in the fields:

Files

How it works

The vllm/vllm-openai:cu130-nightly image includes native qwen3_next support.

Architecture: Hybrid of:

• DeltaNet selective linear attention layers (36 of 48 layers) — subquadratic in sequence length, no KV cache
• Full attention layers (12 of 48) — standard transformer attention with KV cache
• Latent MoE (80B total, 3B active per token — same throughput profile as Nano)

Quantization is via compressed-tensors (llmcompressor) with LLMCOMPRESSOR_MOE_CALIBRATE_ALL_EXPERTS=1 — all 512 MoE experts are calibrated, not just sampled. vLLM auto-detects quantization from quantization_config in config.json; no --quantization flag needed.

Key environment variables

Key flags

Requirements

• Docker with nvidia-container-toolkit
• Image: vllm/vllm-openai:cu130-nightly
• HF token with access to gdubicki/Qwen3-Coder-Next-NVFP4-GB10 (gated — accept license first)
• Model weights cached locally (auto-downloaded on first run, ~45 GB): ~/.cache/huggingface/hub/models--saricles--Qwen3-Coder-Next-NVFP4-GB10/