Qwen3.5-35B-A3B-VKAE

📱 Run it on your phone or a GPU-less PC → POCKET · 🚀 Try it live (CPU chat)

VIDRAFT's on-device family: a 35B model that runs on iPhone and on CPU with no GPU — stock llama.cpp, no fork.

Qwen3.5-35B-A3B — VKAE Accelerated

Ready-to-run, VKAE-accelerated serving of Qwen3.5-35B-A3B (35B-parameter Mixture-of-Experts, ~3B active). Ships as a self-contained container — model weights and an optimized serving runtime in a single image — so anyone can reproduce the numbers on their own GPU.

VKAE (VIDRAFT Kernel Acceleration Engine) is VIDRAFT's proprietary inference-serving optimization. The acceleration recipe is withheld; only the reproducible results are published here.

Measured performance

NVIDIA B200, single GPU, FP8, same-harness before/after.

Realistic varied-content single-stream throughput sits around ~455 tok/s. Accuracy is preserved end to end.

Quick start

docker pull vidraft/qwen35-vkae:601
docker run --gpus all -p 8000:8000 vidraft/qwen35-vkae:601

The container serves an OpenAI-compatible API on port 8000 — point any OpenAI client at http://localhost:8000/v1. A Blackwell (B200) or Hopper (H100/H200) class GPU is recommended.

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen35-vkae","messages":[{"role":"user","content":"Hello!"}]}'

📦 Ready-to-use files in this repo: Dockerfile, docker-compose.yml, run_docker.sh — pull-and-run, no build required.

Links

• Live acceleration leaderboard → VIDraft/vkae
• Docker image → hub.docker.com/r/vidraft/qwen35-vkae
• Collection → FINAL-Bench · VKAE Accelerated

Base model & license

Base weights: Qwen/Qwen3.5-35B-A3B (Apache-2.0), unmodified. This card documents VIDRAFT's accelerated serving of the model; the base model itself is unchanged. The VKAE acceleration method is proprietary and is not distributed in source form.