Qwen-Drive-1.0-4B-VLM-mlx-bf16

Qwen-Drive-1.0-4B VLM · MLX bf16

The vision-language model inside Qwen/Qwen-Drive-1.0-4B, converted to MLX in its original precision for Apple-silicon Macs. It answers questions about driving images, which is the VQA mode of Qwen-Drive-1.0.

Smaller: 8-bit (4.8 GB) and 4-bit (2.8 GB, feature extraction only). Use this bf16 version when you need the same outputs as the original weights.

Only the VLM. Qwen-Drive-1.0 has three parts: the Qwen3.5-4B VLM, a Planning Expert (trajectories) and a BEV perception head (3D boxes, occupancy, map). This repository contains the VLM only. The other two run on a Mac either through PyTorch's MPS backend (apple-silicon branch) or in MLX, which reads this model for its features and is about twice as fast (mlx-port branch).

Not for driving decisions. Research and demonstration only. Answers can be wrong.

Use

pip install "mlx-vlm==0.6.0"
mlx_vlm.generate --model drivetechodyssey/Qwen-Drive-1.0-4B-VLM-mlx-bf16 \
    --image front.jpg --prompt "What should the ego vehicle do next?" \
    --max-tokens 400 --temperature 0

The CLI prints an empty thinking block (</think>) before the answer.

import re
from mlx_vlm import apply_chat_template, generate, load
from mlx_vlm.utils import load_config

path = "drivetechodyssey/Qwen-Drive-1.0-4B-VLM-mlx-bf16"
model, processor = load(path)
config = load_config(path)
prompt = apply_chat_template(processor, config, "What should the ego vehicle do next?",
                             num_images=1, enable_thinking=False)
result = generate(model, processor, prompt, image=["front.jpg"], max_tokens=400, temperature=0.0)
# the answer starts with an empty <think></think> block
print(re.sub(r"^\s*<think>.*?</think>\s*", "", result.text, flags=re.S))

How it was made

1. Extract the VLM. Every VLM tensor in the original model.safetensors starts with vlm.; the prefix is removed (723 tensors; lm_head is tied to the embeddings). config.json is the original vlm_config, with vision_config.model_type set to qwen3_5 because mlx-vlm 0.6.0 does not accept qwen3_5_vision. Tokenizer, processor and chat-template files are copied unchanged.
2. Convert. mlx_vlm.convert --dtype bfloat16.
3. Restore float32 norms. The original keeps each linear-attention layer's A_log and norm.weight in float32 (48 tensors). The converter kept A_log but cast the 24 norm.weight tensors to bf16, which made long greedy answers drift from the original after roughly 700 characters. Those 24 tensors are copied back, so the dtype layout matches the original (675 bf16 + 48 float32).

Checks

Measured on an M5 Max (64 GB) with mlx 0.31.2 and mlx-vlm 0.6.0, greedy decoding, nothing else using the GPU. 16 images from the original repository's demo data (the current front, front-left and front-right frames of the four planning scenes, plus four front frames from the perception demo) × 3 questions: a long English scene description (400 tokens), a one-sentence hazard, and two or three sentences in Korean.

This bf16 version gives character-for-character the same greedy answers as the original weights loaded directly in mlx-vlm, on all 48 prompts.

A 4-bit version (2.8 GB, 147 tok/s) is published for feature extraction only, not for answering questions: none of its 48 answers matched bf16 (mean similarity 0.44), and on three of the sixteen images it reported a pedestrian crossing that is not there.

Behaviour of the model itself: a Korean answer occasionally contains a Chinese word (1 of 16 here, 신호灯的).

Perception and planning also read this model

The perception head and the Planning Expert do not read the model's words. They read the pre-merge vision patches, the image-token hidden states and the full-attention K/V cache. Scored against this repository's demo ground truth, the choice of precision makes no difference there, which is not true of the answers above:

Prompt processing runs at the same speed at every precision (1.9 to 3.0 k tok/s), so a smaller conversion saves memory there rather than time. The MLX implementation of both heads is in the mlx-port branch.

License and citation

Apache License 2.0, the same as the original (see LICENSE). The weights are the Qwen team's (Alibaba Group); this repository only changes their format as described above.

@misc{zhou2026qwendrive10initialstepvisionlanguage,
      title={Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving},
      author={Xin Zhou and Zongchuang Zhao and Zhibo Yang and Mingsheng Li and Humen Zhong and Shuai Bai and Du Chu and Ruizhe Chen and Zhaohai Li and Jun Tang and Qiuyue Wang and Mingkun Yang and Jiazhao Zhang and Dayiheng Liu and Dingkang Liang and Xiang Bai},
      year={2026},
      eprint={2609.00111},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2609.00111},
}