Qwen3-VL-2B-Instruct-LLM

Qwen3-VL-2B-Instruct-LLM

The language model of Qwen/Qwen3-VL-2B-Instruct, extracted as a plain Qwen3ForCausalLM checkpoint (vision tower dropped).

The Qwen3-VL-2B text tower is architecturally identical to Qwen3-1.7B dense (hidden 2048 / 28 layers / 16 Q + 8 KV heads / head_dim 128 / FFN 6144 / vocab 151936 / tied embeddings). Its interleaved MRoPE assigns text tokens the same index on all three axes, which reduces exactly to standard 1D RoPE, so this export keeps the VL rope_theta (5e6) and drops rope_scaling.

Equivalence: text-only logits match the original VL model bit-for-bit in fp32 (max |logit diff| = 0.0 on a held-out prompt).

Load with:

from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("shushengyang/Qwen3-VL-2B-Instruct-LLM")

Produced by personal/shusheng/mllm/misc/convert_qwen3vl_llm_to_qwen3.py (amilabs monorepo); intended as an LLM initialization for ScaleWM MLLM experiments (vision-language-pretrained LLM vs text-only Qwen3-1.7B).