Qwen-AgentWorld-35B-A3B-VL36

Qwen-AgentWorld-35B-A3B-VL36 (vision-grafted, Qwen3.6 ViT)

TL;DR — a weight-grafted multimodal variant of Qwen/Qwen-AgentWorld-35B-A3B: its agentic text backbone, with the vision tower (ViT + projector) transplanted in from Qwen/Qwen3.6-35B-A3B. No training was performed — this is pure weight surgery. Full precision BF16 (not a quantized model).

⚠️ Experimental, cross-generation graft. The vision tower comes from Qwen3.6 while the AgentWorld backbone is a Qwen3.5-generation derivative. The projector (merger.linear_fc2) lands dimensionally exactly on the text hidden size (2048) and the vision config matches bit-for-bit, so it loads and runs. In smoke tests the 3.6 ViT gives clean, accurate grounding (correctly named shapes, colors and OCR'd a number with precise spatial relations). But this is not a natively co-trained VLM and has not been benchmarked. Use at your own risk.

A same-generation variant using the Qwen3.5 ViT also exists (Qwen-AgentWorld-35B-A3B-VL); 3.6's vision tower is the stronger encoder, hence this build.

What we changed vs. Qwen-AgentWorld-35B-A3B

AgentWorld ships text-only weights (693 tensors, all model.language_model.* + lm_head) even though its config already declares a vision_config, image/video token ids, and a vision+tool-capable chat template — it is effectively a Qwen3.5-VL waiting for a vision encoder. We:

1. Extracted the 333 model.visual.* tensors (27-layer ViT + patch/position embeds + merger/projector) from Qwen3.6-35B-A3B (BF16).
2. Added them to the AgentWorld checkpoint without touching a single text tensor — so the agent / tool-calling skills are byte-for-byte preserved.
3. Removed the language_model_only flag from the config so the runner instantiates the tower.

Result: 1026 tensors (693 text + 333 vision), architecture Qwen3_5MoeForConditionalGeneration, projector maps vision hidden 1152 → text hidden 2048.

Serving (vLLM ≥ 0.23)

vllm serve havok2/Qwen-AgentWorld-35B-A3B-VL36 \
  --tensor-parallel-size 4 \
  --reasoning-parser qwen3 \
  --limit-mm-per-prompt '{"image": 4, "video": 1}'
# IMPORTANT: do NOT pass --language-model-only — that disables the vision tower.

Loads as a standard Qwen3.5-MoE VLM. ~70 GB BF16; fits on 4×24 GB with --tensor-parallel-size 4.

Capabilities & limitations

• ✅ Agentic text + tool-calling — identical to AgentWorld (weights untouched).
• ✅ Image / video input — works; smoke-tested on synthetic images (shapes, colors, OCR).
• ⚠️ Vision quality is not benchmarked; cross-generation graft, may differ from a natively co-trained VLM.
• ⚠️ No additional safety tuning or evaluation beyond basic smoke tests.

Reproduce it yourself (no 70 GB download needed)

Rebuild locally from the two source models with the included build_merged_vl36.py. It downloads the AgentWorld backbone and only the Qwen3.6 shards that contain the vision tensors, then performs the graft:

pip install huggingface_hub safetensors torch
python build_merged_vl36.py        # -> ./Qwen-AgentWorld-35B-A3B-VL36

Attribution & license

• Text backbone & agent post-training: Qwen/Qwen-AgentWorld-35B-A3B (Apache-2.0), itself based on Qwen3.5-35B-A3B-Base.
• Vision tower (ViT + projector): Qwen/Qwen3.6-35B-A3B (Apache-2.0).

This derivative is released under Apache-2.0. Modification: vision-tower graft as described above (no training). See LICENSE. Not affiliated with or endorsed by Alibaba / the Qwen team.