Qwen3-VL-2B-Instruct-ExecuTorch

Qwen3-VL-2B-Instruct — ExecuTorch

A vision-language model as one .pte with three entry points. Ask it about a photograph on the device; nothing leaves it.

• Source: Qwen/Qwen3-VL-2B-Instruct — 2.13B parameters, 28-layer decoder, 24-layer vision tower
• License: apache-2.0
• Input: a 512x512 RGB photograph and a question
• Output: text

Methods

Three methods rather than three files because ExecuTorch copies a mutable buffer into every method that names it, so a prefill method and a decode method would each get their own KV cache and neither would see the other's writes. One graph serves a 277-token prompt and a single token.

What the caller owns

Three things, and getting any of them wrong does not throw — it answers about the wrong thing:

1. Splice the 256 vision rows into the prompt embedding at the image token positions.
2. Scatter the three deepstack maps to those same positions, zero everywhere else. Qwen3-VL adds features from vision blocks 5, 11 and 17 into decoder layers 0, 1 and 2, and only at image positions. Upstream writes that as hidden_states[visual_pos_masks, :] += embeds, which does not export; here the maps arrive already placed, so the graph just adds them.
3. Pass Qwen's own 3-D positions from get_rope_index, not arange. The rope is interleaved M-RoPE with mrope_section [24, 20, 20] — time, height, width. A 1-D position array gives every image patch the same angle and reads as a mild degradation rather than an error.

convert/run_qwen3_vl.py does all three in 40 lines.

Verification (Mac arm64, executorch 1.4.0, torch 2.13.0)

Parity is the last-row logits of a real prompt against Qwen3VLForConditionalGeneration in eager fp32, per question, because a model of this size can hold a scene and lose the small print. Both questions are about the same photograph.

The prompt is 277 tokens, of which 256 are the picture. vision_encoder runs once per image, text_model once per prompt and once per generated token.

What it says

Asked for the name on a pub sign, against the eager model's "Lamb & Flag":

• 8da8w — Based on the image provided, the name of the pub is The Lamb & Flag.

This name is visible on the sign above the entrance to the establishment. The sign also includes the address "

• 8da4w — Based on the text visible on the sign above the entrance of the establishment, the name of the pub is Pine Purpose.

The sign is located on the front of the building, just above

The rewrites, and what each was checked against

Neither this repo's generic VL bundle nor export_llm reaches Qwen3-VL. Five things had to change, each measured against the module it replaced:

With the first two in place and no quantisation, the vision wrapper reproduces Qwen3VLVisionModel at max_abs_diff 0.000e+00 across all four outputs, and the decoder wrapper reproduces the full model's first-step logits at corr 1.000000.

Two traps worth knowing

The token embedding table escapes quantisation. It is an nn.Embedding, so a filter written as isinstance(module, nn.Linear) skips it — and tie_word_embeddings does not save it, because quantize_ swaps lm_head.weight for a new tensor while the embedding keeps pointing at the old fp32 one. At 151,936 x 2048 that is 1.24 GB left in fp32: the first export of this model came out at 2630 MB. torchao's own IntxWeightOnlyConfig does reach an nn.Embedding but will not lower — Missing out variants: {'torchao::dequantize_affine'} — so the quantisation is written out by hand, one scale per row.

512x512 is where small text lives or dies, and the resampling filter decides. Asked for the name on the sign, the eager fp32 model answers:

That is before any quantisation. If fine print matters for your use, the preprocessing is not a detail; convert/export_qwen3_vl.py puts the resize behind one helper for that reason.

Not shipped

8da4w — the same bundle with a 4-bit decoder — is 1697.5 MB against 2453.1 MB, and it does not hold: logits correlation 0.740 on the sign question against 0.993, and it reads the sign as "Pine Purpose". Measured where the loss is, one arm at a time: the vision tower at int8 costs corr 0.990 against fp32's 0.99982, the hand-quantised table costs 0.99997, and the 4-bit decoder is the rest. Reported here rather than dropped, because a 1.7 GB 2B vision-language model is worth wanting and this is what it costs.

Conversion

python convert/export_qwen3_vl.py            # check both halves against eager
python convert/export_qwen3_vl.py --export   # write the bundle
python convert/check_qwen3_vl.py 8da8w       # parity and speed
python convert/run_qwen3_vl.py photo.jpg "what is on the sign?"

(conversion scripts: executorch-models)