UI-Venus-2-9B-mlx-4bit-awq

UI-Venus-2-9B-mlx-4bit-awq

Calibrated 4-bit — AWQ over 64 real GUI screenshots, the closest MLX equivalent to a GGUF imatrix quant. Read the coverage caveat before assuming it beats plain 4-bit: on my eval it did not.

MLX quantization of inclusionAI/UI-Venus-2-9B — a GUI-agent vision-language model on the Qwen3.5 architecture (hybrid Gated-DeltaNet linear attention + full attention, SigLIP-style vision tower). Converted with mlx-vlm 0.6.17 / mlx 0.32.2.

The lineup

Measured with Click on: <instruction> over 50 held-out ScreenSpot-Pro samples (images capped at 2048 px; a prediction counts as correct only if the point falls inside the ground-truth bbox). These four numbers are within noise of each other — see below. This is not the paper's evaluation protocol and 50 samples is a small set, so read it as a relative check between quants, not as a benchmark score for the model.

Usage

pip install -U mlx-vlm

python -m mlx_vlm.generate \
  --model lethargicgeek/UI-Venus-2-9B-mlx-4bit-awq \
  --image screenshot.png \
  --prompt "Click on: the Save button" \
  --max-tokens 64

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("lethargicgeek/UI-Venus-2-9B-mlx-4bit-awq")
prompt = apply_chat_template(processor, model.config, "Click on: the Save button", num_images=1)
print(generate(model, processor, prompt, ["screenshot.png"], max_tokens=64))

Output format

Grounding prompts return a <point> tag. Coordinates are normalized to 0–1000, not pixels:

px = x / 1000 * image_width
py = y / 1000 * image_height

Both <point>X Y</point> and <point x1="X" y1="Y" alt="..."> forms occur, so parse for either.

Quantization details

Calibration images and a manifest are included under calibration/ for reproducibility. Unlike llama.cpp imatrix — which is text-only, because the vision tower is split out into mmproj — this calibration routes real screenshots through the vision tower and language model together.

The vision tower is not quantized. All 333 vision tensors stay at bfloat16 in every build here — mlx-vlm's skip_multimodal_module() excludes them. This mirrors how GGUF conversions ship the vision encoder as a separate f16 mmproj. Only the language model is quantized, which is why a "4-bit" build measures 5.06 effective bits per weight rather than 4.

What I actually measured

Grounding is essentially unaffected by quantization. All four builds landed between 56% and 60%. At n=50 with p≈0.6 the standard error is roughly ±7 percentage points, so a 4-point spread is noise. There is no evidence here that 6-bit grounds better than 4-bit, or that calibration helps.

AWQ did not measurably beat plain RTN. Worth stating plainly, since calibrated quants are usually assumed to win. The likely reason is coverage: only 8 of 32 layers received AWQ scaling (layers: 8, groups: 24). The other 24 are linear_attention (Gated DeltaNet) layers that AWQ's scale search does not handle, so they fell back to RTN. On a conventional all-attention transformer AWQ would cover the whole network; on this hybrid it reaches a quarter of it.

Structured description degrades somewhat at 4 bits. Asked to list the buttons in a screenshot, 4bit and mixed_4_6 sometimes emit coordinate tags (<ex>10 110</ex>...) instead of prose, where 6bit and 4bit-awq returned a proper list. That was 5 prompts across 4 builds — suggestive, not conclusive. Ordinary description ("what application is this?", "summarize this screen") worked on every build.

Picking one

• Grounding/clicking only, want it small → any 4-bit build; they measured the same.
• Mixed use, or you want the fewest edge cases → 6bit.
• 4bit-awq is included for completeness and reproducibility, not because it won.

License and attribution

The upstream repo inclusionAI/UI-Venus-2-9B declares no license — no license tag, no cardData.license, no LICENSE file. None is asserted here. Check with inclusionAI before using these weights in a product. (Some third-party conversions label this Apache-2.0; I could not find a basis for that in the source repo.)

Credit for the model goes to inclusionAI (technical report, project page), derived in turn from Qwen/Qwen3.5-9B. This repo contains only a quantized conversion.