Fara1.5-4B-8bit

Fara1.5-4B-8bit

8-bit MLX quantization of microsoft/Fara1.5-4B, a 4.5B computer-use / web-agent VLM built on Qwen3.5-4B. It reads screenshots and acts on interfaces. Converted with mlx-vlm 0.6.8 (mlx 0.32.0) for inference on Apple Silicon.

Every prior MLX conversion of this model is 4-bit, so 8-bit was the open gap. Fidelity was measured against the unquantized bf16 source, not assumed.

pip install mlx-vlm
python -m mlx_vlm generate \
    --model mlx-community/Fara1.5-4B-8bit \
    --image screenshot.png \
    --prompt "Describe the buttons and the total shown in this screenshot." \
    --max-tokens 256

Image input needs mlx-vlm. Stock mlx-lm loads the text path only, which for a computer-use model is not a smaller model but a broken one.

Quantization

Effective bits per weight exceeds the nominal 8 because mlx-vlm quantizes only the language model and leaves the vision tower in bf16 by design. The vision path is preserved, which for a computer-use model is the whole point: verified at tensor level, of 297 vision tensors zero carry a .scales entry.

A conversion trap worth knowing about

This model declares dtype: bfloat16 at the top level of its config but float32 in both text_config and vision_config. mlx_vlm.convert honours the sub-configs, so converting without an explicit --dtype bfloat16 silently keeps the scales, biases and the entire vision tower in fp32:

This repo is the --dtype bfloat16 build. The higher SNR of the fp32 build is not better fidelity: it is a float32 reconstruction being compared against a bf16 source, bought with 0.9 GB of precision that has no business in an 8-bit repo. Rounding those scales to bf16 accounts for the entire 2.5 dB difference.

Measured fidelity

Against the unquantized bf16 source, greedy decoding, on an M2 Pro / 32 GB.

Weight-level metrics stream the source tensor by tensor from disk over all 4,204,789,760 quantized parameters. No prompts, no sampling, hardware-independent.

42.70 dB is the expected value, and that is the point. Six 8-bit affine conversions at group size 64 across six different architectures (AREX-Turbo, Fara-7B, Fara1.5-4B, GLM-4.1V, MagenticBrain, OvisOCR2) all land on 42.66-42.70 dB. A reading outside that band is a defect signal, which is exactly how the fp32-scales issue above was caught.

Highest-error tensors, all in the linear-attention input projections:

rel_l2=0.00841  snr=41.50 dB  language_model.model.layers.25.linear_attn.in_proj_a
rel_l2=0.00839  snr=41.53 dB  language_model.model.layers.28.linear_attn.in_proj_a
rel_l2=0.00834  snr=41.57 dB  language_model.model.layers.26.linear_attn.in_proj_a

1.9x faster, 1.5x less memory than bf16.

Vision path, verified working rather than merely present

Given a synthetic UI screenshot, the 8-bit model read both button labels (SubmitOrder, Cancel) and the total (42.50 USD) correctly.

On the perplexity aggregate

The harness reports aggregate perplexity as exp(sum_nll/sum_tokens) over five passages, and flags it as unreliable here: the bf16 source itself spans 10.3x across those passages, so a single out-of-domain passage can dominate. The per-passage ratios are reported instead, and they agree with the aggregate rather than contradict it, all within 1.8% of the bf16 reference:

BLEU of 81.30 measures drift, not quality: it treats the bf16 output as ground truth, so it penalises valid paraphrase and would penalise a better answer.

What was not measured

• No judged quality. In the open-ended set 11/18 (bf16) and 14/18 (8-bit) answers hit the 640-token cap, so grading them would have measured truncation rather than quality. This model emits a <think> block, which consumes budget.
• No agentic evaluation. No WebArena, no ScreenSpot, no WebVoyager, no end-to-end computer-use task. This model's actual job is multi-step browser automation and none of that is scored here.
• No standard benchmarks (MMLU, GSM8K). The accuracy layer is 8 short verifiable prompts, not a benchmark.
• The vision check is one screenshot, not a dataset, and it was scored for correctness by reading it, not by a metric.
• Throughput figures are for one M2 Pro and do not transfer to other chips.

Related conversions

Credits

All credit for the underlying model belongs to Microsoft. This is a format conversion and quantization; no training or fine-tuning was performed. See the source model card for license, intended use and limitations.