Mistral-Small-3.2-24B-Instruct-2506-mlx-8bit

Mistral-Small-3.2-24B-Instruct-2506 — MLX 8-bit (vision-preserving)

An 8-bit MLX quantization of mistralai/Mistral-Small-3.2-24B-Instruct-2506 that keeps the vision tower — a full text + image VLM for Apple Silicon via mlx-vlm.

Why this exists

The other MLX community quants of this model are text-only — the vision tower and chat template were dropped. This one preserves vision: 218 vision_tower tensors, preprocessor_config.json, and chat_template.jinja are all included, so image understanding works, not just text.

Details

• Base: mistralai/Mistral-Small-3.2-24B-Instruct-2506 (Apache-2.0)
• Converted from unsloth/Mistral-Small-3.2-24B-Instruct-2506: the mistralai repo ships Mistral-native format with no HF preprocessor_config.json, so a straight convert drops vision; unsloth's re-upload includes the HF processor + tokenizer configs this quant needs.
• Quant: 8-bit, ~8.85 bits/weight (mlx_vlm.convert -q --q-bits 8)
• Arch: mistral3 (Pixtral-style vision encoder) — the image processor needs torch+torchvision.

Use

pip install mlx-vlm torch torchvision
python -m mlx_vlm.server --model skibare87/Mistral-Small-3.2-24B-Instruct-2506-mlx-8bit --port 8080
# then POST OpenAI-style chat completions with text and/or image_url content

Verified: correctly edits text and describes images.