Qwen3.8-27B-heretic-VL-MLX-4bit

Qwen3.8-27B-heretic-VL-MLX-4bit

4-bit quantized MLX conversion of trohrbaugh/Qwen3.8-27B-heretic-ara, with the vision tower intact, for Apple Silicon.

Vision works — and was tested

This build keeps the vision encoder: vision_config is present and 333 of the 2180 tensors belong to vision_tower. It reads images.

Verified rather than assumed. A generated test image (red square left, blue circle centre, green triangle right, the caption MAC512 VISION TEST, and the code 7391-ZQ) was fed to all four builds. Every one named the three shapes with the right colours and positions and read both strings back exactly.

Requires mlx-vlm, not mlx-lm. mlx-lm converts the language tower only and would silently drop vision. Text-only siblings live at Qwen3.8-27B-heretic-MLX-* — smaller and slightly leaner if you never send images.

What this is

The base model is an abliterated ("heretic") derivative of Qwen/Qwen3.8-27B: its refusal behaviour has been surgically removed, so it will answer prompts a safety-tuned model would decline. This repo only changes the format and precision — it adds no alignment, and removes none either. Evaluate it before putting it in front of users, and apply your own filtering where your use case needs it.

Lineage: Qwen/Qwen3.8-27B → trohrbaugh/Qwen3.8-27B-heretic-ara → this MLX build.

The full set

Measured on a Mac Studio M3 Ultra (512 GB unified memory), macOS 26.5.2, mlx-vlm 0.6.13, 470-token multimodal prompt (image + text). Single run, one machine, one prompt — an order-of-magnitude guide, not a benchmark. Prompt processing ran at 303-331 tok/s across all four builds; the spread between builds shows up in generation, not prefill.

This repo is the 4-bit build: 15.0 GB, 3 shards, 38.9 tok/s, 19.2 GB peak memory.

Peak memory runs roughly 4-6 GB above the text-only sibling — that is the vision tower plus image-patch activations.

Usage

pip install mlx-vlm

mlx_vlm.generate --model donedynamics/Qwen3.8-27B-heretic-VL-MLX-4bit \\
  --image photo.png --prompt "What does this image show?" --max-tokens 256

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("donedynamics/Qwen3.8-27B-heretic-VL-MLX-4bit")
prompt = apply_chat_template(processor, model.config, "What does this image show?", num_images=1)
print(generate(model, processor, prompt, ["photo.png"], max_tokens=256))

Reasoning mode

The chat template supports enable_thinking and reasoning_effort. Thinking is on by default and consumes tokens before the answer begins, so a small max_tokens budget may return reasoning only.

Conversion

mlx_vlm.convert --hf-path trohrbaugh/Qwen3.8-27B-heretic-ara -q --q-bits 4 \\
  --mlx-path Qwen3.8-27B-heretic-VL-MLX-4bit

Converted with mlx-vlm 0.6.13 from the bf16 weights at revision a67ae100d933c0d17af3232bda35825979fc63ce. Before conversion the source was verified: 7 shards, 1199 tensors, all safetensors headers parsed, no missing files.

License and attribution

Apache-2.0, inherited through the lineage above. Credit for the model goes to the Qwen team at Alibaba Group's Tongyi Lab, and for the abliteration to trohrbaugh. This repo contributes only the MLX conversion.