Qwen-Image-Bench-MLX-4bit

Qwen-Image-Bench - MLX 4-bit

4-bit weight-quantized MLX version of Qwen/Qwen-Image-Bench — Qwen's 27B vision-language judge for text-to-image evaluation (built on Qwen3.6-27B). It scores prompt/image pairs across 5 hierarchical dimensions (Quality, Aesthetics, Alignment, Real-world Fidelity, Creative Generation) and emits JSON-formatted results. The language tower is 4-bit quantized; the vision tower is kept in BF16 (unquantized) for judge fidelity. Smoke-verified on Apple Silicon with this exact payload before publishing (see PROVENANCE.md).

Approximate model size: ~16 GB

Model Specifications

Quickstart

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("majentik/Qwen-Image-Bench-MLX-4bit")

prompt = apply_chat_template(
    processor, model.config,
    "Evaluate this generated image against the prompt: 'a red square'. "
    "Score Quality, Aesthetics, Alignment, Real-world Fidelity, Creative "
    "Generation (0=Fail, 1=Pass, 2=Excel) as JSON.",
    num_images=1,
)
output = generate(model, processor, prompt, image=["generated.png"], max_tokens=512)
print(output.text)

For batch judging, see upstream's judge.py and structured checklists.

Variants in this family

No sub-4bit variants are published: judge quality degrades at very low bit widths and a miscalibrated judge is worse than none. Smoke verification covers load + vision-grounded generation gates only; it is not a benchmark of judge calibration — validate against upstream's benchmark data before relying on absolute scores.