Hemmingway-1-OptiQ-4bit

mlx-community/Hemmingway-1-OptiQ-4bit

Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. Try the Lab · All OptiQ quants · Docs

A mixed-precision MLX quant of Altworld/Hemmingway-1, a writing-oriented variant of the Qwen3.5 27B architecture. Sensitive layers are kept at 8-bit and robust ones at 4-bit, rather than crushing everything to a uniform width.

Quantization details

How the bit-widths were chosen

Stated plainly, because it differs from most OptiQ quants: the per-layer allocation was not measured on this model. It was transferred from mlx-community/Qwen3.5-27B-OptiQ-4bit, whose allocation came from a KL-divergence sensitivity sweep over a six-domain calibration mix (prose, reasoning, code, agent, tool-call, instructions).

That transfer is sound here because the two share an architecture exactly — qwen3_5_text, 64 layers, 24 attention heads, 4 KV heads, head_dim 256, hidden 5120, vocab 248,320 — so every layer in the recipe has a counterpart with the same role and shape. All 498 tensors matched with none unmatched, which is the check that matters: an unmatched tensor would silently fall back to flat 4-bit and make this a uniform quant wearing a mixed-precision name.

What sensitivity measures is how much a layer's role in the architecture suffers from precision loss. What it cannot know is whether this model's own training moved that sensitivity around. If you are quantizing your own fine-tune and want the allocation measured against it, run optiq convert and let the sweep do it.

What was verified

• 498/498 tensors matched the recipe, 0 unmatched.
• Generation checked for correctness, not just fluency: factual recall, arithmetic with working shown (240 km in 3 h → 80 km/h), an iterative Fibonacci that runs, and a technical explanation.
• OptiQ's release contract (artifact layout, metadata, mixed-precision assertions).

Not run for this model: the six-metric Capability Score. The published scores for the Qwen3.5-27B quant describe that model, not this one, and are not claimed here.

Prose style

The variant is writing-oriented, and it measures that way against the two signals that actually separate human from machine prose on our labelled set — em-dashes per 1k words and average sentence length. Same three prompts, same sampling, against the base Qwen3.5-27B quant:

Em-dashes do not separate the two. Sentence length does: the base sits on the AI median, this one on the human median. A small probe, not a benchmark.

Use it

pip install mlx-optiq
optiq serve --model mlx-community/Hemmingway-1-OptiQ-4bit

Or with mlx-lm directly:

from mlx_lm import generate, load

model, tokenizer = load("mlx-community/Hemmingway-1-OptiQ-4bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Write three sentences about shipping software."}],
    add_generation_prompt=True, tokenize=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256))

The per-layer bit map is in optiq/metadata.json and in the quantization block of config.json.