Qwopus3.8-27B-Flash-OptiQ-4bit

Qwopus3.8-27B-Flash-OptiQ-4bit (MLX, mixed-precision)

A mixed-precision 3.97 bpw quantized MLX build of Jackrong/Qwopus3.8-27B-Flash, produced with OptiQ (data-driven, sensitivity-based quantization), packaged in the standard MLX layout with the vision tower restored into the main shards and a working vision chat template.

Attribution

• Model author (original weights): Jackrong — Jackrong/Qwopus3.8-27B-Flash
• Base model: Qwen/Qwen3.8-27B (by Alibaba/Qwen)
• Quantization tool: OptiQ (mlx-optiq 0.4.22)
• Quantized/MLX packaging: ObviousSalamander (this repo)

This repo contains no new training — it is a quantization of Jackrong's published weights. The weights and model architecture are Jackrong's work; only the OptiQ conversion and MLX packaging are ours. All credit for the model itself goes to Jackrong and the Qwen team.

What this is

• Trunk: Jackrong/Qwopus3.8-27B-Flash (bf16, ~52 GB, 18 shards) — Jackrong's efficiency-tuned fine-tune of the dense multimodal Qwen/Qwen3.8-27B (Qwen3_5ForConditionalGeneration, vision + language + MTP draft head).
• Method: OptiQ 0.4.22, optiq convert --target-bpw 4.75 --candidate-bits 4,8. OptiQ ran a per-layer KL-sensitivity analysis and chose the bit width per layer.
• Result: 3.97 bpw, mixed 4-bit + 8-bit layers. ~12.7 GB of weights (this repo ≈ 20 GB on disk including the MTP head and vision tower).
• Layout: standard MLX — model-*.safetensors + model.safetensors.index.json. Vision tower is included as model.visual.* in the shards.

⚠️ This is NOT a uniform 4-bit model. The "4bit" in the name refers to the target budget (4.75 bpw); OptiQ protected the most quality-sensitive layers at 8-bit. In particular, lm_head and embed_tokens are held at 8-bit (group_size 64) so the output head stays accurate — the same strategy Youssofal uses in their MTPLX Optimized-Speed packs of the sibling Qwen3.8-27B.

Why this build

Jackrong's -Flash fine-tune explicitly trades a little accuracy for much lower reasoning cost and faster agent-loop completion (their headline: ~13% faster decoding, ~81% MTP draft acceptance, less runaway reasoning). Our goal was to preserve that efficiency profile on Apple Silicon while keeping the quality-sensitive parts — especially the output head — as exact as possible.

Two decisions shaped the result:

1. 8-bit head (not bf16, not 4-bit). lm_head + embed_tokens (vocab 248,320) are the single most error-sensitive tensors in next-token prediction. OptiQ's per-layer sensitivity analysis promoted them to 8-bit g64 automatically. We deliberately did not graft bf16 heads back (which would add ~2–3 GB and slow the load) because the Flash model is built for speed; 8-bit on a 5120-dim projection already carries near-negligible error.
2. Vision template restored. Jackrong's published chat_template.jinja is text-only and cannot render the multimodal content lists ([{type:image},…]) that mlx-vlm sends, so out-of-the-box vision generation crashes with a TypeError. We replaced it with the proper vision-capable template from the base Qwen/Qwen3.8-27B (same architecture, same image/ video token ids, same vocab). Vision now works correctly (verified).

Why this was uploaded

This build took a long time to produce (download of the ~52 GB bf16 original, the ~12-hour OptiQ sensitivity conversion on a Mac mini M4 Pro, the vision-sidecar re-merge into a standard MLX layout, and the chat-template fix). It is published so anyone can use the result directly — or audit the specific OptiQ recipe + 8-bit-head decision — without repeating that work.

Verifying / using

# text
python -m mlx_lm generate --model ObviousSalamander/Qwopus3.8-27B-Flash-OptiQ-4bit \
  --prompt "What is 2+2?" --max-tokens 40

# vision (uses the restored base vision template)
python -m mlx_vlm.generate \
  --model ObviousSalamander/Qwopus3.8-27B-Flash-OptiQ-4bit \
  --image <image.png> --prompt "Describe this image." --max-tokens 60

Measured locally on a Mac mini M4 Pro (64 GB) with mlx-vlm: text ~13 tok/s decode, ~19.5 GB peak; vision ~13 tok/s decode, ~22.4 GB peak. An MTP draft head is included in optiq/mtp.safetensors (forge contract fields in config.json) for MTPLX-style speculative decoding.

Files

• model-0000{1-5}-of-00005.safetensors — quantized weights (4 language shards + 1 vision shard, all model.visual.* folded in)
• optiq/mtp.safetensors — MTP draft head
• chat_template.jinja — base Qwen3.8-27B vision-capable template (fixes upstream bug)
• config.json, processor_config.json, tokenizer*

License

The original Qwopus3.8-27B-Flash model inherits the Apache-2.0 license of its Qwen base. This MLX quantized build is distributed under the same terms.