Qwythos-9B-Claude-Mythos-5-1M-oQ4-MLX-MTP-Visual

Qwythos-9B-Claude-Mythos-5-1M — oQ4 MLX (MTP + Vision)

4-bit oMLX-quantized MLX conversion of empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF (Qwen3.5-9B, hybrid GatedDeltaNet + attention) with the MTP head and the Qwen3.5-9B vision tower.

Pipeline: MTP-BF16 GGUF → float16 MLX (via popfido/gguf2mlx) → oQ level 4 quantization (oMLX Universal Dynamic Quantization). ~5.8 GB, runs on much less RAM than the full-precision variant. For maximum fidelity use the BF16 repo; for the un-quantized 8-bit source see Q8.

Quantization

• oQ4 — 4-bit base, mixed precision: sensitivity-driven higher bits (5–6) on the GatedDeltaNet in_proj_a/b, out_proj, and down_proj layers; affine, group size 64.
• MTP preserved (preserve_mtp=True): the mtp.* tensors and mtp_num_hidden_layers are kept so the Native MTP toggle works after quantization.
• Minor quality drift vs full precision is expected at 4-bit (e.g. occasional factual slips on greedy decode); core generation is coherent and correct.

The underlying float16 conversion was verified bit-exact vs llama.cpp (text), and the vision tower matches llama.cpp mtmd output — see the BF16 repo card for the full verification + Qwen3.5 conversion notes (zero-centered RMSNorm, A_log = log(−ssm_a), strided→grouped value-heads, YaRN).

Usage

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("popfido/Qwythos-9B-Claude-Mythos-5-1M-oQ4-MLX-MTP-Visual")
prompt = apply_chat_template(processor, model.config, "Describe this image.", num_images=1)
print(generate(model, processor, prompt, image=["image.png"], max_tokens=128).text)

Recommended sampling: temperature 0.6, top_p 0.95, top_k 20, repetition_penalty 1.05.

License & attribution

Inherits apache-2.0 from the base model. All credit for the model to empero-ai; this repo is an MLX-format conversion + oQ quantization only.