Qwythos-9B-v3-Claude-Mythos-5-1M-MLX-oQ8-mtp

Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ8-mtp

An oQ8 quantized MLX version of Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp with integrated Multi-Token Prediction (MTP) head for accelerated speculative decoding on Apple Silicon.

Quantized with oMLX oQ8 — the highest-fidelity mixed-quantization variant that preserves the most layers at near-bf16 quality, providing the best accuracy among oQ variants at the cost of higher memory usage.

Model Summary

• Format: MLX safetensors, oQ8 quantized
• Precision: Mixed (oQ8 smart quantization)
• Parameters: ~9B total
• Context length: 1,048,576 tokens
• Architecture: Qwen3.5-style hybrid attention text model with MTP head
• MTP layers: 1 (embedded, mtp_num_hidden_layers: 1)
• Primary use: local text generation and reasoning on Apple Silicon with MTP-accelerated decoding
• Base model: xunkutech-ai/Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp

Compatibility Notice

oMLX is required to load this model with MTP support. Standard mlx-vlm, mlx-lm, and LM Studio do NOT support MTP and will fail with: ValueError: Received 15 parameters not in model: language_model.mtp.*.

Installation

pip install -U git+https://github.com/jundot/omlx.git

Usage with oMLX

# Start server with MTP draft-mode
omlx serve --model /path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ8-mtp

from omlx import load, generate

model_id = "/path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ8-mtp"
model, tokenizer = load(model_id)

result = generate(
    model=model,
    tokenizer=tokenizer,
    prompt="Explain speculative decoding with MTP.",
    max_tokens=2048,
    temperature=0.6,
)
print(result)

Files

Recommended Sampling

generation_kwargs = {
    "temperature": 0.6,
    "top_p": 0.95,
    "top_k": 20,
    "repetition_penalty": 1.05,
    "max_tokens": 4096,
}

License

Released under the same license as the upstream model: Apache-2.0.

Acknowledgements

• Original model: empero-ai/Qwythos-9B-Claude-Mythos-5-1M
• Original developers: Empero AI
• Base family: Qwen3.5
• MTP support: oMLX
• MLX ecosystem: Apple MLX