Swift-Qwen3.8-27b-oQ8-mtp

Swift-Qwen3.8-27b-oQ8-mtp

Unofficial 8-bit MLX quantization of ukisai/Swift-Qwen3.8-27b for Apple Silicon. The MTP head is retained. This is an MLX safetensors repository, not GGUF or PyTorch.

The source model and its license belong to UkisAI. I am not affiliated with UkisAI. See License and attribution before using or redistributing these weights.

This variant

These values come from the shipped config.json, model.safetensors.index.json and local files. The quantization_config has a uniform 8-bit setting; unlike the oQ4 and oQ6 variants, it contains no module-specific bit overrides. Quantization is lossy, and this variant has no published post-quantization benchmark results.

Pick a variant

All three retain the MTP head. Task-level accuracy and speed after quantization have not been measured here. Runtime memory also depends on context length and KV cache.

Use with oMLX

Download the repository into the oMLX model directory:

hf download TokenAI-zer/Swift-Qwen3.8-27b-oQ8-mtp --local-dir ~/.omlx/models/Swift-Qwen3.8-27b-oQ8-mtp
omlx serve --model-dir ~/.omlx/models --port 8000

The model ID is Swift-Qwen3.8-27b-oQ8-mtp. In oMLX, enable the MTP speculative head to use the included MTP weights. Without that setting, the model can still run but the extra head does not accelerate decoding.

The architecture is multimodal. Use an MLX runtime with Qwen3.5 vision support. The 262k context value is an architecture limit, not a memory guarantee.

Sampling

The shipped generation_config.json comes from the upstream checkpoint. Start with its settings for thinking mode: temperature 1.0, top_p 0.95, top_k 20 and repetition_penalty 1.0. Adjust for your task after checking output quality.

License and attribution

The upstream model identifies its license as Swift Open License v1.0. Its published terms apply to this derivative quantization; this repository does not grant a separate license to the upstream weights. See LICENSE-NOTICE.md for attribution and upstream links.

Citation

@misc{swift-qwen3.8-27b-oq8-mtp,
  title = {Swift-Qwen3.8-27b-oQ8-mtp: 8-bit MLX quantization with MTP head},
  author = {TokenAI-zer},
  year = {2026},
  url = {https://huggingface.co/TokenAI-zer/Swift-Qwen3.8-27b-oQ8-mtp},
  note = {Unofficial quantization of ukisai/Swift-Qwen3.8-27b}
}