Qwen-9B-Claude-Fable-5-1M-MLX-4bit

<p align="center"> <img src="https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M/resolve/main/assets/qwythos.png" alt="Base model artwork" width="550"/> </p>

Qwen-9B-Claude-Fable-5-1M-MLX-4bit

A 4-bit MLX quantization of empero-ai/Qwythos-9B-Claude-Mythos-5-1M, published as Qwen-9B-Claude-Fable-5-1M-MLX-4bit and optimized for Apple Silicon with a minimal memory footprint. LM Studio compatibilty fixed, please update.

Model Details

• Base model: empero-ai/Qwythos-9B-Claude-Mythos-5-1M
• Architecture: Qwen 3.5 (32 layers, 4096 hidden, 16 heads, 4 KV heads)
• Quantization: 4-bit affine, group size 64 (~4.5 bits per weight)
• Context length: 1M tokens (native)
• Model size: ~5.04 GB (safetensors)
• Chat template: Updated to the upstream v3 template hotfix for preserved reasoning, long generations, and agentic/tool-use harnesses.

Usage (MLX)

sfw pip install mlx-lm

Remove sfw if you don't have sfw from Socket installed. It's free and an improvement over running pip or npm raw.

from mlx_lm import load, generate

model, tokenizer = load("shamsghi/Qwen-9B-Claude-Fable-5-1M-MLX-4bit")
response = generate(model, tokenizer, prompt="Hello!", max_tokens=256)
print(response)

Benchmarks

See base model card for evaluation results. 4-bit quantization may introduce minor quality degradation vs. 8-bit.

License

Apache 2.0 — same as the base model.