Downloads · 30 days
26
3% of all-time downloads
Qwen3.5-397B-A17B-TurboQuant-MLX-2bit
majentik/Qwen3.5-397B-A17B-TurboQuant-MLX-2bit
Qwen3.5-397B-A17B-TurboQuant-MLX-2bit is a image-text-to-text model from majentik. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
2-bit MLX weight-quantized build of Qwen/Qwen3.5-397B-A17B (397B total / 17B active Sparse MoE, multimodal) — re-quantized from the 4-bit TurboQuant MLX checkpoint for maximum compression. Optimized for Apple Silicon…
All-time downloads
765
Public
Parameters
397B
125 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors125 GB · 100%
Parameter types
How the weights are stored.
U32396B · 100%
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3.5-397B-A17B-TurboQuant-MLX-2bit
2-bit MLX weight-quantized build of Qwen/Qwen3.5-397B-A17B (397B total / 17B active Sparse MoE, multimodal) — re-quantized from the 4-bit TurboQuant MLX checkpoint for maximum compression. Optimized for Apple Silicon via MLX.
This is an experimental extreme-compression variant intended for running a ~400B MoE model on high-end consumer Apple Silicon. Expect noticeable quality degradation vs 4-bit — test on your workload before relying on it.
Quickstart
from mlx_lm import load, generate
model, tokenizer = load("majentik/Qwen3.5-397B-A17B-TurboQuant-MLX-2bit")
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": "Give me a one-sentence description of MoE routing."}],
add_generation_prompt=True,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=128, verbose=True))
Model Specs
| Property | Value |
|---|---|
| Base model | Qwen/Qwen3.5-397B-A17B |
| Architecture | Sparse Mixture-of-Experts (MoE) |
| Total parameters | 397B |
| Active per token | 17B |
| Modalities | Image + Text → Text (image-text-to-text) |
| Context window | 256K tokens |
| Weight quantization | 2-bit MLX (re-quantized from 4-bit TurboQuant) |
| Approx. disk footprint | ~135 GB |
| License | Apache 2.0 |
RotorQuant vs TurboQuant
| Aspect | TurboQuant (this repo) | RotorQuant |
|---|---|---|
| Rotation | Randomized Hadamard (static) | Learned orthogonal rotors (data-calibrated) |
| Calibration | Zero-shot | ~512 sample calibration pass |
| Accuracy @ 2-bit | ~93–95% of FP16 baseline (task-dependent) | ~95–97% of FP16 baseline (task-dependent) |
| Best for | Squeezing the model into small VRAM | Squeezing the model in with the best quality |
Memory Estimates (2-bit MLX)
| Context | Active memory (approx.) |
|---|---|
| 8K | ~143 GB |
| 32K | ~153 GB |
| 128K | ~183 GB |
| 256K | ~213 GB |
Hardware Requirements
- Minimum: Apple Silicon with 192 GB unified memory for short/medium contexts
- Recommended: 256 GB+ unified memory for full 256K context
- Fits on top-end Mac Studio M-series configurations; does not fit on 96 GB or 128 GB Macs
Caveats
- Re-quantized from the 4-bit TurboQuant MLX checkpoint (not directly from FP16)
- Expect visible regressions on multi-step reasoning, code generation, and multilingual tasks vs 4-bit
- For production use, prefer the 4-bit or higher variants when your hardware allows
See Also
- RotorQuant MLX 2-bit: majentik/Qwen3.5-397B-A17B-RotorQuant-MLX-2bit
- Base model: Qwen/Qwen3.5-397B-A17B
- Task
- Image-Text-to-Text
- Library
- mlx
- Type
- qwen3_5_moe
- License
- apache-2.0
- Languages
- en, zh, multilingual
- Created
- Apr 15, 2026
- Updated
- Jul 20, 2026