Downloads · 30 days
45
1% of all-time downloads
vlbosch/Qwen3-235B-A22B-MLX-mixed-4bit
Qwen3-235B-A22B-MLX-mixed-4bit is a machine learning model from vlbosch. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
This is a mixed MLX-quantization of Qwen 3 235B-A22B based on the recipe for the ikllama.cpp quant by ubergarm (https://huggingface.co/ubergarm/Qwen3-235B-A22B-GGUF) In my own experience, this quant performs better th…
Downloads · 30 days
45
1% of all-time downloads
All-time downloads
4.1K
Public
Parameters
235B
116 GB on disk
Likes
3
Public
Click a slice to open those files.
.safetensors116 GB · 100%
How the weights are stored.
U32235B · 100%
From the Hugging Face model README
This is a mixed MLX-quantization of Qwen 3 235B-A22B based on the recipe for the ik_llama.cpp quant by ubergarm (https://huggingface.co/ubergarm/Qwen3-235B-A22B-GGUF) In my own experience, this quant performs better than a standard 4-bit MLX-quant with group-size 128.
I can run this quite comfortably on an M3 Max with 128GB RAM with full context (40k tokens) using "sudo sysctl iogpu.wired_limit_mb=121000"
Let me know what your experience is compared to other quants! :-)