Downloads · 30 days
42
45% of all-time downloads
grug-35b-v2-MLX-4bit
pgmharikrishnan/grug-35b-v2-MLX-4bit
grug-35b-v2-MLX-4bit is a text generation model from pgmharikrishnan. Use it when you need the model to write or continue text. It is set up for mlx-serve. The card lists the license as apache-2.0.
An Apache-2.0 MLX affine quantization of ProCreations/grug-35b-v2, matching the base model's license.
All-time downloads
93
Public
Parameters
34.7B
19.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors19.5 GB · 100%
Parameter types
How the weights are stored.
U3234.7B · 100%
Try a prompt
Base models
Model card
From the Hugging Face model README
grug 35B v2 for mlx-serve (4-bit)
An Apache-2.0 MLX affine quantization of ProCreations/grug-35b-v2, matching the base model's license.
This is a text-only pack; it does not include the source model's vision assets.
Size and quantization
Safetensors payload: 19.51 GB / 18.17 GiB. Affine 4-bit quantization uses group size 64. MoE router gates and shared-expert gates are retained at 8-bit; norms, scales, and biases remain BF16.
Benchmarks
Benchmarks pending a reproducible rerun.
Usage
Requires mlx-serve v26.10.2 or later.
mlx-serve serve --model pgmharikrishnan/grug-35b-v2-MLX-4bit
- Task
- Text Generation
- Library
- mlx-serve
- Type
- qwen3_5_moe
- License
- apache-2.0
- Created
- Sep 7, 2026
- Updated
- Oct 15, 2026