Huihui-GLM-5.1-abliterated-MLX-3.9bit

Huihui GLM 5.1 abliterated optimized to run on a Mac Studio M3 512. Non-abliterated versions here: larger, smaller.

• This is NOT a faithful recreation of the original GGUF, so much as a "I wonder if..." science project. It worked! But YMMV.
• Converted from a Q3_K GGUF, with important layers merged with Unsloth's UD_Q3_K_XL to offset quantization loss.
• Fits into ~360 GB memory, leaving plenty of room to run parallel models (ex: Qwen 3.6 35B).

Usage

# Start server at http://localhost:8080/chat/completions
uvx --from mlx-lm mlx_lm.server \
  --host 127.0.0.1 \
  --port 8080 \
  --model spicyneuron/Huihui-GLM-5.1-abliterated-MLX-3.9bit

Benchmarks

Tested on a Mac Studio M3 Ultra with:

mlx_lm.kld --baseline-model path/to/mlx-full-precision
mlx_lm.perplexity --sequence-length 2048 --seed 123
mlx_lm.benchmark --prompt-tokens 1024 --generation-tokens 512 --num-trials 5
mlx_lm.evaluate --tasks piqa --seed 123 --num-shots 0 --limit 500

Note:

• mlx_lm.kld is approximate, based on top_k not full logits. Here's the code.
• GLM 5.1 KL divergence calculated against the largest quant I could run locally (~495 GB), so real KL is higher.

Methodology

Created with a custom workflow that:

1. Compared GGUF quants for similarity
2. Merged select higher-quant layers
3. Dequantized to F32
4. Requantized to MLX