Qwen3.5-9B-HLWQ-MLX-4bit

[!IMPORTANT] Naming notice (2026-04-10). The "HLWQ" technique used in this model is being rebranded to HLWQ (Hadamard-Lloyd Weight Quantization). The change is only the name; the algorithm and the weights in this repository are unchanged.

The rebrand resolves a name collision with an unrelated, earlier KV cache quantization method also named HLWQ (Han et al., arXiv:2502.02617, 2025). HLWQ addresses weight quantization with a deterministic Walsh-Hadamard rotation and Lloyd-Max scalar codebook; Han et al.'s HLWQ addresses KV cache quantization with a random polar rotation. The two methods are technically distinct.

Existing loaders that load this repository by ID continue to work without changes. Future model uploads will use the HLWQ name.

Reference paper for this technique: arXiv:2603.29078 (v2 in preparation; v1 still uses the old name).

🍎 HLWQ MLX 4-bit -- Qwen3.5-9B

Run Qwen3.5-9B on your Mac with HLWQ quality -- 19.7 tok/s at only 4.8 GB memory.

HLWQ MLX brings the benefits of Hadamard rotation + Lloyd-Max optimal quantization to Apple Silicon via the MLX framework. Get better-than-naive-Q4 quality with native Metal acceleration.

---

🎯 Key Results

---

📊 Cross-Platform Comparison

Runs comfortably on any Mac with 8 GB+ unified memory. No GPU required -- Metal handles everything.

---

🚀 Quick Start

Option 1: mlx-lm (Easiest)

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("caiovicentino1/Qwen3.5-9B-HLWQ-MLX-4bit")

response = generate(
    model, tokenizer,
    prompt="Explain the theory of relativity in simple terms:",
    max_tokens=200,
    verbose=True  # Shows tok/s
)
print(response)

Option 2: Command Line

mlx_lm.generate \
    --model caiovicentino1/Qwen3.5-9B-HLWQ-MLX-4bit \
    --prompt "Write a Python function to sort a list:" \
    --max-tokens 300

Option 3: Chat Mode

mlx_lm.chat \
    --model caiovicentino1/Qwen3.5-9B-HLWQ-MLX-4bit \
    --max-tokens 500

---

🔬 Why HLWQ on MLX?

Standard MLX quantization uses simple absmax rounding. HLWQ improves on this by:

1. Hadamard Rotation: Transforms weight blocks to follow a Gaussian distribution using a deterministic 128x128 Walsh-Hadamard matrix
2. Lloyd-Max Centroids: Uses MSE-optimal quantization levels for Gaussian data instead of uniform spacing

This combination reduces quantization error by up to 54% compared to absmax at the same bit width.

Original Weights --> Normalize --> Hadamard Rotate --> Lloyd-Max Q4 --> MLX Format
                                                                         |
                       Metal Acceleration <-- MLX Inference <-- Dequant --+

---

🔧 Technical Details

---

💻 System Requirements

---

🔗 Links

• \U0001f4c4 Paper (arXiv) -- HLWQ: Optimal Gaussian Weight Quantization
• 💻 Code (GitHub) -- Full research codebase
• 🖥️ CUDA Version -- For NVIDIA GPUs (43 tok/s)
• \U0001f50c vLLM Plugin -- Production inference integration

---

📖 Citation

@article{vicentino2026polarquant,
  title={HLWQ: Optimal Gaussian Weight Quantization via Hadamard Rotation for LLM Compression},
  author={Vicentino, Caio},
  journal={arXiv preprint arXiv:2603.7424577},
  year={2026}
}

---

🙏 Acknowledgements

Built with MLX by Apple, mlx-lm, and the Qwen team's open-weight models.