Downloads · 30 days
14
1% of all-time downloads
caiovicentino1/Harmonic-27B-HLWQ-Q5
Harmonic-27B-HLWQ-Q5 is a text generation model from caiovicentino1. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
[!IMPORTANT] Naming notice (2026-04-10). The "HLWQ" technique used in this model is being rebranded to HLWQ (Hadamard-Lloyd Weight Quantization). The change is only the name; the algorithm and the weights in this repo…
Downloads · 30 days
14
1% of all-time downloads
All-time downloads
945
Public
Parameters
25.8B
15.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors15.1 GB · 100%
How the weights are stored.
I3224.3B · 94%
From the Hugging Face model README
[!IMPORTANT] Naming notice (2026-04-10). The "HLWQ" technique used in this model is being rebranded to HLWQ (Hadamard-Lloyd Weight Quantization). The change is only the name; the algorithm and the weights in this repository are unchanged.
The rebrand resolves a name collision with an unrelated, earlier KV cache quantization method also named HLWQ (Han et al., arXiv:2502.02617, 2025). HLWQ addresses weight quantization with a deterministic Walsh-Hadamard rotation and Lloyd-Max scalar codebook; Han et al.'s HLWQ addresses KV cache quantization with a random polar rotation. The two methods are technically distinct.
Existing loaders that load this repository by ID continue to work without changes. Future model uploads will use the HLWQ name.
Reference paper for this technique: arXiv:2603.29078 (v2 in preparation; v1 still uses the old name).
Native vLLM. Marlin kernel. Zero plugin.
HLWQ Q5 preprocessing produces better INT4 weights than direct quantization — stored in CompressedTensors format for native vLLM inference.
pip install vllm
vllm serve caiovicentino1/Harmonic-27B-HLWQ-Q5 --language-model-only --enforce-eager
That's it. No plugin, no pip install polarquant, no custom code.
Tested results:
| GPU | tok/s |
|---|---|
| A100 80GB | 168 tok/s (9B) |
| RTX PRO 6000 96GB | 44 tok/s (9B) / 18 tok/s (27B) |
pip install polarquant
import polarengine_vllm # auto-registers with transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("caiovicentino1/Harmonic-27B-HLWQ-Q5", device_map="auto", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("caiovicentino1/Harmonic-27B-HLWQ-Q5", trust_remote_code=True)
inputs = tokenizer("Hello!", return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(out[0], skip_special_tokens=True))
| GPU | VRAM | Works? | Expected tok/s |
|---|---|---|---|
| RTX 4090 | 24 GB | YES (tight) | ~10 |
| A100 / H100 | 80 GB | YES | ~18-50 |
| RTX PRO 6000 | 96 GB | YES | ~18 |
Standard INT4 (GPTQ/AWQ) quantizes weights directly — outliers cause errors.
HLWQ adds a preprocessing step:
| Method | PPL (lower = better) |
|---|---|
| BF16 baseline | 6.37 |
| HLWQ → INT4 | 6.56 |
| Direct INT4 | 6.68 |
Same speed as GPTQ/AWQ, better quality.
| Flag | Why |
|---|---|
--language-model-only | Qwen3.5 is multimodal — this skips the vision encoder (we only quantized text) |
--enforce-eager | Required on Blackwell GPUs (cc 12.0). Optional on A100/H100 (faster without it) |
pip install polarquant