Downloads · 30 days
55
5% of all-time downloads
pirola/K2-Horizon-MoVA-36B-A4B-exl3-4.0bpw-hq
K2-Horizon-MoVA-36B-A4B-exl3-4.0bpw-hq is a text generation model from pirola. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
This is an EXL3 quantization of IFM/K2-Horizon-MoVA-36B-A4B. It is a community conversion, not affiliated with IFM. For the model itself (training, benchmarks, chat template, tool-call formats), see the original model…
Downloads · 30 days
55
5% of all-time downloads
All-time downloads
1K
Public
Parameters
10.2B
20.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors20.5 GB · 99%
How the weights are stored.
I169.5B · 93%
From the Hugging Face model README
This is an EXL3 quantization of IFM/K2-Horizon-MoVA-36B-A4B. It is a community conversion, not affiliated with IFM. For the model itself (training, benchmarks, chat template, tool-call formats), see the original model card.
| Source | IFM/K2-Horizon-MoVA-36B-A4B, BF16 (69.8 GiB) |
| Target | 4.0 bpw with -hq; final 4.11 bpw excluding the head; head 6-bit |
| Bit allocation | attention (q/k/o/gate_proj), dense v_proj and shared experts at 6 bits; routed MoE experts and MoVA value experts at 4 bits |
| Size | 19.1 GiB (3 shards) |
| Calibration | exllamav3's built-in calibration mix, 250 rows x 2048 tokens (IFM's own training datasets are not public) |
| Codebook | mul1 |
| Converter | exllamav3 v1.5.1 plus the k2_horizon architecture patch in conversion/ |
exllamav3 v1.5.1 has no k2_horizon architecture. K2-Horizon's Mixture-of-Values (MoVA) attention replaces the V projection with 64 routed value experts (top-4). To load this model, apply the patch in conversion/k2-horizon-exllamav3-v1.5.1.patch to exllamav3 v1.5.1. The patch is Python-only and needs no kernel rebuild; the release wheel's extension works.
git clone --branch v1.5.1 https://github.com/turboderp-org/exllamav3
cd exllamav3 && git apply /path/to/k2-horizon-exllamav3-v1.5.1.patch
# install the matching exllamav3 1.5.1 wheel for your torch/CUDA, then run with PYTHONPATH=$PWD
What the patch implements:
The exact conversion command and environment are in conversion/run.sh, and the full converter log in conversion/convert.log. Command:
python convert.py -i K2-Horizon-MoVA-36B-A4B -o out -w work -b 4.0 -hq
Against the original model (IFM's modeling_k2_horizon.py, fp32 reference) on layers 0-4 of a truncated 5-layer model:
Full-model validation is in progress: KL divergence and perplexity against BF16, per-layer routing agreement over all 48 layers, and long-context needle tests up to 256K with FP16 and quantized KV caches. The results will be added to validation/ in this repo. Until then, treat quality as unverified on the full model.
IFM notes that the router is numerically sensitive: their SGLang recipe uses an override "which preserves the checkpoint's router numerics". In this conversion the MoE router weights are stored in fp16 (converted from bf16) and computed in fp32.
Apache-2.0, as the original model. All credit for the model goes to IFM.