Downloads · 30 days
3.3K
5% of all-time downloads
NANI-Nithin/K2-Horizon-MoVA-36B-A4B-GGUF
K2-Horizon-MoVA-36B-A4B-GGUF is a text generation model from NANI-Nithin. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
GGUF quantizations of IFM/K2-Horizon-MoVA-36B-A4B, produced with the MBZUAI-IFM fork of llama.cpp (model/K2Horizon branch) that adds native k2-horizon architecture support.
Downloads · 30 days
3.3K
5% of all-time downloads
All-time downloads
65K
Public
Repo size
623 GB
Likes
43
Public
Click a slice to open those files.
.gguf623 GB · 100%
From the Hugging Face model README
GGUF quantizations of IFM/K2-Horizon-MoVA-36B-A4B,
produced with the MBZUAI-IFM fork of llama.cpp
(model/K2Horizon branch) that adds native k2-horizon architecture support.
Architecture note: Upstream llama.cpp does not yet support the
k2_horizon/K2HorizonForCausalLMarchitecture (PR in progress as of September 2026). You must use the MBZUAI-IFM fork linked above — or upstream once the PR merges — to load these GGUFs.
K2-Horizon-MoVA-36B-A4B is the sparse member of the K2-Horizon family released by the Institute of Foundation Models (IFM) at MBZUAI on 3 September 2026. It is a Mixture-of-Experts (MoE) model augmented with a novel Mixture-of-Values (MoVA) attention mechanism, storing ~36 B parameters while activating only ~4 B per token.
| Property | Value |
|---|---|
| Architecture | k2_horizon (K2HorizonForCausalLM) |
| Total parameters | ~36 B |
| Active parameters / token | ~4 B |
| Layers | 48 (3 leading dense + 45 MoE) |
| Hidden size | 2 560 |
| MoE routing | 100 routed experts / 8 active + 1 shared expert per MoE layer |
| MoVA attention | 64 value experts / 4 active per attention layer |
| Context window | 512 K (524 288 tokens, native) |
| Embedding tying | Untied (separate input / output embeddings) |
| Precision (source) | BF16 |
| License | Apache 2.0 |
Mixture-of-Values (MoVA) applies the same sparse routing philosophy as MoE, but inside the attention mechanism: the value projections are replaced by a learned pool of value experts, and only a small subset is activated per token. This lets the model store a much richer set of representational "styles" in the value space without paying for all of them at inference time, complementing the feed-forward MoE sparsity.
| File | Quant type | Approx. size | Notes |
|---|---|---|---|
K2-Horizon-MoVA-36B-A4B-BF16.gguf | BF16 | ~70 GiB | Original precision; quantization source |
K2-Horizon-MoVA-36B-A4B-Q8_0.gguf | Q8_0 | ~35 GiB | Near-lossless |
K2-Horizon-MoVA-36B-A4B-Q6_K.gguf | Q6_K | ~27 GiB | Very high quality |
K2-Horizon-MoVA-36B-A4B-Q5_K_M.gguf | Q5_K_M | ~23 GiB | High quality, recommended |
K2-Horizon-MoVA-36B-A4B-Q5_K_S.gguf | Q5_K_S | ~22 GiB | |
K2-Horizon-MoVA-36B-A4B-Q5_1.gguf | Q5_1 | ~22 GiB | |
K2-Horizon-MoVA-36B-A4B-Q5_0.gguf | Q5_0 | ~21 GiB | |
K2-Horizon-MoVA-36B-A4B-Q4_K_M.gguf | Q4_K_M | ~20 GiB | Good balance of size and quality |
K2-Horizon-MoVA-36B-A4B-Q4_K_S.gguf | Q4_K_S | ~19 GiB | |
K2-Horizon-MoVA-36B-A4B-Q4_1.gguf | Q4_1 | ~20 GiB | |
K2-Horizon-MoVA-36B-A4B-Q4_0.gguf | Q4_0 | ~19 GiB | |
K2-Horizon-MoVA-36B-A4B-Q3_K_L.gguf | Q3_K_L | ~16 GiB | |
K2-Horizon-MoVA-36B-A4B-Q3_K_M.gguf | Q3_K_M | ~15 GiB | |
K2-Horizon-MoVA-36B-A4B-Q3_K_S.gguf | Q3_K_S | ~14 GiB | |
K2-Horizon-MoVA-36B-A4B-Q2_K.gguf | Q2_K | ~10 GiB | Smallest; used as imatrix source |
K2-Horizon-MoVA-36B-A4B-IQ4_NL.gguf | IQ4_NL | ~20 GiB | imatrix-calibrated |
K2-Horizon-MoVA-36B-A4B-IQ4_XS.gguf | IQ4_XS | ~19 GiB | imatrix-calibrated |
K2-Horizon-MoVA-36B-A4B-IQ3_M.gguf | IQ3_M | ~16 GiB | imatrix-calibrated |
K2-Horizon-MoVA-36B-A4B-IQ3_XS.gguf | IQ3_XS | ~14 GiB | imatrix-calibrated |
K2-Horizon-MoVA-36B-A4B-IQ3_XXS.gguf | IQ3_XXS | ~13 GiB | imatrix-calibrated |
K2-Horizon-MoVA-36B-A4B-IQ2_M.gguf | IQ2_M | ~11 GiB | imatrix-calibrated |
IQ quants (those beginning with IQ) were generated using an importance matrix
computed on the Q2_K quant with 8 × 512-token chunks of WikiText-2 calibration data
(Salesforce/wikitext).
All quants were produced from the BF16 GGUF published by IFM at IFM/K2-Horizon-MoVA-36B-A4B-GGUF. No safetensors conversion was performed; the IFM BF16 GGUF already includes tokenizer metadata and a llama.cpp-compatible chat template.
These figures are weight-only estimates and exclude KV cache, runtime overhead, and context length. They are not minimum VRAM/RAM requirements.
| Quant | Est. weight size | Practical minimum RAM/VRAM |
|---|---|---|
| BF16 | ~70 GiB | 2 × 48 GB GPU (A6000 / H100 80 GB) |
| Q8_0 | ~35 GiB | 2 × 24 GB GPU |
| Q5_K_M | ~23 GiB | 1 × 24 GB GPU or 2 × 16 GB |
| Q4_K_M | ~20 GiB | 1 × 24 GB GPU |
| Q3_K_M | ~15 GiB | 1 × 16 GB GPU |
| Q2_K / IQ2_M | ~10–11 GiB | 1 × 12 GB GPU |
Memory caveat: "4B active" means 4B parameters are computed per token, not that the model fits in 4B-worth of memory. All 36B weights must be resident in RAM/VRAM.
Prerequisite: build from the MBZUAI-IFM fork (
model/K2Horizonbranch) until upstream llama.cpp merges the PR.
git clone --depth 1 --branch model/K2Horizon \
https://github.com/MBZUAI-IFM/llama.cpp.git llama.cpp-k2horizon
cd llama.cpp-k2horizon
cmake -B build -DGGML_NATIVE=OFF
cmake --build build -j4
Basic inference:
./build/bin/llama-cli \
-m K2-Horizon-MoVA-36B-A4B-Q4_K_M.gguf \
-p "Explain the difference between MoE and MoVA architectures." \
-n 1024 \
--temp 1.0 \
--top-p 0.95 \
-c 8192
The original BF16 safetensors checkpoint can be served with vLLM or SGLang.
vLLM:
vllm serve IFM/K2-Horizon-MoVA-36B-A4B \
--revision main \
--tensor-parallel-size 2 \
--enable-expert-parallel \
--trust-remote-code \
--dtype bfloat16 \
--max-model-len 131072 \
--reasoning-parser k2_horizon \
--tool-call-parser k2_horizon \
--enable-auto-tool-choice
SGLang (validated on 2 × H200):
python3 -m sglang.launch_server \
--model-path IFM/K2-Horizon-MoVA-36B-A4B \
--revision main \
--tp 2 \
--ep 2 \
--dtype bfloat16 \
--attention-backend fa3 \
--json-model-override-args '{"xllm_source_router_gemm_partitions":2}' \
--reasoning-parser k2_horizon \
--tool-call-parser k2_horizon \
--host 0.0.0.0 --port 30000
reasoning_effort="high" — all reported benchmark scores were
obtained at high reasoning effort.temperature=1.0, top_p=0.95,
max_new_tokens ≥ 32 768.k2_horizon tool-call parser when using
the vLLM or SGLang API.-c 8192 or -c 32768 unless you have sufficient RAM.Results from the IFM model card. All scores in %. Bold = best in row. K2-Horizon-MoVA-36B-A4B uses high reasoning effort; other open models are in their respective reasoning modes.
| Benchmark | K2-Horizon-MoVA-36B-A4B (4B active) | Nemotron 3 Ultra (55B active) | Nemotron 3 Super (12B active) | G9v3-39A5B (5B active) | Qwen3.6-35B-A3B (3B active) | Muse Glimmer-30B (Dense) | Gemma 4 31B-it (Dense) |
|---|---|---|---|---|---|---|---|
| Agents | |||||||
| tau3-Banking (tool use) | 26.8 | 14.2 | 10.3 | 22.1 | 9.3 | 23.5 | 14.8 |
| Coding | |||||||
| Terminal-Bench 2.1 (agentic) | 58.6 | 53.9 | 38.6 | 32.6 | 44.9 | 51.7 | 43.4 |
| SciCode (scientific coding) | 38.9 | 39.9 | 36.0 | 34.0 | 35.8 | 43.6 | 43.4 |
| Scientific Reasoning | |||||||
| Humanity's Last Exam | 25.2 | 28.4 | 20.8 | 17.5 | 22.2 | 22.0 | 23.6 |
| GPQA Diamond | 80.8 | 86.7 | 80.0 | 80.5 | 84.1 | 83.5 | 85.7 |
| CritPt (frontier physics) | 2.1 | 3.1 | 3.1 | 0.3 | 0.3 | 2.6 | 1.4 |
| General | |||||||
| AA-LCR (long-context) | 66.3 | 71.0 | 60.3 | 62.0 | 66.7 | 80.0 | 68.3 |
| AA-Omniscience Accuracy | 18.8 | 22.6 | 24.3 | 14.9 | 18.8 | 27.0 | 20.0 |
| AA-Omniscience Non-Hallucination | 69.2 | 70.3 | 13.0 | 87.0 | 49.5 | 18.1 | 15.0 |
These GGUFs were generated with the following pipeline, mirroring the approach used for other large MoE checkpoints in this workspace:
model/K2Horizon branch is cloned to a
dedicated directory and compiled with CMake (GGML_NATIVE=OFF for portability).IFM/K2-Horizon-MoVA-36B-A4B-GGUF instead of converting from safetensors
(a direct BF16 or Q8_0 conversion would exceed the ~109 GiB free on /tmp).BF16 (~70 GiB) + one quant at any given moment.status.json file tracks which quants have been generated and
uploaded so interrupted runs resume from where they left off.The original model and these GGUF quantizations are released under the Apache 2.0 License. See LICENSE on the upstream repository for full terms.
@misc{k2horizon2026,
title = {Introducing K2 Horizon: Frontier Performance, Radically Open},
author = {{IFM Team}},
year = {2026},
url = {https://ifm.ai/blog/k2/},
}