K2-Horizon-7B-GGUF

K2-Horizon-7B GGUF — imatrix quants

GGUF quantizations of IFM/K2-Horizon-7B, made directly from the publisher's BF16 GGUF rather than from the FP8 checkpoint.

Files

Q6_K is the llama.cpp quant name corresponding to “Q6 K”; there is no Q6_K_M format.

SHA-256 checksums:

5a98a289aba5c8c99ef05c9261287f19e8f586fd679bab45b632eb86b47809bf  K2-Horizon-7B-Q8_0.gguf
4f0449e4582118df22e401b1dd31dcece63d0e431695836e0c5812901e7c51ac  K2-Horizon-7B-Q6_K.gguf
66371e17b1ee10b8ae36c6c1c9c8ebbb29ae80d006f8951e15dfc4bea082fb4a  K2-Horizon-7B-Q4_K_M.gguf
4095ee9e7d91bc46e73955fc5139196c2fcae65099eef058b75a6f3b5fa25136  K2-Horizon-7B-imatrix-WikiText2-100K.gguf

Quantization quality

Q6_K and Q4_K_M were produced with a 252-tensor importance matrix collected over 200 × 512-token chunks (~100k tokens) from the WikiText-2 training split. The imatrix was collected with the Q8_0 model because the 16.76 GiB BF16 graph exceeded the local Metal working-set limit. Q8_0 is a close proxy to BF16 here (measured mean KLD 0.000467). Output and token embedding tensors intentionally use llama.cpp's normal mixed-quant fallback because imatrix collection does not process output.weight by default.

Small held-out WikiText-2 guardrail, measured against BF16 logits over two 256-token chunks with llama.cpp commit 35999d101cf2233fc54f09c3c8d599da7303ce02:

This is a deliberately small local regression test, not a general capability benchmark. Task-specific evaluation is still recommended.

Compatibility

K2 Horizon support is not yet in a released upstream llama.cpp build at the time of publication. Use the publisher's model/K2Horizon fork or a future upstream build that includes K2 Horizon support. These files were generated and validated with fork commit 35999d101cf2233fc54f09c3c8d599da7303ce02.

Example:

llama-cli \
  -m K2-Horizon-7B-Q4_K_M.gguf \
  -ngl 99 \
  -c 32768 \
  --temp 1.0 \
  --top-p 0.95 \
  -n 32768

The embedded chat template defaults to high reasoning effort, matching the publisher's recommendation. Context length is 524,288 tokens, but practical context depends on available memory and KV-cache settings.

Reproducibility

• Source repository revision: 7d69990f2d27e67a3f52aaf2e24d34c1dcfdc630
• Source file: K2-Horizon-7B-BF16.gguf
• Source SHA-256: 088c5d0814ef955d137fd1073ee3a68f6a53411fca44b75dc58a320640000444
• llama.cpp fork commit: 35999d101cf2233fc54f09c3c8d599da7303ce02
• Calibration corpus: WikiText-2 raw training split, first 200 chunks at context 512
• Evaluation corpus: WikiText-2 raw test split, first two chunks at context 256

Quantization commands:

llama-imatrix \
  -m K2-Horizon-7B-Q8_0.gguf \
  -f wiki.train.raw \
  -o K2-Horizon-7B-imatrix-WikiText2-100K.gguf \
  -ngl 99 -c 512 -b 512 -ub 512 \
  --chunks 200 --no-ppl

llama-quantize --imatrix K2-Horizon-7B-imatrix-WikiText2-100K.gguf \
  K2-Horizon-7B-BF16.gguf K2-Horizon-7B-Q6_K.gguf Q6_K

llama-quantize --imatrix K2-Horizon-7B-imatrix-WikiText2-100K.gguf \
  K2-Horizon-7B-BF16.gguf K2-Horizon-7B-Q4_K_M.gguf Q4_K_M

License and credit

The original model is released by IFM under Apache-2.0. See the upstream model card for architecture, training, usage guidance, limitations, and citation details.