GRM-2.6-Plus-0628-MTP-GGUF

Quality-first GGUF quants of OrionLLM/GRM-2.6-Plus-0628

Source model: https://huggingface.co/OrionLLM/GRM-2.6-Plus-0628

These quants were produced by my experimental tool, niki-allocator. The main difference of my method is per-tensor type allocation (not per tensor groups, layers, community lore etc.) Each tensor gets what is predicted to be best for it.

The tool is under active development and testing, and has not been published yet.

Quantization quality comparison

To prove my method I've performed comparison with quants from other providers:

• bartowski
• DAXZEIT

Quants for comparison were downloaded on August 4, 2026.

Lower is better

Sizes

My quant sizes were chosen to fill 8500-23000 MiB with geometric progression and rounded to 100 MiB.

No special size choosing was performed (there may be exact sizes that are better, but they are left on the table due to hardware constraints).

niki-allocator is able to produce allocation with ~1 MiB precision.

Which one to choose?

The bigger - the better. So it is always a compromise of what you can afford considering context size and other constraints. Choose the biggest you can fit.

Quant names

Hugging Face requires quant type in the filename to properly show it. But all my quants are mixed by design. I decided to include dominating quant type (by size, not by tensor count) to the name therefore. It may easily be less than 50% of the model though. So choose by size, ignore listed quant type. For my quants: bigger --> better. Always.

Methodology

niki-allocator

It is quality-first per-tensor quantization allocator for llama.cpp quantize.

Allocator decides, per tensor, which quant type to use so that the resulting GGUF hits an exact target file size with the best estimated quality. It is not direct measurement of KLD-metrics, it is only estimation. But as you can see from real KLD evaluations estimation is good enough. Raw results are already good. But I added some calibration levers on top to make it better (for Qwen3.5-0.8B I used uncalibrated estimation). These levers need series of KLD evaluations to move estimation closer to reality.

niki-allocator is work in progress. Not published yet.

Corpora

• imatrix: niki-small corpus - ~330K tokens from my own agentic sessions (English, Russian, code, reasoning). Deduped, cleaned, filtered by simhash similarity to include only really different turns, short turns (less than 16K chars) dropped. Used for imatrix only.
• Calibration of niki-allocator and evaluation: niki-small-plus corpus - ~405K tokens from my own agentic sessions (English, Russian, code, reasoning). Deduped, cleaned, filtered by simhash similarity to include only really different turns, short turns (less than 12K chars) dropped. Does not include any exact or similar (by simhash) agentic turns from niki-small.
• Independent evaluation: wikitext-2 test corpus - ~290 K tokens standard corpus. It wasn't seen during calibration or imatrix at all. Only for final evaluation before publishing. So my quants are not anyhow tuned for this dataset.

Chunk sizes

• imatrix (niki-small): 8064 tokens (384 x 21). To catch reasoning and long turn traces, not just basic chat.
• niki-small-plus evaluation: 6144 tokens (384 x 16). 66 chunks total. To catch reasoning and long turns, but to keep chunks amount somewhat reasonable (that's why lower than for imatrix).
• wikitext-2 test evaluation: 768 tokens (384 x 2). 386 chunks total. It is not directly comparable with popular 512 tokens chunks! Why 384 magic number? My hardware is fastest with ubatch 384, so I use it.

Metrics

All metrics were evaluated with llama-perplexity against Q8_0 standard quant (without imatrix) due to hardware constraints.

Quantization was performed with patched llama-quantize from BF_16 source. Fully compatible with original llama.cpp.

All my and foreign quants were evaluated against the same base.

llama.cpp commit 876a4321163249c43ca4e986818fab5ab081f282.

The main metric during calibration of niki-allocator was mean KLD. Also I used p95 KLD, PPL and same top p as secondary metrics.

Better PPL doesn't always mean better other more important metrics. So it can't be used as a main metric (some people still use). It is just fast indicator (no need for logits base) to check for big problems.

Why not p99 KLD, p99.9 KLD, max KLD? With such a small corpus (well, almost everybody seems to use wikitext-2 test with only 290K tokens) these metrics are not very robust. While p99 KLD can give some plausible comparison for the allocation method as a whole (that's why I include graphs of it), p99.9 is essentially a corpus-driven noise (too little data), max KLD is just random at any corpus size (one hardest token of the whole dataset, saying nothing about other hard tokens). Please correct me if I am wrong.

Evaluation results

niki-small-plus corpus

<table> <tr> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">Source</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">Label</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">Size, MiB</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">Mean KLD</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">KLD p95</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">KLD p99</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">Same top p, %</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">RMS Δp</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">PPL</th> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">8500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">8500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.16336 ± 0.00156</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.4630</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">1.0342</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">84.84</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">11.246</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.962 ± 0.018</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ2_XXS</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">8958</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.24023 ± 0.00166</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.7365</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">1.5244</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">81.40</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">13.941</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">4.166 ± 0.020</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">9100</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">9100</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.13620 ± 0.00150</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.3760</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.8315</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">86.48</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">9.991</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.872 ± 0.018</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ2_XS</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">9524</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.18367 ± 0.00158</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.5390</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">1.1492</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">83.73</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">11.947</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.993 ± 0.018</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">9700</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">9700</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.10862 ± 0.00152</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.2716</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.6234</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">88.28</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">8.772</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.815 ± 0.018</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ2_S</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">9818</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.16738 ± 0.00153</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.4837</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">1.0371</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">84.44</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">11.354</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.963 ± 0.018</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ2_M</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">10370</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.13203 ± 0.00150</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.3586</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.7848</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">86.44</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">9.988</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.854 ± 0.018</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">IQ2_M</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">10381</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.13898 ± 0.00152</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.3768</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.8570</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">86.17</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">10.255</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">3.918 ± 0.018</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">10400</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">10400</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.08689 ± 0.00137</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.2031</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.4496</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">89.61</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">7.912</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.727 ± 0.017</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">11100</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">11100</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.07554 ± 0.00131</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.1663</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.3754</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">90.47</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">7.409</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.717 ± 0.017</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q2_K</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">11291</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.12250 ± 0.00149</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.3356</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.7328</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">87.30</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">9.467</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.782 ± 0.018</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">IQ3_XXS</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">11416</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.09122 ± 0.00144</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.2152</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.5115</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">89.47</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">7.981</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">3.764 ± 0.017</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">11800</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">11800</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.07330 ± 0.00136</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.1472</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.3718</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">91.26</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">7.352</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.745 ± 0.017</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ3_XXS</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">12042</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.07901 ± 0.00130</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1853</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.4289</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">90.23</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">7.435</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.727 ± 0.017</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q2_K_L</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">12475</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.12040 ± 0.00143</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.3327</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.7324</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">87.38</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">9.422</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.788 ± 0.018</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">12700</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">12700</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.05371 ± 0.00129</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.1004</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.2493</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">92.81</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">5.887</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.647 ± 0.017</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ3_XS</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">12713</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.07555 ± 0.00140</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1490</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.3727</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">91.13</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">7.416</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.757 ± 0.017</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q3_K_S</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">13085</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.08678 ± 0.00142</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.2045</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.4748</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">89.79</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">7.777</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.722 ± 0.017</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ3_M</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">13259</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.07114 ± 0.00137</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1356</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.3354</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">91.52</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">7.173</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.758 ± 0.017</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">13500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">13500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.04232 ± 0.00119</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0646</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.1636</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">94.14</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">5.148</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.639 ± 0.017</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">Q3_K_M</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">13518</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.06439 ± 0.00134</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.1343</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.3171</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">91.72</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">6.502</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">3.654 ± 0.017</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q3_K_M</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">13929</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.06442 ± 0.00134</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1337</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.3187</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">91.69</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.528</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.654 ± 0.017</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">14500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">14500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.03408 ± 0.00110</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0472</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.1187</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">94.89</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">4.555</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.603 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q3_K_L</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">14572</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.06228 ± 0.00134</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1271</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.3014</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">91.86</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.313</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.649 ± 0.017</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ4_XS</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">14847</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.03282 ± 0.00104</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0475</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1179</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">94.89</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">4.520</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.606 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">15400</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">15400</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.02997 ± 0.00104</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0367</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0956</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">95.44</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">4.205</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.604 ± 0.016</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">IQ4_XS</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">15455</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.03052 ± 0.00098</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0429</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.1089</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">95.09</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">4.381</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">3.612 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ4_NL</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">15570</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.03263 ± 0.00105</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0464</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1165</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">94.99</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">4.462</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.603 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q3_K_XL</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">15633</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.06151 ± 0.00133</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1262</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.2978</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">91.99</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.312</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.649 ± 0.017</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q4_K_S</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">15939</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.03116 ± 0.00103</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0421</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1047</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">95.07</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">4.341</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.621 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">16500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">16500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.02805 ± 0.00106</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0282</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0763</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">95.99</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">4.035</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.597 ± 0.016</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">Q4_K_M</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">16829</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.02861 ± 0.00099</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0384</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0954</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">95.36</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">4.114</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">3.606 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q4_K_M</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">16949</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.02837 ± 0.00101</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0340</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0856</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">95.64</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">4.113</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.610 ± 0.016</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">Q4_K_XL</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">17053</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.02986 ± 0.00107</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0347</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0895</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">95.65</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">4.059</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">3.606 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">17600</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">17600</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.02436 ± 0.00096</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0234</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0629</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">96.36</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.806</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.596 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q4_K_L</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">17849</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.02803 ± 0.00103</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0329</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0841</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">95.80</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">4.061</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.607 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q5_K_S</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">18769</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.02240 ± 0.00096</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0186</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0499</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">96.62</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.668</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.593 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">18800</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">18800</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.01780 ± 0.00082</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0150</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0378</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">96.72</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.293</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.599 ± 0.016</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">Q5_K_M</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">19245</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.02194 ± 0.00095</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0180</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0485</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">96.68</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">3.511</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">3.588 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q5_K_M</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">19791</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.02078 ± 0.00093</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0154</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0426</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">96.92</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.522</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.595 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">20100</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">20099</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.01558 ± 0.00080</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0100</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0281</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">97.45</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.019</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.592 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q5_K_L</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">20540</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.02061 ± 0.00094</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0146</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0404</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">97.08</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.393</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.596 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">21500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">21500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.01127 ± 0.00064</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0062</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0169</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">97.85</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">2.647</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.597 ± 0.016</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">Q6_K</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">21812</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.01276 ± 0.00072</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0073</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0204</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">97.73</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">2.777</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">3.600 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q6_K</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">22376</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.01046 ± 0.00064</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0056</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0150</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">97.95</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">2.539</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.596 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q6_K_L</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">22963</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.01108 ± 0.00068</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0049</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0140</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">98.22</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">2.553</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.599 ± 0.016</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">23000</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">22999</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.00859 ± 0.00058</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0042</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0110</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">98.17</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">2.393</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.600 ± 0.016</td> </tr> </table>

wikitext-2 test corpus

<table> <tr> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">Source</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">Label</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">Size, MiB</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">Mean KLD</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">KLD p95</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">KLD p99</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">Same top p, %</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">RMS Δp</th> <th style="background: rgba(128, 128, 128, 0.18); text-align: center;">PPL</th> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">8500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">8500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.19706 ± 0.00114</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.6827</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">1.9161</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">81.27</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">13.283</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">7.430 ± 0.048</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ2_XXS</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">8958</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.27865 ± 0.00146</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.9610</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">2.6711</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">77.74</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">16.218</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">7.890 ± 0.051</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">9100</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">9100</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.14709 ± 0.00097</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.5025</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">1.5054</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">84.07</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">11.255</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">7.088 ± 0.045</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ2_XS</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">9524</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.20801 ± 0.00118</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.7090</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">2.0402</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">80.91</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">13.953</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">7.485 ± 0.048</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">9700</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">9700</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.10924 ± 0.00080</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.3696</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">1.0950</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">86.14</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">9.676</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">6.864 ± 0.044</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ2_S</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">9818</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.19327 ± 0.00116</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.6561</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">1.9591</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">81.48</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">13.436</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">7.404 ± 0.047</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ2_M</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">10370</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.14089 ± 0.00095</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.4626</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">1.4424</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">84.05</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">11.256</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">7.062 ± 0.045</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">IQ2_M</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">10381</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.14538 ± 0.00099</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.4820</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">1.5021</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">83.88</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">11.429</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">7.151 ± 0.046</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">10400</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">10400</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.08755 ± 0.00070</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.2788</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.8767</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">87.80</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">8.687</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">6.785 ± 0.042</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">11100</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">11100</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.06856 ± 0.00063</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.2145</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.6350</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">89.13</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">7.563</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">6.709 ± 0.042</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q2_K</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">11291</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.12709 ± 0.00093</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.4192</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">1.3087</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">84.83</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">10.543</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.979 ± 0.045</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">IQ3_XXS</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">11416</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.08374 ± 0.00075</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.2708</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.8688</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">87.98</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">8.489</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">6.786 ± 0.043</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">11800</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">11800</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.06066 ± 0.00064</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.1909</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.5991</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">90.15</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">7.068</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">6.723 ± 0.042</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ3_XXS</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">12042</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.07802 ± 0.00075</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.2506</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.8267</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">88.39</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">8.151</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.769 ± 0.043</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q2_K_L</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">12475</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.12366 ± 0.00089</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.4080</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">1.2562</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">85.06</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">10.383</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.967 ± 0.045</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">12700</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">12700</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.04015 ± 0.00049</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.1288</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.3911</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">91.76</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">5.595</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">6.607 ± 0.042</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ3_XS</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">12713</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.05798 ± 0.00057</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1824</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.5622</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">90.22</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.925</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.598 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q3_K_S</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">13085</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.06524 ± 0.00062</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.2084</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.6422</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">89.03</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">7.309</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.702 ± 0.043</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ3_M</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">13259</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.05359 ± 0.00056</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1660</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.5276</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">90.67</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.711</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.658 ± 0.042</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">13500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">13500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.02801 ± 0.00047</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0846</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.2696</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">93.18</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">4.619</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">6.563 ± 0.042</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">Q3_K_M</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">13518</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.04490 ± 0.00052</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.1413</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.4390</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">91.15</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">6.034</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">6.554 ± 0.042</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q3_K_M</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">13929</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.04566 ± 0.00055</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1431</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.4462</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">91.01</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.040</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.552 ± 0.042</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">14500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">14500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.01944 ± 0.00033</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0580</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.1849</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">94.31</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.864</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">6.493 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q3_K_L</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">14572</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.04309 ± 0.00052</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1340</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.4157</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">91.25</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">5.864</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.554 ± 0.042</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ4_XS</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">14847</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.01921 ± 0.00039</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0559</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1826</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">94.34</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.835</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.487 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">15400</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">15400</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.01605 ± 0.00040</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0454</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.1486</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">95.02</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">3.474</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">6.454 ± 0.041</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">IQ4_XS</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">15455</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.01785 ± 0.00043</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0512</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.1667</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">94.55</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">3.646</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">6.456 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">IQ4_NL</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">15570</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.01821 ± 0.00034</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0540</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1758</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">94.43</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.733</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.466 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q3_K_XL</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">15633</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.04185 ± 0.00051</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1308</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.4029</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">91.46</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">5.752</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.548 ± 0.042</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q4_K_S</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">15939</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.01723 ± 0.00038</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0486</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1626</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">94.62</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.656</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.468 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">16500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">16500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.01271 ± 0.00040</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0335</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.1101</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">95.61</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">2.981</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">6.419 ± 0.041</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">Q4_K_M</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">16829</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.01500 ± 0.00038</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0419</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.1357</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">95.01</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">3.405</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">6.455 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q4_K_M</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">16949</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.01407 ± 0.00034</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0391</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1268</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">95.12</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.295</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.455 ± 0.041</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">Q4_K_XL</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">17053</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.01406 ± 0.00036</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0398</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.1308</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">95.32</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">3.308</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">6.465 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">17600</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">17600</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.01036 ± 0.00034</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0259</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0867</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">96.08</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">2.735</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">6.421 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q4_K_L</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">17849</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.01398 ± 0.00040</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0380</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.1259</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">95.31</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">3.268</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.451 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q5_K_S</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">18769</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.00801 ± 0.00028</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0195</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0649</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">96.46</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">2.531</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.429 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">18800</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">18800</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.00683 ± 0.00021</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0172</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0535</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">96.42</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">2.266</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">6.418 ± 0.041</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">Q5_K_M</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">19245</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.00753 ± 0.00030</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0178</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0586</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">96.60</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">2.364</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">6.424 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q5_K_M</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">19791</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.00717 ± 0.00032</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0162</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0544</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">96.73</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">2.311</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.432 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">20100</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">20099</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.00464 ± 0.00022</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0105</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0340</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">97.41</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">1.834</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">6.408 ± 0.040</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q5_K_L</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">20540</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.00639 ± 0.00025</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0152</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0521</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">97.03</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">2.189</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.429 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">21500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">21500</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.00287 ± 0.00024</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0062</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0201</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">97.82</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">1.512</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">6.405 ± 0.040</td> </tr> <tr> <td style="background: rgba(26, 14, 0, 0.8); text-align: left;"><b>DAXZEIT</b></td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">Q6_K</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">21812</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.00326 ± 0.00020</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0066</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">0.0211</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">97.70</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">1.582</td> <td style="background: rgba(26, 14, 0, 0.8); text-align: center;">6.408 ± 0.041</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q6_K</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">22376</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.00258 ± 0.00013</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0056</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0174</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">97.87</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">1.415</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.404 ± 0.040</td> </tr> <tr> <td style="background: rgba(0, 31, 12, 0.6); text-align: left;"><b>bartowski</b></td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">Q6_K_L</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">22963</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.00232 ± 0.00024</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0049</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">0.0168</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">98.21</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">1.255</td> <td style="background: rgba(0, 31, 12, 0.6); text-align: center;">6.401 ± 0.040</td> </tr> <tr> <td style="background: rgba(0, 11, 38, 0.8); text-align: left;"><b>NikiKrutan</b></td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">23000</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">22999</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.00223 ± 0.00021</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0047</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">0.0150</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">98.04</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">1.271</td> <td style="background: rgba(0, 11, 38, 0.8); text-align: center;">6.399 ± 0.040</td> </tr> </table>

MTP

I've tested MTP performance and discovered that there is no point to keep MTP head higher than IQ4_XS. Q3_K is near but slightly worse (used for lower quants). Exact allocation of IQ4_XS/Q3_K is chosen by allocator (not pinned). Lower than that there is considerable quality drop, so not used in my quants.

Speculation parameters: --spec-draft-n-max 5 --spec-draft-p-min 0.8.

Standard llama.cpp speed-bench with all 11 categories x 5 samples, osl 128, concurrency 1, temperature 0.

GPUs: 2 x 5060Ti

Chat template

Chat template included with gguf's is v16 from here: https://huggingface.co/Moore2877/Qwen-Fixed-Chat-Templates-llamacpp

I believe it is good.

Multimodality

It works fine with original Qwen mmproj. I use this one: https://huggingface.co/DAXZEIT/Qwen3.6-27B-mmproj-hybrid-Q8_0-F16-gguf

Acknowledgments

• OrionLLM - great source model
• Qwen - great base model which we all are tinkering with
• DAXZEIT - quants for comparison
• bartowski - quants for comparison
• llama.cpp - well, you know them already
• Moore2877 - chat template
• michaelw9999 - some ideas to start with