Downloads · 30 days
15
0% of all-time downloads
kairusama/Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS-GGUF
Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS-GGUF is a machine learning model from kairusama. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
IQ3XXS quantization of ManniX-ITA/Qwen3.6-27B-Omnimerge-v4-MTP-GGUF, generated with llama.cpp using the original imatrix.dat.
Downloads · 30 days
15
0% of all-time downloads
All-time downloads
5.6K
Public
Repo size
11.4 GB
Likes
3
Public
Click a slice to open those files.
.gguf11.4 GB · 100%
From the Hugging Face model README
IQ3_XXS quantization of ManniX-ITA/Qwen3.6-27B-Omnimerge-v4-MTP-GGUF, generated with llama.cpp using the original imatrix.dat.
Source
Qwen3.6-27B-Omnimerge-v4-MTP-F16.ggufOutput
Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS.ggufQ4_K using tensor-specific quantization overrides due to reasons as documented in the original model.llama-quantize \
--imatrix imatrix.dat \
--tensor-type blk.64.attn_k.weight=q4_K \
--tensor-type blk.64.attn_q.weight=q4_K \
--tensor-type blk.64.attn_output.weight=q4_K \
--tensor-type blk.64.ffn_down.weight=q4_K \
--tensor-type blk.64.ffn_gate.weight=q4_K \
--tensor-type blk.64.ffn_up.weight=q4_K \
--tensor-type blk.64.nextn.eh_proj.weight=q4_K \
Qwen3.6-27B-Omnimerge-v4-MTP-F16.gguf \
Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS.gguf \
IQ3_XXS
llama-server \
-m Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 2
Evaluated using:
lm-evaluation-harness/v1/completionsManniX-ITA/Qwen3.6-27B-Omnimerge-v4)llama.cpp CUDA build b9632| Model | Tokenizer | HumanEval pass@1 | Size |
|---|---|---|---|
| Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS | ManniX-ITA/Qwen3.6-27B-Omnimerge-v4 | 80.49% ± 3.10% | 10.6 GB |
| ggufbench/Qwen3.6-27B-4bpw-16GB-VRAM | unsloth/Qwen3.6-27B | 81.10% ± 3.07% | 12.58 GB |
| unsloth/Qwen3.6-27B-MTP-GGUF:IQ2_M | unsloth/Qwen3.6-27B | 71.95% ± 3.52% | 10.2 GB |
The IQ3_XXS Omnimerge quant is within approximately one HumanEval problem of the larger 4bpw/16GB-VRAM quant in this setup, while remaining much closer in size to the 10GB-class IQ2_M quant.
python -m lm_eval \
--model local-completions \
--model_args "model=Qwen3.6-27B-Omnimerge-v4-MTP-IQ3_XXS,base_url=http://127.0.0.1:1234/v1/completions,tokenizer=ManniX-ITA/Qwen3.6-27B-Omnimerge-v4,tokenizer_backend=huggingface" \
--tasks humaneval \
--batch_size 1 \
--confirm_run_unsafe_code
/v1/completions rather than chat-completions.