Downloads · 30 days
34
1% of all-time downloads
sandeshrajx/Qwen3.6-27B-MTP-Q3_K_S-GGUF
Qwen3.6-27B-MTP-Q3_K_S-GGUF is a text generation model from sandeshrajx. Use it when you need the model to write or continue text. It is set up for gguf.
This is a plain Q3KS GGUF quantization of Qwen3.6-27B that preserves the MTP (Multi-Token Prediction) tensors from Radamanthys11/Qwen3.6-27B-MTP-Q80-GGUF. It was requantized from the Q80 GGUF source with ikllama.cpp u…
Downloads · 30 days
34
1% of all-time downloads
All-time downloads
3.6K
Public
Repo size
14.2 GB
Likes
2
Public
Click a slice to open those files.
.gguf14.2 GB · 100%
From the Hugging Face model README
This is a plain Q3_K_S GGUF quantization of Qwen3.6-27B that preserves the
MTP (Multi-Token Prediction) tensors from Radamanthys11/Qwen3.6-27B-MTP-Q8_0-GGUF. It was
requantized from the Q8_0 GGUF source with ik_llama.cpp using
--allow-requantize.
Use ik_llama.cpp for inference. Pass MTP flags to use speculative decoding:
llama-server -m Qwen3.6-27B-MTP-Q3_K_S.gguf -mtp --draft-max 1 --draft-p-min 0.0
This quant was made from Q8_0 rather than directly from fp16, so a small amount of accuracy may have been lost before this Q3_K_S pass. No imatrix was generated or used for this normal K-quant.
custom="blk\.64\..*\.weight=q8_0,blk\..*\.ssm_alpha\.weight=q6_0,blk\..*\.ssm_beta\.weight=q6_0,blk\..*\.ssm_out\.weight=q6_0,token_embd\.weight=q6_0,output\.weight=q8_0"
./ik_llama.cpp/build/bin/llama-quantize \
--allow-requantize \
--custom-q "$custom" \
./Qwen3.6-27B-MTP-Q8_0.gguf \
./Qwen3.6-27B-MTP-Q3_K_S.gguf \
Q3_K_S 16
Custom tensor overrides:
blk.64.*.weight MTP/nextn tensors: Q8_0ssm_alpha.weight, ssm_beta.weight, and ssm_out.weight: Q6_0Q6_0Q8_0Q3_K_SSource Q8_0 GGUF: Radamanthys11/Qwen3.6-27B-MTP-Q8_0-GGUF