Downloads · 30 days
25
4% of all-time downloads
Qwen3.5-27B-GGUF
mkinc/Qwen3.5-27B-GGUF
Qwen3.5-27B-GGUF is a machine learning model from mkinc. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Quantization was performed using the following template:
All-time downloads
662
Public
Repo size
23.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf23.7 GB · 100%
At a glance
- Access
- Public
- Created
- Mar 28, 2026
- Updated
- Mar 28, 2026
- SHA
- dd5ebfda
Base models
Model card
From the Hugging Face model README
Overview
| Parameter | Value |
|---|---|
| Model | Qwen3.5 27B |
| Format | GGUF |
| Quantization | Mixed (q5_k / q6_k / F32 / bf16) |
| Imatrix | ❌ Not used |
Quantization Parameters
Quantization was performed using the following template:
blk\..*\.attn_gate\.weight=q5_k
blk\..*\.attn_norm\.weight=F32
blk\..*\.attn_qkv\.weight=q5_k
blk\..*\.post_attention_norm\.weight=F32
blk\..*\.attn_output\.weight=q5_k
blk\..*\.attn_q\.weight=q5_k
blk\..*\.attn_k\.weight=bf16
blk\..*\.attn_v\.weight=bf16
blk\..*\.ssm_alpha\.weight=bf16
blk\..*\.ssm_beta\.weight=bf16
blk\..*\.ssm_out\.weight=q5_k
blk\..*\.ssm_norm\.weight=F32
blk\..*\.ssm_dt.bias\.weight=F32
blk\..*\.ssm_conv1d\.weight=F32
blk\..*\.ssm_a\.weight=F32
blk\..*\.ffn_down\.weight=q5_k
blk\..*\.ffn_(gate|up)\.weight=q5_k
token_embd\.weight=F32
output.weight=q6_k
output_norm.weight=F32
Quantization Features
- No imatrix — to preserve accuracy in rare knowledge domains (avoiding distortions)
- Critical weights preserved — important small weights kept in original formats (F32/bf16)
token_embd.weight=F32— specifically increased from bf16 to F32 for CPU offloading capability ("token_embd.weight=CPU")
Usage Recommendations
llama.cpp Version - build 8507 (commit 3fc6f1aed) - Requires bf16 quantization support -ctv bf16 -ctk bf16 -ot "token_embd.weight=CPU"
- Created
- Mar 28, 2026
- Updated
- Mar 28, 2026