Downloads · 30 days
53
2% of all-time downloads
DAXZEIT/Qwen3.6-27B-mmproj-hybrid-Q8_0-F16-gguf
Qwen3.6-27B-mmproj-hybrid-Q8_0-F16-gguf is a machine learning model from DAXZEIT. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Vision projector (mmproj) for Qwen3.6-27B in hybrid quantization, compatible with llama.cpp and text-generation-webui.
Downloads · 30 days
53
2% of all-time downloads
All-time downloads
2.8K
Public
Repo size
629 MB
Likes
4
Public
Click a slice to open those files.
.gguf629 MB · 100%
From the Hugging Face model README
Vision projector (mmproj) for Qwen3.6-27B in hybrid quantization, compatible with llama.cpp and text-generation-webui.
Converted from the official BF16 mmproj published by unsloth/Qwen3.6-27B-GGUF (mmproj-BF16.gguf), Apache 2.0.
Hybrid per-tensor quantization — same strategy as the Kimi K2.5 reference mmproj:
| Tensor type | Quantization | Count |
|---|---|---|
attn_out.weight, attn_qkv.weight, ffn_up.weight, mm.0.weight, mm.2.weight | Q8_0 | 83 |
ffn_down.weight | F16 | 27 |
| Biases, norms, embeddings, positional | F32 | 224 |
| Total | 334 |
Clip metadata and tensor shapes are identical between Qwen3.5-27B and Qwen3.6-27B mmproj files. This file works with both architectures in llama.cpp.
clip.vision.image_size = 768
clip.vision.patch_size = 16
clip.vision.embedding_length = 1152
clip.vision.feed_forward_length = 4304
clip.vision.block_count = 27
clip.vision.head_count = 16
clip.vision.projection_dim = 5120
clip.projector_type = qwen3vl_merger
llama-server \
--model Qwen3.6-27B-UD-Q4_K_XL.gguf \
--mmproj Qwen3.6-27B-mmproj-hybrid-Q8_0-F16.gguf \
--ctx-size 131072 \
--n-gpu-layers 65 \
--cache-type-k q4_0 \
--jinja \
--skip-chat-parsing
"""
Convert mmproj-BF16.gguf to hybrid Q8_0/F16 quantization.
Source: unsloth/Qwen3.6-27B-GGUF
"""
import sys, numpy as np
from pathlib import Path
sys.path.insert(0, '/path/to/llama.cpp/gguf-py')
from gguf import GGUFReader, GGUFWriter, GGMLQuantizationType
from gguf.quants import quantize
from gguf.constants import GGUFValueType
SRC = 'mmproj-BF16.gguf'
DST = 'Qwen3.6-27B-mmproj-hybrid-Q8_0-F16.gguf'
Q8_BLOCK_ELEMS, Q8_BLOCK_BYTES = 32, 34
def target_type(name):
if name.endswith('ffn_down.weight'): return GGMLQuantizationType.F16
if any(name.endswith(s) for s in (
'attn_out.weight','attn_qkv.weight','ffn_up.weight','mm.0.weight','mm.2.weight')):
return GGMLQuantizationType.Q8_0
return GGMLQuantizationType.F32
def get_f32(tensor):
if tensor.tensor_type == GGMLQuantizationType.F32: return tensor.data
raw = tensor.data.view(np.uint16)
return (raw.astype(np.uint32) << 16).view(np.float32)
def to_q8_0(data_f32):
orig_shape = data_f32.shape
flat = data_f32.flatten()
quantized = quantize(flat, GGMLQuantizationType.Q8_0)
if len(orig_shape) == 1: return quantized.reshape(1, -1)
H = int(np.prod(orig_shape[:-1]))
W_bytes = orig_shape[-1] // Q8_BLOCK_ELEMS * Q8_BLOCK_BYTES
return quantized.reshape(H, W_bytes)
reader = GGUFReader(SRC)
writer = GGUFWriter(DST, arch='clip')
# Copy scalar metadata
for key, field in reader.fields.items():
if key.startswith('GGUF.'): continue
# ... (see full script in convert_mmproj_hybrid.py)
# Arrays — must be added explicitly
writer.add_array('clip.vision.image_mean', [0.5, 0.5, 0.5])
writer.add_array('clip.vision.image_std', [0.5, 0.5, 0.5])
writer.add_array('clip.vision.is_deepstack_layers', [False] * 27)
writer.add_string('general.name', 'Qwen3.6 27B mmproj hybrid Q8_0-F16')
for tensor in reader.tensors:
ttype = target_type(tensor.name)
data_f32 = get_f32(tensor)
if ttype == GGMLQuantizationType.F32:
writer.add_tensor(tensor.name, np.ascontiguousarray(data_f32), raw_dtype=ttype)
elif ttype == GGMLQuantizationType.F16:
writer.add_tensor(tensor.name, np.ascontiguousarray(data_f32.astype(np.float16)), raw_dtype=ttype)
elif ttype == GGMLQuantizationType.Q8_0:
writer.add_tensor(tensor.name, to_q8_0(data_f32), raw_dtype=ttype)
writer.write_header_to_file()
writer.write_kv_data_to_file()
writer.write_tensors_to_file()
writer.close()
Apache 2.0 — same as the source model.