Downloads · 30 days
187
26% of all-time downloads
CptJerry/Huihui-Qwen3.8-27B-abliterated-UD-Q6_K_XL-CR-imatrix
Huihui-Qwen3.8-27B-abliterated-UD-Q6_K_XL-CR-imatrix is a image-text-to-text model from CptJerry. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
An experimental ConvRot GGUF quantisation of huihui-ai/Huihui-Qwen3.8-27B-abliterated, based on the tensor types of Unsloth's Dynamic V3.0 UD-Q6KXL with Q5K / Q6K / Q8K tensors swapped to Q5CR / Q6CR / Q8CR respectively.
Downloads · 30 days
187
26% of all-time downloads
All-time downloads
720
Public
Repo size
25.3 GB
Likes
0
Public
Click a slice to open those files.
.gguf25.3 GB · 100%
From the Hugging Face model README
An experimental ConvRot GGUF quantisation of huihui-ai/Huihui-Qwen3.8-27B-abliterated, based on the tensor types of Unsloth's Dynamic V3.0 UD-Q6_K_XL with Q5_K / Q6_K / Q8_K tensors swapped to Q5_CR / Q6_CR / Q8_CR respectively.
The quantised model was compared against the BF16 model. Two separate held-out evaluation corpora were used:
| Metric | Result |
|---|---|
| BF16 PPL | 6.008330 |
| Quant PPL | 6.011147 |
| PPL ratio | 1.000469 |
| Relative PPL increase | +0.0469% |
| Mean KLD | 0.000508 |
| Median KLD | 0.000195 |
| p95 KLD | 0.001682 |
| p99 KLD | 0.005023 |
| p99.9 KLD | 0.020512 |
| RMS Δp | 0.652% |
| Same top prediction | 98.965% |
| Metric | Result |
|---|---|
| BF16 PPL | 10.854175 |
| Quant PPL | 10.860803 |
| PPL ratio | 1.000611 |
| Relative PPL increase | +0.0611% |
| Mean KLD | 0.000725 |
| Median KLD | 0.000331 |
| p95 KLD | 0.002420 |
| p99 KLD | 0.007094 |
| p99.9 KLD | 0.023688 |
| RMS Δp | 0.710% |
| Same top prediction | 98.721% |
The model supports an optional vision module for visual inputs, but this repo does not include the corresponding mmproj GGUF.
Please refer to unsloth/Qwen3.8-27B-GGUF to download the mmproj GGUF.
The quantisation uses an independently collected imatrix mainly composed of Chinese & English. See IMATRIX.md for more information.
The quantisation method is a proof of concept which covers ConvRot tensors and token embedding and MTP layers. See QUANTIZATION.md for more information.
This model contains non-official CR tensor types. You therefore need a llama.cpp-compatible runtime that implements the corresponding:
Q5_CR
Q6_CR
Q8_CR
tensor formats and kernels, such as TheTom/llama-cpp-turboquant.