Downloads · 30 days
1.4K
17% of all-time downloads
lued/Qwen3.8-27B-huihui-abliterated-INT8-W8A16-MTP
Qwen3.8-27B-huihui-abliterated-INT8-W8A16-MTP is a image-text-to-text model from lued. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for vllm. The card lists the license as apache-2.0.
<div align="center" <img src="https://qianwen-res.oss-accelerate-overseas.aliyuncs.com/logoqwen3.png" width="380" alt="Qwen" <h1Qwen3.8-27B Abliterated · INT8 W8A16 · BF16 MTP</h1 <p<strongAn abliterated, Ampere-optim…
Downloads · 30 days
1.4K
17% of all-time downloads
All-time downloads
8.3K
Public
Parameters
27.8B
31.6 GB on disk
Likes
7
Trending 1
Click a slice to open those files.
.safetensors31.6 GB · 100%
How the weights are stored.
I3224.3B · 88%
From the Hugging Face model README
[!NOTE] This is a numerical W8A16 quantization of huihui-ai/Huihui-Qwen3.8-27B-abliterated, an abliterated (uncensored) version of Qwen/Qwen3.8-27B. All model credit belongs to huihui-ai (the abliteration) and Qwen (the base model); refer to the upstream model card for architecture, capabilities, and usage guidance. This repository changes numerics only.
[!TIP] This build targets vLLM on Ampere-class GPUs where native FP8 tensor-core execution is unavailable. It retains the model's vision tower, thinking controls, native context, and the byte-preserved BF16 MTP head.
This checkpoint is the abliterated counterpart of lued/Qwen3.8-27B-INT8-W8A16-MTP. The repositories share:
compressed-tensors recipe (400 packed GEMMs, same BF16 preservation set, group-128 symmetric data-free RTN);The only deviation is the base language-model weights: this checkpoint's LM weights come from huihui-ai's abliterated BF16 model instead of stock Qwen3.8-27B.
Per the upstream card, the source is an uncensored version of Qwen/Qwen3.8-27B created with remove-refusals-with-transformers — a crude, proof-of-concept refusal-removal technique that does not use TransformerLens. The first 15 layers were retained without ablation; MTP and visual components were not modified. These are the upstream card's claims, not measurements made here. The W8A16 step touches only stored numerics of the 400 GEMMs and does not change the refusal profile.
Measured on this exact checkpoint against its own BF16 teacher (the huihui source) on 2026-08-17 with tools/measure_kld.py (teacher-forced, full-vocabulary FP32 log-softmax, FP64 accumulation, nats/token; PyTorch 2.11.0+cu128, Transformers 5.10.1, compressed-tensors 0.17.1):
| Candidate | Mean KLD vs BF16 | Top-1 agreement | Weight files |
|---|---|---|---|
| This W8 / BF16 A16 | 0.000705 | 98.72% | 29.44 GiB |
The KLD result measures checkpoint-weight drift over 467 teacher-forced positions (fixed 8-prompt short suite). It is not a functional quality score; tool use, coding, multimodal quality, and long-context recall require separate behavioral evaluation.
| Prompt | Category | Tokens | Positions | Mean KLD | Top-1 agreement |
|---|---|---|---|---|---|
factual | factual | 13 | 12 | 0.000415 | 1.0000 |
code-fib | code | 87 | 86 | 0.000606 | 0.9884 |
physics-uncertainty | physics | 86 | 85 | 0.000826 | 0.9882 |
math-train | math | 58 | 57 | 0.000637 | 1.0000 |
sql-top5 | technical | 60 | 59 | 0.000894 | 0.9831 |
narrative-clock | narrative | 68 | 67 | 0.000784 | 1.0000 |
instruction-stack | instruction | 43 | 42 | 0.000589 | 0.9762 |
history-industrial | history | 60 | 59 | 0.000606 | 0.9661 |
| Token-weighted mean | 467 | 0.000705 |
kld_result.json in this repository is the single source of truth for these numbers.
| Property | Value |
|---|---|
| Quantization | Data-free symmetric RTN W8A16, group size 128 |
| Runtime format | compressed-tensors / pack-quantized |
| Quantized GEMMs | 400: 192 MLP, 64 full-attention, 144 dense GDN projections |
| Preserved precision | BF16 vision tower, lm_head, MTP (15 tensors, byte-identical to Qwen's original MTP), and recurrent GDN gates (96) |
| Structural audit | STRUCTURAL AUDIT PASS (packed set, ignore list, index contract) |
| MTP audit | MTP AUDIT PASS (15 tensors dtype-, shape-, and bit-identical to the source) |
| Shard layout | 7 main shards (5.4 GiB max) + model_mtp.safetensors (849,400,392 bytes) |
| Runtime | vLLM; this repository is not a GGUF checkpoint |
The MTP head is the exact BF16 MTP from the original Qwen3.8-27B — never touched by the abliteration and never re-quantized:
model_mtp.safetensors (849,400,392 bytes).1d8268aa85ace093a561e3e7b63b9d390dac1cd55a90cd55b5ec509c3c9da9fe — identical to the MTP bytes in the Qwen3.8-27B source, in the AEON source, and in the sibling release lued/Qwen3.8-27B-INT8-W8A16-MTP.bfloat16, shape-equal, and bit-equal to the huihui source (strict audit, not a count check).The immutable source is huihui-ai/Huihui-Qwen3.8-27B-abliterated commit d42ca8978c5a66e92c3446d46e8adfe03ef692ff. The exact selective W8A16 configuration is recorded in recipe.yaml and config.json. A conforming rebuild must take native MTP from that source, reproduce the source-derived packed projection set, and prove all 15 MTP tensors bit-identical before resharing for publication.
Built on 2026-08-17: both release audits PASS, KLD measured, published to
lued/Qwen3.8-27B-huihui-abliterated-INT8-W8A16-MTP.
Serving validated on 2026-08-18: boots through llama-swap from the lued/*
cache entry and serves a smoke request.