Downloads · 30 days
116
26% of all-time downloads
gearwave00001/Huihui-Qwen3.8-27B-Quark-AWQ-MXFP4-MtpFp8
Huihui-Qwen3.8-27B-Quark-AWQ-MXFP4-MtpFp8 is a machine learning model from gearwave00001. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for quark. The card lists the license as apache-2.0.
The repository contains huihui-ai/Huihui-Qwen3.8-27B-abliterated converted to the AMD Quark MXFP4 format using the AMD specified 28 calibration samples @ 512 sequences.
Downloads · 30 days
116
26% of all-time downloads
All-time downloads
440
Public
Parameters
15.6B
38.8 GB on disk
Likes
3
Public
Click a slice to open those files.
.safetensors19.4 GB · 100%
How the weights are stored.
U812.2B · 78%
From the Hugging Face model README
The repository contains huihui-ai/Huihui-Qwen3.8-27B-abliterated converted to the AMD Quark MXFP4 format using the AMD specified 28 calibration samples @ 512 sequences.
This model has had the drafter converted to FP8 using https://codeberg.org/ggz14/radiance-vllm-mxfp4/src/branch/main/fp8_mtp.py for the ggz14 radiance-vllm-mxfp4 image (need to compile local and run via run_mxfp4_074.sh). https://codeberg.org/ggz14/radiance-vllm-mxfp4/
The process was driven on a system with 128GB DDR4, 5900x, and 2x R9700 gpus. The last 3 layers had to be offloaded to cpu but it's not yet determined if that affected quality. PPL was slightly higher than the AMD reference model. My testing and any PPL hit the original BF16 model took from uncensoring are suspected factors. Initial testing feels ok with good results.
Reference model: https://huggingface.co/amd/Qwen3.8-27B-Quark-AWQ-MXFP4
License Apache 2.0, inherited from Qwen/Qwen3.8-27B.