Downloads · 30 days
718
19% of all-time downloads
KostkaIT/Qwen3.8-27B-Huihui-Abliterated-oQ4e-MTP-MLX
Qwen3.8-27B-Huihui-Abliterated-oQ4e-MTP-MLX is a image-text-to-text model from KostkaIT. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
<h1 align="center"Huihui Qwen3.8-27B Abliterated — oQ4e MLX with Native MTP</h1
Downloads · 30 days
718
19% of all-time downloads
All-time downloads
3.9K
Public
Parameters
27.8B
17 GB on disk
Likes
4
Public
Click a slice to open those files.
.safetensors17 GB · 100%
How the weights are stored.
U3227.3B · 98%
From the Hugging Face model README
In one sentence:
KostkaIT/Qwen3.8-27B-Huihui-Abliterated-oQ4e-MTP-MLXis a quality-oriented mixed-precision MLX/oMLX conversion of huihui-ai/Huihui-Qwen3.8-27B-abliterated, prepared as a lower-footprint local multimodal option with native MTP support on Apple Silicon.
This release is maintained by KostkaIT, an independent technology project by Łukasz Frąckowiak (vsnake87).
| Property | Value |
|---|---|
| Model family | Qwen3.8-27B |
| Upstream derivative | huihui-ai/Huihui-Qwen3.8-27B-abliterated |
| Release | oQ4e MLX quantization |
| Quantization | 4-bit affine base, group_size: 64 |
| Mixed-precision overrides | Selected tensors preserved at 5-bit and 6-bit |
| Format | MLX safetensors, 4 weight shards |
| Runtime target | oMLX and compatible MLX runtimes |
| Hardware target | Apple Silicon |
| Vision | Vision configuration, processor files and vision tensors included |
| Native MTP | Native MTP tensors included; one MTP hidden layer declared |
| MTP control | Enable or disable from compatible oMLX model Advanced settings |
| Primary use | Local coding, reasoning, multimodal assistants and research |
Choose this release when you want native MTP and multimodal capability in a more compact mixed-precision quantization.
If maximum quality preservation is more important than footprint, compare the oQ6e MLX variant.
The following block is intended to be copied directly into Terminal. It installs the Hugging Face Hub CLI and downloads the complete model repository, including the model shards, tokenizer, chat template, configuration, processor files and safetensors index.
# Install or update the Hugging Face Hub CLI
python3 -m pip install -U huggingface_hub
# Download the complete model repository
hf download KostkaIT/Qwen3.8-27B-Huihui-Abliterated-oQ4e-MTP-MLX \
--local-dir ./KostkaIT/Qwen3.8-27B-Huihui-Abliterated-oQ4e-MTP-MLX
After downloading, keep the directory structure intact. Do not download only individual shard files: the index, tokenizer, processor and configuration files are required for a normal load.
Tip for beginners: use the copy icon in the upper-right corner of the code block, paste the complete block into Terminal, and press Enter.
The model is saved relative to the directory in which you run the command. To place it in a dedicated models folder, replace the destination with a path such as ~/Models/KostkaIT/Qwen3.8-27B-Huihui-Abliterated-oQ4e-MTP-MLX.
Manual editing of model_settings.json should not be required in a compatible oMLX version.
Do not enable a separate VLM-MTP or external drafter as a substitute for the native MTP weights included in this checkpoint.
This repository contains actual native MTP checkpoint data under the language_model.mtp.* namespace. It is not only a configuration flag.
Native MTP is different from:
This is an image-text-to-text model package with vision assets included:
The VISION badge means that the model repository contains the relevant multimodal assets. Actual image or video support still depends on the runtime and the workflow used to load the model. Text generation and native MTP loading do not automatically prove that every vision or video path works.
| Stage | Model |
|---|---|
| Official base model | Qwen/Qwen3.8-27B |
| Abliterated derivative | huihui-ai/Huihui-Qwen3.8-27B-abliterated |
| This release | oQ4e MLX quantization with native MTP preserved |
The upstream Huihui model is an abliterated derivative of Qwen3.8-27B. According to the upstream model card, the first 15 layers were retained without abliteration, while the MTP and visual components were not modified in that release.
The source revision used for this conversion was:
d42ca8978c5a66e92c3446d46e8adfe03ef692ff
This repository contains a quantized format conversion. It is not presented as a new fine-tune or as an official continuation of Qwen or Huihui.
Thanks to huihui-ai for creating and releasing the upstream model.
This release uses oQ4e quantization created with an importance matrix (imatrix). The exact tensor layout and quantization metadata included in this repository are authoritative.
The quantization configuration is:
default: 4-bit affine, group_size=64
selected tensor overrides: 5-bit and 6-bit affine, group_size=64
The mixed-precision layout was designed to prioritize:
The target was not the smallest possible model or the highest possible raw decode speed. Quantization can change the output distribution compared with the original BF16 model, so this release should not be considered bit-identical or lossless.
The local imatrix calibration cache and generation report are intentionally not included. They are not required for inference and may contain machine-specific paths and private calibration metadata.
Qwen3.8 uses a structured chat template with support for reasoning controls. Final behavior depends on the runtime and request parameters, including:
Use the tokenizer and chat template shipped with this release. Replacing them with a generic template may change the model's behavior.
This release primarily targets:
Other MLX-compatible runtimes may load the model files, but native MTP support depends on the runtime. Always verify the MTP and vision paths separately in the runtime you intend to use.
This is an abliterated model derivative. Abliteration changes refusal behavior; it does not make the model more truthful, more reliable or inherently safe.
Compared with a safety-aligned model, this model may produce content that the original model would refuse. It can also generate factually incorrect information, unsafe or harmful instructions, biased or inappropriate content, insecure code, and confident medical, legal or financial advice.
Users and deployers are responsible for applying safeguards appropriate to their use case.
Do not use this model as the sole decision-maker for:
For public or multi-user deployment, implement authentication, authorization, input and output moderation, rate limiting, prompt and data isolation, logging, and human review for high-impact outputs.
Local inference can reduce network exposure, but it does not guarantee privacy. Operating-system services, runtimes, applications, logs, caches or monitoring tools may still retain prompts and generated outputs.
No. This is an independent MLX quantized format conversion of the Huihui model. It is not an official Qwen or Huihui release.
No separate external drafter is required for the native MTP path when the target runtime supports the included MTP checkpoint data. Use the model-level MTP control in compatible oMLX versions and verify the runtime log.
Yes. Compatible oMLX versions expose MTP as a per-model Advanced setting. Set it to ON or OFF, reload if requested, and verify the selected state in the runtime log.
No. The repository includes multimodal configuration and assets, but image and video support depends on the runtime and the specific loading path.
Choose oQ4e when a lower-footprint mixed-precision option is more convenient. Choose oQ6e when quality-oriented 6-bit quantization and higher preservation priority matter most.
The upstream Qwen3.8 and Huihui model repositories are marked as Apache-2.0. The full license text is included in LICENSE.
Relevant sources:
This repository is an independent community conversion and is not affiliated with or endorsed by Qwen or Huihui.