Downloads · 30 days
852
52% of all-time downloads
daguoagi/Huihui-Qwen3.8-27B-abliterated-MLX-4bit
Huihui-Qwen3.8-27B-abliterated-MLX-4bit is a image-text-to-text model from daguoagi. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
This is an unofficial MLX 4-bit conversion of huihui-ai/Huihui-Qwen3.8-27B-abliterated, prepared for local inference on Apple silicon with mlx-vlm.
Downloads · 30 days
852
52% of all-time downloads
All-time downloads
1.6K
Public
Parameters
27.4B
16.1 GB on disk
Likes
3
Public
Click a slice to open those files.
.safetensors16.1 GB · 100%
How the weights are stored.
U3226.9B · 98%
From the Hugging Face model README
This is an unofficial MLX 4-bit conversion of
huihui-ai/Huihui-Qwen3.8-27B-abliterated,
prepared for local inference on Apple silicon with
mlx-vlm.
The conversion is pinned to upstream revision
739e3c5b89849f6c238ce1e5b70008612ae42cdd.
No additional fine-tuning or abliteration was performed during conversion.
Qwen3_5ForConditionalGeneration architecture in MLX Safetensors format.mlx-vlm.mtp.* checkpoint tensors.The upstream model reports that layers 18 through 51 were ablated while its visual components were left unmodified. See the upstream model card for details about the abliteration procedure and the original model.
| Item | Value |
|---|---|
| Source model | huihui-ai/Huihui-Qwen3.8-27B-abliterated |
| Source revision | 739e3c5b89849f6c238ce1e5b70008612ae42cdd |
| Upstream revision date | 2026-08-24 03:47:39 UTC |
| Conversion date | 2026-08-25 |
| Conversion tool | mlx-vlm 0.6.16 |
| MLX version | 0.32.2 |
| Source loading dtype | bfloat16 |
| Quantization | 4-bit affine RTN, group size 64 |
| Converter-reported average | 4.695 bits per weight |
| Vision quantization | Not applied; source precision retained |
| MTP | Not included |
Equivalent conversion settings:
python -m pip install "mlx-vlm==0.6.16" "mlx==0.32.2" jinja2
mlx_vlm.convert \
--hf-path huihui-ai/Huihui-Qwen3.8-27B-abliterated \
--revision 739e3c5b89849f6c238ce1e5b70008612ae42cdd \
--mlx-path ./Huihui-Qwen3.8-27B-abliterated-MLX-4bit \
--quantize \
--q-bits 4 \
--q-group-size 64 \
--q-mode affine \
--quant-method rtn
Install a recent mlx-vlm. Version 0.6.16 is the version used and tested for
this conversion.
python -m pip install -U "mlx-vlm>=0.6.16"
The examples below use the full Hugging Face repository ID. A local model directory can be used instead.
MODEL="daguoagi/Huihui-Qwen3.8-27B-abliterated-MLX-4bit"
mlx_vlm.generate \
--model "$MODEL" \
--prompt "Explain why the sky appears blue." \
--max-tokens 512 \
--temperature 1.0 \
--top-p 0.95 \
--top-k 20
MODEL="daguoagi/Huihui-Qwen3.8-27B-abliterated-MLX-4bit"
mlx_vlm.generate \
--model "$MODEL" \
--image /absolute/path/to/image.jpg \
--prompt "Describe this image in detail." \
--max-tokens 512 \
--temperature 1.0 \
--top-p 0.95 \
--top-k 20
from mlx_vlm import generate, load
from mlx_vlm.prompt_utils import apply_chat_template
model_path = "daguoagi/Huihui-Qwen3.8-27B-abliterated-MLX-4bit"
image_path = "/absolute/path/to/image.jpg"
model, processor = load(model_path)
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "Describe this image in detail."}],
}
]
prompt = apply_chat_template(
processor,
model.config,
messages,
num_images=1,
)
result = generate(
model,
processor,
prompt,
image=[image_path],
max_tokens=512,
temperature=1.0,
top_p=0.95,
top_k=20,
)
print(result.text)
The bundled generation_config.json uses temperature=1.0, top_p=0.95, and
top_k=20. These are useful starting points rather than mandatory settings.
For speculative decoding, the recommended pairing is
z-lab/Qwen3.8-27B-DFlash2,
an Apache-2.0 DFlash 2 draft checkpoint for Qwen/Qwen3.8-27B. It is a separate
dependency and is not included in this repository. The draft is not a
standalone language model; a compatible runtime uses it to propose tokens that
are verified by this target model.
The published draft checkpoint uses a block size of 8 and a sliding window of 2048. It has been locally compatibility-tested with this abliterated MLX 4-bit target. Because the target weights differ from the original Qwen3.8-27B, acceptance rates vary by prompt and workload. A compatible speculative runtime still verifies output against the target model, but the speedup is runtime- and workload-dependent.
mlx-vlm 0.6.16 can load this DFlash2 checkpoint directly:
MODEL="daguoagi/Huihui-Qwen3.8-27B-abliterated-MLX-4bit"
mlx_vlm.generate \
--model "$MODEL" \
--prompt "Explain why the sky appears blue." \
--max-tokens 512 \
--temperature 1.0 \
--top-p 0.95 \
--top-k 20 \
--draft-model z-lab/Qwen3.8-27B-DFlash2 \
--draft-kind dflash \
--draft-block-size 8
In mlx-vlm 0.6.16, the draft checkpoint is loaded at its native precision;
that version does not expose the runtime draft-quantization controls described
below for oMLX.
The following production configuration was tested with oMLX 0.6.3rc3:
| Setting | Value |
|---|---|
| Draft model | z-lab/Qwen3.8-27B-DFlash2 |
| Draft quantization | Q4 weights / A16 activations / group size 64 |
| Draft window | 2048 (checkpoint default / automatic) |
| Draft sink | 0 |
| Block size | 8 |
| Verification | Adaptive |
| In-memory draft cache | 1 entry, 2 GB limit |
| SSD draft cache | Disabled |
With this setup, text requests use the DFlash2 path while image requests retain the VLM fallback path. The draft model and its license should be reviewed and downloaded separately. Local throughput results are intentionally not presented as portable performance claims because they depend heavily on the Apple silicon device, runtime implementation, context length, and draft acceptance rate.
The converted checkpoint has been locally validated with the following checks:
mlx-vlm 0.6.16.oMLX 0.6.3rc3.These checks establish conversion and runtime compatibility. They are not a formal accuracy, safety, or multimodal benchmark.
uncensored or abliterated label does not guarantee that every
refusal has been removed, nor does it guarantee factual or safe output.Users are responsible for evaluating outputs and ensuring that their use complies with applicable laws, policies, and the upstream model license.
The upstream model is released under the Apache License 2.0. This conversion
retains that license. Please also review the
huihui-ai/Huihui-Qwen3.8-27B-abliterated
and Qwen/Qwen3.8-27B model cards.