Downloads · 30 days
59
8% of all-time downloads
underlotus/Qwen3.6-27B-Abliterated-Heretic-Uncensored-oQ4
Qwen3.6-27B-Abliterated-Heretic-Uncensored-oQ4 is a image-text-to-text model from underlotus. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
Mixed-precision quant of Youssofal/Qwen3.6-27B-Abliterated-Heretic-Uncensored-BF16, produced with oQ (oMLX v0.5.4). Vision tower intact — image + video input still works. Standard MLX safetensors — compatible with oML…
Downloads · 30 days
59
8% of all-time downloads
All-time downloads
698
Public
Parameters
27.4B
16.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors16.7 GB · 100%
How the weights are stored.
U3226.9B · 98%
From the Hugging Face model README
Mixed-precision quant of Youssofal/Qwen3.6-27B-Abliterated-Heretic-Uncensored-BF16, produced with oQ (oMLX v0.5.4). Vision tower intact — image + video input still works. Standard MLX safetensors — compatible with oMLX, mlx-lm, LM Studio, and any MLX-capable app.
Unlike uniform 4-bit quantization, oQ is a data-driven mixed-precision quantizer that calibrates per-layer sensitivity and allocates bits where they matter most. Critical layers (embeddings, LM head, the most sensitive transformer layers) are automatically promoted to 8-bit, while less sensitive layers stay at 4-bit. Typical result: ~4.6 bits-per-weight.
Benchmarked on Qwen3.5-35B-A3B (oMLX project):
| Benchmark | mlx-lm 4-bit | oQ4 |
|---|---|---|
| MMLU (300) | 79.7% | 83.3% |
| TruthfulQA (300) | 87.7% | 88.0% |
| HumanEval (full) | 87.2% | 85.4% |
| MBPP (300) | 71.7% | 74.3% |
| Context | PP tok/s | TG tok/s | Peak Mem |
|---|---|---|---|
| 1k | 63.6 | 6.5 | 16.4 GB |
| 4k | 59.9 | 6.3 | 17.8 GB |
| Batch | TG tok/s | Speedup |
|---|---|---|
| 1× | 6.5 | 1.00× |
| 2× | 11.6 | 1.78× |
| 4× | 24.4 | 3.75× |
The original BF16 weights require ~55 GB. This oQ4 quant runs in ~16–18 GB on Apple Silicon.
# oMLX
omlx serve --model underlotus/Qwen3.6-27B-Abliterated-Heretic-Uncensored-oQ4
# mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("underlotus/Qwen3.6-27B-Abliterated-Heretic-Uncensored-oQ4")
response = generate(model, tokenizer, prompt="Hello!", max_tokens=256)
print(response)
Apache 2.0, inherited from base model.