Downloads · 30 days
494
7% of all-time downloads
Qwen3.8-27B-heretic-ara-oQ4e-mtp
underlotus/Qwen3.8-27B-heretic-ara-oQ4e-mtp
Qwen3.8-27B-heretic-ara-oQ4e-mtp is a image-text-to-text model from underlotus. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
Mixed-precision quant of trohrbaugh/Qwen3.8-27B-heretic-ara, produced with oQ (oMLX v0.5.8.dev1). Vision tower intact — image + video input still works. MTP head preserved. Standard MLX safetensors — compatible with o…
All-time downloads
7.5K
Public
Parameters
27.8B
17 GB on disk
Likes
6
Public
Repo makeup
Click a slice to open those files.
.safetensors17 GB · 100%
Parameter types
How the weights are stored.
U3227.3B · 98%
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3.8-27B heretic-ara — oQ4e (MTP preserved)
Mixed-precision quant of trohrbaugh/Qwen3.8-27B-heretic-ara, produced with oQ (oMLX v0.5.8.dev1). Vision tower intact — image + video input still works. MTP head preserved. Standard MLX safetensors — compatible with oMLX, mlx-lm, LM Studio, and any MLX-capable app.
What is oQ?
Unlike uniform 4-bit quantization, oQ is a data-driven mixed-precision quantizer that calibrates per-layer sensitivity and allocates bits where they matter most. Critical layers (embeddings, LM head, the most sensitive transformer layers) are automatically promoted to higher precision, while less sensitive layers stay at 4-bit. Typical result: ~4.6 bits-per-weight.
Benchmarked on Qwen3.5-35B-A3B (oMLX project):
| Benchmark | mlx-lm 4-bit | oQ4 |
|---|---|---|
| MMLU (300) | 79.7% | 83.3% |
| TruthfulQA (300) | 87.7% | 88.0% |
| HumanEval (full) | 87.2% | 85.4% |
| MBPP (300) | 71.7% | 74.3% |
Performance (oMLX on M4 10-core)
| Context | PP tok/s | TG tok/s | Peak Mem |
|---|---|---|---|
| 1k | 62.4 | 12.7 | 16.7 GB |
| 4k | 60.2 | 12.1 | 18.2 GB |
| Batch | TG tok/s | Speedup |
|---|---|---|
| 1× | 12.7 | 1.00× |
| 2× | 12.0 | 0.94× |
| 4× | 19.3 | 1.52× |
Why this quant
The original BF16 weights require ~55 GB. This oQ4e quant runs in ~16–18 GB on Apple Silicon while keeping the vision tower and MTP head intact.
Quick start
# oMLX
omlx serve --model underlotus/Qwen3.8-27B-heretic-ara-oQ4e-mtp
# mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("underlotus/Qwen3.8-27B-heretic-ara-oQ4e-mtp")
response = generate(model, tokenizer, prompt="Hello!", max_tokens=256)
print(response)
Original model
- Base: Qwen/Qwen3.8-27B
- Decensored: Heretic v1.2.0 (custom fork) with Arbitrary-Rank Ablation (ARA) — 0/100 refusals, KL divergence 0.0535
- Vision: Full multimodal — image + video input supported
- MTP: Multi-token prediction head preserved
License
Apache 2.0, inherited from base model.
- Task
- Image-Text-to-Text
- Library
- mlx
- Type
- qwen3_5
- License
- apache-2.0
- Languages
- en, zh
- Created
- Aug 15, 2026
- Updated
- Aug 15, 2026