Downloads · 30 days
315
18% of all-time downloads
AtomicChat/Qwen3.8-27B-MLX-AD-3.70bpw-DWQ-13.4GB
Qwen3.8-27B-MLX-AD-3.70bpw-DWQ-13.4GB is a image-text-to-text model from AtomicChat. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx.
<div style="display: flex; gap: 8px; align-items: center; margin-top: 10px; margin-bottom: 10px;" <a href="https://atomic.chat/?utmsource=huggingface&utmmedium=referral&utmcampaign=hfqwen3827bmlxad370bpwdwq134gb&utmco…
Downloads · 30 days
315
18% of all-time downloads
All-time downloads
1.8K
Public
Parameters
27.4B
13.4 GB on disk
Likes
3
Public
Click a slice to open those files.
.safetensors13.4 GB · 100%
How the weights are stored.
U3226.9B · 98%
From the Hugging Face model README
Qwen/Qwen3.8-27B for MLX on Apple Silicon, with the vision tower: 3.70 bits per weight in the language model, 13.37 GB on disk in one model.safetensors.
mixed_3_4 recipe, not one of our own layouts: 3-bit everywhere, with 4-bit for the token embeddings, the output head, and half of the MLP down projections (32 of 64) and attention value projections (8 of 16). Group size 64.mixed_3_4 conversion with a clipping search for each group's quantization range, then DWQ, which tunes the quantization scales and offsets by distillation with our 8-bit MLX build as the teacher, in a short run of one to two hours on one GPU.Against our bf16 MLX conversion, on held-out neutral text (24 chunks of 4096 tokens, the second half of each chunk scored): mean KL divergence 0.166, and it picks the same next token as the bf16 model 82.4% of the time (log: logs/kld-Qwen3.8-27B-MLX-mixed_3_4-CLIP-DWQ.json). Our MLX harness on 1x H200 NVL, mlx-lm 0.31.3 and mlx-vlm 0.6.15.
| Repository | Layout | Bits per weight | On disk | Mean KLD | Same top-1 |
|---|---|---|---|---|---|
AD-3.50bpw-DWQ-12.7GB | uniform 3-bit | 3.50 | 12.69 GB | 0.215 | 79.6% |
AD-3.70bpw-DWQ-13.4GB (this one) | mlx-lm mixed_3_4 | 3.70 | 13.37 GB | 0.166 | 82.4% |
AD-3.80bpw-DWQ-13.7GB | per-tensor, ours | 3.80 | 13.69 GB | 0.148 | 82.3% |
AD-4.50bpw-DWQ-16.0GB | uniform 4-bit | 4.50 | 16.05 GB | 0.053 | 89.5% |
AD-5.00bpw-DWQ-17.7GB | per-tensor, ours | 5.00 | 17.74 GB | 0.031 | 91.5% |
For scale, plain 3-bit and 4-bit conversions (group 64, no clipping search, no DWQ) measured 0.223 / 78.5% and 0.056 / 89.1% on the same stand.
These are MLX numbers. llama.cpp tokenizes and batches the text differently, so they cannot be compared with the KL divergence in our GGUF card.
pip install mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("AtomicChat/Qwen3.8-27B-MLX-AD-3.70bpw-DWQ-13.4GB")
prompt = "hello"
if tokenizer.chat_template is not None:
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_dict=False,
)
response = generate(model, tokenizer, prompt=prompt, verbose=True)
mlx-lm runs the text model. For image input, use mlx-vlm (pip install mlx-vlm) with the same repository id.
Quantization and evaluation by AtomicChat.