Downloads · 30 days
343
23% of all-time downloads
sahilchachra/orcarouter-Qwen3.8-27B-Uncensored-MXFP4
orcarouter-Qwen3.8-27B-Uncensored-MXFP4 is a image-text-to-text model from sahilchachra. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
MLX MXFP4 quantization of orcarouter/Qwen3.8-27B-Uncensored, an uncensored fine-tune of Qwen3.8-27B. Qwen3.8-27B is a qwen35 vision-language model with a hybrid GatedDeltaNet linear-attention + full-attention text bac…
Downloads · 30 days
343
23% of all-time downloads
All-time downloads
1.5K
Public
Parameters
27.4B
15.2 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors15.2 GB · 100%
How the weights are stored.
U3226.9B · 98%
From the Hugging Face model README
MLX MXFP4 quantization of orcarouter/Qwen3.8-27B-Uncensored,
an uncensored fine-tune of Qwen3.8-27B. Qwen3.8-27B is a qwen3_5 vision-language model with
a hybrid GatedDeltaNet linear-attention + full-attention text backbone (64
layers, full attention every 4th) and a Qwen3-VL vision tower. Runs on Apple
Silicon via mlx-vlm. Stays
image-text-to-text — the vision tower is kept in bf16; only the text backbone
is quantized.
| Precision | MXFP4 (E2M1 + E8M0 shared scale, group size 32) |
| Bits per weight | 4.449 bpw |
| On-disk size | 14 GB (13 shards) |
| Quantized | text backbone (incl. the ~1.27B lm_head) |
| Kept in bf16 | Qwen3-VL vision tower |
| Variant | Bits | Size | |
|---|---|---|---|
orcarouter-Qwen3.8-27B-Uncensored-MXFP4 | 4.449 bpw | 14 GB | ← this repo |
orcarouter-Qwen3.8-27B-Uncensored-MXFP8 | 8.381 bpw | 27 GB | higher fidelity / for 32 GB+ |
Smoke-tested on Apple Silicon via mlx-vlm with deterministic greedy decoding, inspecting raw token IDs (not just detokenized text):
Text — coherent and correct:
Vision (the bf16 vision tower, through the quantized text backbone):
| Image | Question | Answer | |
|---|---|---|---|
| solid red | main color? | Red | ✅ |
| solid blue | main color? | Blue | ✅ |
| green circle | shape and color? | "green circle" | ✅ |
Color and shape are read correctly — the vision path is live.
pip install -U mlx-vlm # needs the qwen3_5 architecture (>= 0.6.12)
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
model, processor = load("sahilchachra/orcarouter-Qwen3.8-27B-Uncensored-MXFP4")
config = model.config
prompt = apply_chat_template(processor, config, "What is the capital of France?")
print(generate(model, processor, prompt, max_tokens=256, verbose=True))
This is a reasoning model; it produces a <think> channel before its answer.
Loads and runs in LM Studio (tested on 0.4.20, mlx-llm runtime): the
qwen3_5 architecture is recognized, the model indexes cleanly (format: mlx
present), and the ChatML template runs as-is. This is a reasoning model — it
emits a thinking channel (reasoning_content) before the final content, so give
it enough max_tokens (e.g. 200+) or the answer can be empty while it is still
thinking. Verified: "capital of France" → reasoning + content = "Paris".
Loads on a 24 GB Mac (14.2 GiB weights + small context).
mtp.* weights on load. Text
and vision are unaffected; only spec-decode is not available.tie_word_embeddings=false, so the ~1.27B lm_head is a separate matrix and is
quantized. Verified correct on the MXFP4 build above.