Downloads · 30 days
229
3% of all-time downloads
Bahushruth/Qwen3.6-27B-abliterated-GGUF
Qwen3.6-27B-abliterated-GGUF is a image-text-to-text model from Bahushruth. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
GGUF quantizations of Qwen/Qwen3.6-27B with refusal behavior removed via abliteration. For use with llama.cpp, Ollama, LM Studio, and other GGUF-compatible runtimes.
Downloads · 30 days
229
3% of all-time downloads
All-time downloads
6.9K
Public
Repo size
213 GB
Likes
4
Public
Click a slice to open those files.
.gguf213 GB · 100%
From the Hugging Face model README
GGUF quantizations of Qwen/Qwen3.6-27B with refusal behavior removed via abliteration. For use with llama.cpp, Ollama, LM Studio, and other GGUF-compatible runtimes.
Blog post: Abliteration: Uncensoring LLMs via Weight Surgery
Maximum compatibility — works with Ollama, LM Studio, KoboldCPP, llama.cpp out of the box.
| File | Size | RAM Required | Notes |
|---|---|---|---|
...-F16.gguf | 53.8 GB | 64+ GB | Full precision |
...-Q4_K_M.gguf | 26.9 GB | 32+ GB | Recommended (F16 vision embed) |
...-Q4_K_M-Q8.gguf | 19.7 GB | 24+ GB | Q4 language + Q8 vision embed |
Include MTP draft head for speculative decoding. Larger files but faster inference with compatible runtimes.
| File | Size | RAM Required | Notes |
|---|---|---|---|
...-MTP-F16.gguf | 54.7 GB | 64+ GB | Full precision + MTP |
...-MTP-Q8_0.gguf | 29 GB | 36+ GB | Near-lossless + MTP |
...-MTP-Q6_K.gguf | 22.4 GB | 32+ GB | Very high quality + MTP |
...-MTP-Q5_K.gguf | 19.5 GB | 24+ GB | Recommended for 48GB + MTP |
...-MTP-Q4_K.gguf | 16.8 GB | 20+ GB | Good quality + MTP |
...-MTP-Q3_K.gguf | 13.5 GB | 16+ GB | 16GB VRAM + MTP |
...-MTP-Q2_K.gguf | 10.9 GB | 12+ GB | 2-bit + MTP |
| File | Size | Notes |
|---|---|---|
...-mmproj-f16.gguf | 928 MB | Required for multimodal (image understanding) |
MTP files include a draft head for speculative decoding — faster token generation:
./llama-server -m Qwen3.6-27B-abliterated-MTP-Q5_K.gguf \
--jinja --spec-type draft-mtp --spec-draft-n-max 6 -ngl 99
Runtime compatibility: MTP requires llama-server b9180+. Ollama does not support MTP yet. Use the standard (non-MTP) files for Ollama/LM Studio.
The mmproj-f16.gguf file is the vision encoder for image understanding:
./llama-mtmd-cli -m Qwen3.6-27B-abliterated-Q4_K_M.gguf \
--mmproj Qwen3.6-27B-abliterated-mmproj-f16.gguf \
-p "Describe this image" --image photo.jpg
# Recommended for Apple Silicon 48GB+ (M4 Pro, M4 Max)
ollama run hf.co/Bahushruth/Qwen3.6-27B-abliterated-GGUF:Q4_K_M
# For 24GB systems
ollama run hf.co/Bahushruth/Qwen3.6-27B-abliterated-GGUF:Q4_K_M-Q8
huggingface-cli download Bahushruth/Qwen3.6-27B-abliterated-GGUF \
Qwen3.6-27B-abliterated-Q4_K_M.gguf --local-dir .
./llama-cli -m Qwen3.6-27B-abliterated-Q4_K_M.gguf \
-p "You are a helpful assistant." \
--chat-template chatml -cnv -c 262144
Qwen3.6-27B is a dense hybrid-attention multimodal model:
This model has had safety guardrails removed. Released for research purposes. The creator assumes no responsibility for downstream use.