Qwen3.8-27B-GGUF

Qwen3.8-27B-GGUF

<p> <a href="https://huggingface.co/Qwen/Qwen3.8-27B"><img src="https://img.shields.io/badge/Base_Model-Qwen3.8--27B-blue" alt="Base Model"></a> <a href="https://huggingface.co/Qwen/Qwen3.8-27B/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache_2.0-green" alt="License"></a> </p>

GGUF quantized versions of Qwen/Qwen3.8-27B for use with llama.cpp, Ollama, LM Studio, GPT4All, and other GGUF-compatible inference engines.

Available Quantizations

How to Choose

• High VRAM (32GB+): Use Q8_0 or Q6_K for best quality
• Medium VRAM (16-24GB): Use Q5_K_M or Q4_K_M for the best quality/size trade-off
• Low VRAM (8-16GB): Use Q3_K_M or Q2_K

Quickstart

llama.cpp

./llama-cli -m qwen3.8-27b-q4_k_m.gguf -p "Hello, tell me about yourself" -n 512

Ollama

Create a Modelfile:

FROM ./qwen3.8-27b-q4_k_m.gguf
TEMPLATE """{{ .Prompt }}"""
PARAMETER stop "<|im_end|>"

ollama create qwen3.8-27b -f Modelfile
ollama run qwen3.8-27b

LM Studio

Download any GGUF file and open it directly in LM Studio.

Base Model Information

• Model: Qwen3.8-27B
• Parameters: 27.78B
• Architecture: Hybrid (Gated DeltaNet + Gated Attention)
• Context Length: 262,144 tokens (extensible to 1M+)
• License: Apache 2.0
• Capabilities: Text generation, coding, reasoning, vision understanding, tool calling

Credits

• Original model by Qwen Team
• Quantization by douyamv using llama.cpp

Related Models

• douyamv/Qwen3.8-27B-FP8 — FP8 quantized safetensors
• douyamv/Qwen3.8-27B-abliterated — Abliterated (uncensored) version
• douyamv/Qwen3.8-27B-abliterated-GGUF — Abliterated GGUF quantizations