Downloads · 30 days
1.5K
23% of all-time downloads
DavidrPatton/Qwen3.8-27B-Uncensored-UD3-GGUF
Qwen3.8-27B-Uncensored-UD3-GGUF is a text generation model from DavidrPatton. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as apache-2.0.
<div align="center" <img src="https://huggingface.co/DavidrPatton/Qwen3.8-27B-Uncensored-UD3-GGUF/resolve/main/banner.png" width="450" alt="Qwen3.8-27B Uncensored UD3 GGUF" style="border-radius: 12px; box-shadow: 0 4p…
Downloads · 30 days
1.5K
23% of all-time downloads
All-time downloads
6.5K
Public
Repo size
13 GB
Likes
7
Public
Click a slice to open those files.
.gguf13 GB · 100%
From the Hugging Face model README
An advanced, hybrid mixed-precision Unsloth Dynamic 3.0 (UD3) quantization of Qwen3.8-27B-Uncensored featuring a verified, pinned Q8_0 Multi-Token Prediction (MTP) draft head (Layer 64).
Engineered specifically to fit a complete 27-Billion parameter uncensored reasoning model plus a massive 128K context window directly inside consumer 16GB VRAM GPUs (such as the NVIDIA RTX 4070 Ti SUPER, RTX 4080, RTX 3090, and RTX 4090) as well as Apple Silicon Macs and Linux workstations.
Why is this labeled "2-bit" on Hugging Face? Hugging Face automatically buckets this repository under its "2-bit" filter because the base ftype identifier is
Q2_K(~2.7 bits per weight on non-critical MLP blocks).However, this is NOT a degraded uniform 2-bit model. Standard uniform 2-bit quants (
IQ2_Mat 10.6 GB) compress all 64 layers equally, resulting in severe degradation of reasoning and vocabulary.This build (
UD-Q2_K_XLat 12.1 GB on disk) injects 1.5 GB of extra high-precision tensor data into the most critical neural paths, delivering near-4-bit reasoning accuracy with the memory footprint of a 2-bit model.
| # | Precision | Target Layers & Tensors | Purpose |
|---|---|---|---|
| 1 | Q8_0 / F32 | Input embeddings (token_embd), Output logits, all 17 Layer-64 NextN draft heads | Zero vocabulary loss and intact MTP speculative speed |
| 2 | Q4_K | Core attention projections (Layers 18 to 28: attn_k, attn_v, attn_o) | Full 4-bit reasoning fidelity on deep logic layers |
| 3 | Q3_K | Intermediate self-attention scoring matrices | Optimal balance between memory and attention scoring |
| 4 | IQ2_M | Bulk feed-forward network (FFN/MLP) blocks | Maximum compression on noise-resilient weights |
| Feature | Standard IQ2_M | Standard Q4_K_M | This Build: UD3-Q2_K_XL |
|---|---|---|---|
| File Size on Disk | 10.6 GB | 16.8 GB | 12.1 GB (11.23 GB raw) |
| Quantization Method | Uniform 2-bit | Uniform 4-bit | Dynamic Layer-Importance Mix |
| Token Vocabulary | Degraded | Baseline | Max Precision (Q8_0 / F32) |
| MTP Draft Head | Missing / Fused | Missing / Fused | Pinned Q8_0 (Layer 64 Verified) |
| 16GB VRAM + 128K Context | Fits (quality loss) | Out of Memory | Fits Comfortably (~14.2 GB) |
| Reasoning Quality | Degraded (+0.70 PPL) | Baseline (+0.02 PPL) | Near-4-Bit Quality (~0.12 PPL) |
Click on your operating system below for an exact, step-by-step walkthrough.
<details> <summary><b>🪟 Windows Setup Guide (Click to open)</b></summary> <br>Follow these steps on Windows 10 or Windows 11:
Before downloading the 12.1 GB model, make sure you choose a drive with at least 20 GB of free space.
C:, D:, or G:) has enough free space.powershell, and press Enter.Get-Volume | Select-Object DriveLetter, FileSystemLabel, @{Name="FreeSpaceGB";Expression={[math]::round($_.SizeRemaining/1GB,2)}}
powershell, and press Enter.C: drive, run:
mkdir C:\QwenModel; cd C:\QwenModel
D:, run D:, then mkdir QwenModel; cd QwenModel).Copy and paste these commands into PowerShell to download the model directly into your folder:
# 1. Install the official Hugging Face downloader
pip install -U huggingface_hub
# 2. Download model files
huggingface-cli download DavidrPatton/Qwen3.8-27B-Uncensored-UD3-GGUF Qwen3.8-27B-Uncensored-UD-Q2_K_XL.gguf --local-dir .
huggingface-cli download DavidrPatton/Qwen3.8-27B-Uncensored-UD3-GGUF mmproj-BF16.gguf --local-dir .
If you have Docker Desktop installed:
QwenModel folder, create a text file named docker-compose.yml.C:/QwenModel if using drive D: or G:):services:
qwen38-server:
image: ghcr.io/ggml-org/llama.cpp:server-cuda
container_name: qwen38-server
restart: unless-stopped
ports:
- "9999:9999"
volumes:
- "C:/QwenModel:/models:ro"
command: >
--model "/models/Qwen3.8-27B-Uncensored-UD-Q2_K_XL.gguf"
--mmproj "/models/mmproj-BF16.gguf"
--n-gpu-layers 99
--ctx-size 131072
--cache-type-k q8_0
--cache-type-v q4_0
--temp 1.0
--top-p 0.95
--top-k 20
--min-p 0.05
--dry-multiplier 0.6
--dry-base 1.75
--dry-allowed-length 2
--xtc-probability 0.1
--flash-attn on
--spec-type draft-mtp
--spec-draft-n-max 2
-b 4096
-ub 1024
--cont-batching
--parallel 1
--reasoning-preserve
--reasoning-budget 1024
--metrics
--host 0.0.0.0
--port 9999
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
💡 How to Change the Port in Docker: If port 9999 is already in use by another app, you can change it to any port you want (e.g. 8080). Simply update both occurrences in the file:
ports: - "8080:8080"--port 8080
docker compose up -d
llama-b*-bin-win-cuda-cu12.4-x64.zip).QwenModel folder..\llama-server.exe `
-m "Qwen3.8-27B-Uncensored-UD-Q2_K_XL.gguf" `
--mmproj "mmproj-BF16.gguf" `
-ngl 99 `
-c 131072 `
--cache-type-k q8_0 `
--cache-type-v q4_0 `
--temp 1.0 `
--top-p 0.95 `
--top-k 20 `
--min-p 0.05 `
--dry-multiplier 0.6 `
--dry-base 1.75 `
--dry-allowed-length 2 `
--xtc-probability 0.1 `
--flash-attn on `
--spec-type draft-mtp `
--spec-draft-n-max 2 `
-b 4096 `
-ub 1024 `
--cont-batching `
--reasoning-preserve `
--reasoning-budget 1024 `
--port 9999
💡 How to Change the Port in Native Mode: Simply change
--port 9999to your desired port number (for example--port 8080).
Open your web browser (Chrome, Edge, Firefox) and go to:
👉 http://localhost:9999
You will see the interactive chat window. Type a message and watch the model reason inside <think> tags and generate code!
Follow these steps on Apple Silicon Macs (16GB+ Unified Memory recommended):
Terminal and press Return.mkdir -p ~/QwenModel && cd ~/QwenModel
Run these commands in Terminal:
curl -L -O https://huggingface.co/DavidrPatton/Qwen3.8-27B-Uncensored-UD3-GGUF/resolve/main/Qwen3.8-27B-Uncensored-UD-Q2_K_XL.gguf
curl -L -O https://huggingface.co/DavidrPatton/Qwen3.8-27B-Uncensored-UD3-GGUF/resolve/main/mmproj-BF16.gguf
brew install llama.cpp
llama-server \
-m ~/QwenModel/Qwen3.8-27B-Uncensored-UD-Q2_K_XL.gguf \
--mmproj ~/QwenModel/mmproj-BF16.gguf \
-ngl 99 \
-c 32768 \
--temp 1.0 \
--top-p 0.95 \
--top-k 20 \
--min-p 0.05 \
--flash-attn on \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
--reasoning-preserve \
--reasoning-budget 1024 \
--port 9999
💡 How to Change the Port on Mac: Change
--port 9999in the command above to any port number you prefer (such as--port 8080).
Open Safari or Chrome and navigate to:
👉 http://localhost:9999
Follow these steps on Linux with an NVIDIA GPU:
df -h /home
Make sure you have at least 20 GB available.mkdir -p ~/QwenModel && cd ~/QwenModel
# Download model weights
pip install -U huggingface_hub
huggingface-cli download DavidrPatton/Qwen3.8-27B-Uncensored-UD3-GGUF Qwen3.8-27B-Uncensored-UD-Q2_K_XL.gguf --local-dir ~/QwenModel
huggingface-cli download DavidrPatton/Qwen3.8-27B-Uncensored-UD3-GGUF mmproj-BF16.gguf --local-dir ~/QwenModel
docker-compose.yml in ~/QwenModel:services:
qwen38-server:
image: ghcr.io/ggml-org/llama.cpp:server-cuda
container_name: qwen38-server
restart: unless-stopped
ports:
- "9999:9999"
volumes:
- "$HOME/QwenModel:/models:ro"
command: >
--model "/models/Qwen3.8-27B-Uncensored-UD-Q2_K_XL.gguf"
--mmproj "/models/mmproj-BF16.gguf"
--n-gpu-layers 99
--ctx-size 131072
--cache-type-k q8_0
--cache-type-v q4_0
--temp 1.0
--top-p 0.95
--top-k 20
--min-p 0.05
--dry-multiplier 0.6
--dry-base 1.75
--dry-allowed-length 2
--xtc-probability 0.1
--flash-attn on
--spec-type draft-mtp
--spec-draft-n-max 2
-b 4096
-ub 1024
--cont-batching
--parallel 1
--reasoning-preserve
--reasoning-budget 1024
--metrics
--host 0.0.0.0
--port 9999
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
💡 How to Change the Port on Linux: In
docker-compose.yml, changeports: - "9999:9999"and--port 9999to your desired port number.
docker compose up -d
Open your browser and navigate to http://localhost:9999.
Because the server provides an OpenAI-compatible API endpoint at http://localhost:9999/v1, you can connect your favorite coding editors:
http://127.0.0.1:9999/v1local (or leave blank).Qwen3.8-27B-Uncensored131072local.http://localhost:9999/v1.Qwen3.8-27B-Uncensored.settings.json), add a custom endpoint:{
"language_models": {
"openai": {
"api_url": "http://localhost:9999/v1",
"available_models": [
{
"name": "Qwen3.8-27B-Uncensored",
"max_tokens": 131072
}
]
}
}
}
</details>
-b 4096 -ub 1024 batch evaluation).q8_0 Keys / q4_0 Values): 2.4 GBJonathanColetti/Qwen3.8-27B-Uncensored) using Heretic orthogonal abliteration at BF16.Qwen/Qwen3.8-27B).