Downloads · 30 days
283
20% of all-time downloads
ghost-actual/Qwen3.8-Flash-Next-Abliterated-EXL3-2.50bpw
Qwen3.8-Flash-Next-Abliterated-EXL3-2.50bpw is a text generation model from ghost-actual. Use it when you need the model to write or continue text. The card lists the license as other.
Downloads · 30 days
283
20% of all-time downloads
All-time downloads
1.5K
Public
Parameters
22.3B
64.6 GB on disk
Likes
5
Public
Click a slice to open those files.
.safetensors64.4 GB · 100%
How the weights are stored.
I1620.5B · 92%
From the Hugging Face model README

A refusal-removed (abliterated) build of Qwen's Qwen3.8-Flash-Next, quantized to EXL3 2.50 bpw so the full model runs on a single 24 GB card (RTX 3090 / 4090) using MoE CPU-offload — with the vision tower, MTP head, native 262,144-token context, and the PLE n-gram table all intact.
| layer group | bitrate | flag |
|---|---|---|
| routed experts / decoder | 2.50 bpw (per-tensor alloc, ~2.0–3.0) | -b 2.50 |
| attention + shared experts | higher ("hq") | -hq |
| MTP head | 4 bpw | -mb 4 |
| vision tower | 6 bpw | -vb 6 |
| n-gram embedding table | 3 bpw | -ngb 3 |
Total artifact: ~61 GB (41.5 GB weights in 6 shards + 19 GB n-gram table). Measured average bitrate ~2.52 bpw.
# engine (MoE CPU-offload capable):
git clone -b dflash2-pathway https://github.com/r0b0tlab/exllamav3
# convert (from the abliterated BF16):
cd exllamav3 && python3 convert.py \
-i ../Qwen3.8-Flash-Next-Uncensored \
-o ../qwen38-flash-next-abliterated-exl3-b250 \
-w ../work/target-b250 \
-b 2.50 -mb 4 -vb 6 -hq -ngb 3 -cr 250
Requires the same MoE CPU-offload setup as the stock 2.50bpw pack: ~59 GB host RAM for the CPU expert tail and a fast NVMe for the streamed n-gram table.
# interactive (no MTP — see note below):
python3 examples/chat.py -m ./qwen38-flash-next-abliterated-exl3-b250 \
-mode chatml -cs 32768 -cq 3 -mcs 320 -mct 16
Serving flags — the two that actually matter:
-mct must equal your physical core count. The upstream README's -mct 6 was "6 = their 6-core machine." On a 16-core CPU, -mct 16 jumps from ~15–24 tok/s to ~26–47. This knob alone is roughly a 2× swing.-mcs 320 = speed preset (fewer experts offloaded → more resident on GPU → faster, ~23 GB VRAM). -mcs 384 = long-context preset (~20 GB, slower). 320 fits a 24.5 GB card; go much lower and you OOM.Measured — RTX 3090 Ti + Ryzen 9 5950X (16-core) + DDR4: ~47 tok/s decode with MTP off (mcs 320, 16 threads), settling upward from ~37 cold as the offload's hot-expert set warms in VRAM. MTP only pays with greedy sampling — on the interactive temp-0.8 sampler acceptance is ~47% and decode roughly 3× slower, so leave -mtp off for chat. Full native context: -cs 262144 -cq 3 (the 3-bit KV cache is ~1.1 GB at 262K).
Fired on the 2.50 bpw output, not the BF16, to confirm the ablation survived quantization:
thinking tags intactdflash2-pathway ExLlamaV3 fork, the MoE CPU-offload approach, and the exact -b 2.50 -mb 4 -vb 6 -hq -ngb 3 conversion recipe are theirs. This repo is essentially "their pack, but abliterated."Underlying weights: Qwen License (see parent). Ablitration: OrcaRouter's work. Quant: EXL3 (ExLlamaV3, MIT).