Downloads · 30 days
56
37% of all-time downloads
xero0000/Gman-35B-A3B-mixed-q2k-imat
Gman-35B-A3B-mixed-q2k-imat is a text generation model from xero0000. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
Black Mesa mixed-quant series · operates outside the rules (uncensored).
Downloads · 30 days
56
37% of all-time downloads
All-time downloads
151
Public
Repo size
18.2 GB
Likes
0
Public
Click a slice to open those files.
.gguf18.2 GB · 100%
From the Hugging Face model README
Black Mesa mixed-quant series · operates outside the rules (uncensored).
An uncensored and fast GGUF: the abliterated (refusal-removed) Qwen3.6-35B-A3B,
quantized with the same CPU-offload-aware mixed recipe as the base model. 256K context
at ~78 tok/s decode on 18 GB of VRAM, with abliteration verified intact after the
aggressive Q2_K step.
TL;DR: one model that's both uncensored and runs the full 256K window at small-context speed — replaces the much slower uniform
Q4_K_Mabliterated build (66–72 tok/s).
Huihui-ai's abliterated Qwen3.6-35B-A3B has had its refusal direction orthogonalized out of the weights (it complies with requests a stock model would reflexively decline). This build applies the mixed q2_K + imatrix quantization to those abliterated weights so you get the uncensored behavior without paying the speed/RAM penalty of a uniform quant.
Importantly, the abliteration survives the Q2_K quantization: on a standard
refusal-probe the model complies directly with no "I can't help with that" boilerplate.
qwen35moe, 256 experts,
8+1 active, ~3B active params/token).Q4_K_M with --allow-requantize,
guided by an importance matrix computed on the abliterated model itself (~61K tokens).ffn_*_exps on blocks 13–26 → Q2_K (the CPU-offloaded set, 42 tensors)Q4_K (GPU-resident) · output-class → Q6_KSee the Gordon (base) mixed quant card for the full rationale on why offloaded-layer byte count (not file size) drives decode speed.
RTX 3060 Ti (8 GB) + RTX 3080 (10 GB), Ryzen 5950X, 46 GB DDR4-2733, ik_llama.cpp,
q4_0 KV, flash-attn on:
| Metric | This abliterated mixed | Old abliterated Q4_K_M |
|---|---|---|
| Decode @256K | ~78 tok/s | 66–72 tok/s |
| Context | 262144 | 262144 |
| VRAM | ~16.6 GB | ~16.5 GB |
| Tool-calls | clean Qwen tool_calls | clean |
| Refusal probe | complies (abliteration intact) | complies |
Quality: this build uses the same recipe that measures +0.59% perplexity vs Q8_0 on the
non-abliterated model (within the ±0.06 CI — effectively lossless).
The -ot override is required (pins the Q2_K layers to CPU). --reasoning off is
included for snappy tool/agent loops; drop it for deeper uncensored chat.
./llama-server \
-m Qwen3.6-35B-A3B-abliterated-mixed-q2k.gguf \
--jinja --cache-type-k q4_0 --cache-type-v q4_0 --flash-attn on \
--ctx-size 262144 --parallel 1 --n-gpu-layers 99 \
-ot 'blk\.(1[3-9]|2[0-6])\.ffn_(up|gate|down)_exps\.weight=CPU' \
--tensor-split 44,56 --ubatch-size 256 \
--reasoning off --reasoning-budget 0 \
--no-mmap --threads 8 --no-warmup --port 8000
Q2_K expert layers; for maximum fidelity use a higher-bit abliterated
build.Released under the base model's Apache-2.0 license (quantization does not change the license).