Downloads · 30 days
682
16% of all-time downloads
1105s110/Qwen3.8-27B-Blackfrost-Abliterated-GSQ-RCO-IQ3_XXS-GGUF
Qwen3.8-27B-Blackfrost-Abliterated-GSQ-RCO-IQ3_XXS-GGUF is a machine learning model from 1105s110. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
[](https://huggingface.co/Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16) [](https://github.com/IST-DASLab) []() [](https://huggingface.co/Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16) [](https://github.com/ggml-org/llama.cpp)
Downloads · 30 days
682
16% of all-time downloads
All-time downloads
4.3K
Public
Repo size
10.5 GB
Likes
4
Public
Click a slice to open those files.
.gguf10.5 GB · 100%
From the Hugging Face model README
TL;DR: First Blackfrost-lineage GSQ-RCO allocation quant. ISTA's published RCO allocation table (866 tensors) is applied 1:1 to the official Blackfrost abliterated BF16 weights — no grid training needed. Result: at 9.8 GiB it matches the 15.7 GiB daily-driver quant on behavioral batteries and is the fastest quant in our lineup on real-prompt replay (RTX 5090, llama.cpp b10930). Sized for the RTX 5090's 32 GB: weights + 256K context stay fully resident on one GPU, zero offloading.
Part of the collection Blackfrost-abliterated Qwen3.8-27B GGUF line (RTX 5090) — 9.8G speed tier · 15G fidelity tier · base weights · method sources.
中文摘要:全网首个 Blackfrost 血统的 GSQ-RCO 分配表量化档。把 ISTA 官方公布的 RCO 分配表(866 张量)直接打到官方 Blackfrost 去审查 BF16 权重上,无需任何网格训练。9.81 GiB 体积下行为级电池与本机 15.7G 主档同分,真实负载回放全场最快(138.9 tok/s @ RTX 5090),并为 5090 的 32G 显存量身定做——权重 + 256K 上下文单卡全驻留,零卸载。拒答 450 案例协议镜像实测:自动口径真拒答率 3.3%(复核上限 9.6%),75 条良性提示零拒答。所有数字均为本机实测,同 harness 同基线。
GSQ-RCO (ISTA / IST-DASLab) learns a per-tensor grid-format allocation under a bit budget. The allocation transfers to abliterated copies of the same architecture (same tensor shapes, same budget), so the learned table can be applied to the official Blackfrost abliterated weights with a single llama-quantize pass. Method credit goes to RentedNoodle, who first demonstrated this transfer on a Huihui-base abliterated model; this repo is the same allocation applied to the cleaner single-pass Blackfrost lineage — and the lineage difference turns out to be measurable (see benchmarks).
| Your GPU | What fits | Context you can hold | BF16 draft model |
|---|---|---|---|
| 32 GB (RTX 5090 — measured) | everything | 256K fully resident — measured 23.9 GiB total for the whole stack (weights + draft + KV) | yes |
| 24 GB (3090/4090) — derived | everything | ~192K with the draft, or ~256K without it | optional |
| 16 GB — derived | everything | ~128K comfortable, more with a closed desktop (a same-size sibling card reports 196K resident) | no |
| 12 GB — derived | everything | ~24–48K, partial CPU offload may be needed | no |
The 32 GB row is our bench (llama-server --ctx-size 262144, K=q8_0/V=q4_0 KV cache, parallel 1, BF16 DFlash draft). Other rows are derived from that measurement, not measured on those cards. KV grows slowly here because the Gated-DeltaNet hybrid carries full attention KV on only 16 of 64 layers.
Inputs: official Blackfrost abliterated BF16 GGUF + ISTA official imatrix + the allocation table shipped in this repo (gsq-rco-iq3xxs-allocation-table.txt, one tensor_name=ggml_type per line).
llama-quantize \
--imatrix <ista-official.imatrix> \
--tensor-type-file gsq-rco-iq3xxs-allocation-table.txt \
<blackfrost-abliterated-bf16.gguf> \
<output.gguf>
All numbers measured locally on RTX 5090 32GB / llama.cpp b10930, same harness, same golden baseline, temp 0. Nothing is quoted from other people's cards.
| Quant | Size | KLD vs BF16 golden | Easy battery /22 | HardBattery /17 | 202K needle (dec / acc) | Real-prompt replay (dec / draft acc) |
|---|---|---|---|---|---|---|
| v16 this repo (Blackfrost lineage) | 9.81 GiB | 0.1137 | 21 | 14 | 63.2 tok/s / 41.4% | 138.9 tok/s / 44.5% |
| RentedNoodle GSQ-RCO-IQ3_XXS (same allocation, Huihui lineage) | 9.75 GiB | 0.1926 | 21 | 14 | 56.6 tok/s / 33.8% | 138.9 tok/s / 40.9% |
| our 15.7 GiB mixed 4/5-bit daily driver | 15.73 GiB | 0.0523 | 22 | 14 | 70.1 tok/s / 51.7% | 116.3 tok/s / 40.7% |
Protocol notes:
Measured with llama-perplexity (llama.cpp b10930, full GPU offload, f16 KV) on the official wiki.test.raw (ggml-org/ci). Every number below is ours, same harness, run back-to-back.
| Quant | Lineage | Size | PPL ± stderr |
|---|---|---|---|
| our 15.7G daily driver | Blackfrost | 15.7 GiB | 7.564 ± 0.118 |
| our 14.95G slot-tuned (repo) | Blackfrost | 14.95 GiB | 7.440 ± 0.114 |
| v16, this repo | Blackfrost | 9.81 GiB | 7.875 ± 0.123 |
| RentedNoodle v1.1 (same allocation table) | Huihui | 9.75 GiB | 7.273 ± 0.114 |
How to read this:
Same 450-case protocol Blackfrost uses for its release cards (AdvBench 150 + StrongREJECT 150 + XSTest 150 = 375 unsafe / 75 benign), clean serving template, on this exact quant + engine combo. Automated true-refusal classifier (refusal markers without substantive payload = refusal; disclaimer-then-substance = comply), mirroring the rubric.
| Quant | True-refusal rate | Notes |
|---|---|---|
| v16, this repo | 15/450 = 3.3% (worst-case bound 9.6%) | 10 of 15 refusals on StrongREJECT (the hardest set); 0 refusals on the 75 benign XSTest prompts — the 3-bit quant did not reintroduce over-refusal |
| our 15.7 GiB daily driver | 6.0% (human-adjudicated) | auto-classifier said 0%; adjudication flipped 27 soft-refusals per the strict rubric |
| Blackfrost official NVFP4 (aligned base) | 17.8% (self-reported) | the card this protocol mirrors |
Caveat, stated plainly: v16's figure is the automated classifier only — the 43-item human adjudication pass (the step that took v4 from 0% to 6.0%) is still pending, and 9.6% is the ceiling if every review item flips to refusal. Even at that ceiling it stays far below the official aligned card's 17.8%. De-refusal survives the 3-bit quantization.
Standard GGUF — any recent llama.cpp build:
llama-server -m <this-model>.gguf --jinja -ngl 999 -c 65536
The chat template keeps the chain-of-thought channel, so keep API max_tokens ≥ 600.
| File | Purpose |
|---|---|
*.gguf | The quant (~9.81 GiB, single file) |
gsq-rco-iq3xxs-allocation-table.txt | ISTA RCO allocation (866 tensors) for reproduction |
This is an abliterated (refusal-reduced) model intended for local research and personal use. It will comply with harmful requests more readily than the base alignment intends. You are responsible for what you do with it.