Downloads · 30 days
23
20% of all-time downloads
sunnyyy/qwen38-27b-huihui-slim-lora
qwen38-27b-huihui-slim-lora is a machine learning model from sunnyyy. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
The delta between huihui-ai/Huihui-Qwen3.8-27B-abliterated and Qwen/Qwen3.8-27B, extracted as a rank-64 LoRA and energy-pruned from 6.1 GB → 203 MB (bf16) / 108 MB (q80 GGUF).
Downloads · 30 days
23
20% of all-time downloads
All-time downloads
116
Public
Repo size
537 MB
Likes
1
Public
Click a slice to open those files.
.gguf325 MB · 60%
From the Hugging Face model README
The delta between huihui-ai/Huihui-Qwen3.8-27B-abliterated and Qwen/Qwen3.8-27B,
extracted as a rank-64 LoRA and energy-pruned from 6.1 GB → 203 MB (bf16) / 108 MB (q8_0 GGUF).
Only the 98 modules whose weights actually changed are kept:
linear_attn.out_proj × 36, mlp.down_proj × 49, self_attn.o_proj × 13.
Zero-diff modules, the vision tower and full-rank embed/lm_head weights were pruned away.
| path | what it is | size |
|---|---|---|
lora/ | PEFT adapter (transformers / vLLM), bf16, r=64, alpha=64 | 203 MB |
gguf/huihui-slim-lora-q8_0.gguf | LoRA-GGUF for llama.cpp runtime mounting (--lora) | 108 MB |
gguf/huihui-slim-lora-bf16.gguf | lossless bf16 LoRA-GGUF (verification baseline) | 203 MB |
The base must be a Qwen3.8-27B GGUF (general.architecture = qwen35), e.g.
ggml-org/Qwen3.8-27B-GGUF or any of your own quants:
llama-server -m Qwen3.8-27B-Q8_0.gguf \
--lora-scaled gguf/huihui-slim-lora-q8_0.gguf:0.1 \
-ngl 99 -c 16384 --jinja
Scale = how strongly the abliteration delta is applied:
| scale | effect |
|---|---|
0.1 (recommended starting point) | light touch, behavior stays close to the base |
0.5 | moderate |
1.0 (plain --lora FILE) | full-strength abliteration |
Multiple adapters can be stacked: --lora-scaled a.gguf:0.5,b.gguf:1.0.
⚠️ If you get garbage output (
(begin-#: (begin, ...), first verify the base file is really Qwen3.8-27B:llama-gguf info <file> | grep -E "architecture|name".
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.8-27B", torch_dtype="auto")
model = PeftModel.from_pretrained(base, "sunnyyy/qwen38-27b-huihui-slim-lora/lora") # or local path
out_proj)Everything used to produce this adapter — patched mergekit (Qwen3.5 dense support + tmpfs streaming extraction), energy-pruning slimmer, and the LoRA→GGUF converter with the Qwen3.5 V-head reorder fix:
https://github.com/sunnyyangyangyang/qwen38-27b-abliterated-lora