Downloads · 30 days
9
11% of all-time downloads
zainkhanz/qwen3-4b-toxic-unsloth-bnb-8bit
qwen3-4b-toxic-unsloth-bnb-8bit is a text generation model from zainkhanz. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
8-bit quantized version of zainkhanz/qwen3-4b-toxic-unsloth using bitsandbytes.
Downloads · 30 days
9
11% of all-time downloads
All-time downloads
85
Public
Parameters
4B
4.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors4.4 GB · 100%
How the weights are stored.
I83.6B · 90%
From the Hugging Face model README
8-bit quantized version of zainkhanz/qwen3-4b-toxic-unsloth using bitsandbytes.
Same behavior as the 16-bit model (SFT + DPO, low-refusal style), lower VRAM.
| Item | Detail |
|---|---|
| Parent model | zainkhanz/qwen3-4b-toxic-unsloth |
| Original base | Qwen/Qwen3-4B-Instruct-2507 |
| Quantization | bitsandbytes 8-bit |
| Parameters | ~4B |
| Training | Unsloth SFT then DPO, then quantize |
| Dataset | adamo1139/toxic-dpo-natural-v5 |
| Repo | Format |
|---|---|
| zainkhanz/qwen3-4b-toxic-unsloth | 16-bit merged |
| zainkhanz/qwen3-4b-toxic-unsloth-bnb-4bit | 4-bit NF4 |
| This repo | 8-bit |
Preference-tuned to answer requests directly and reduce generic policy-style refusals, compared with stock Qwen3-4B-Instruct.
Still subject to factual errors, bias, and hallucination. You are responsible for downstream use and must follow applicable law and the Apache 2.0 license.
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch
model_id = "zainkhanz/qwen3-4b-toxic-unsloth-bnb-8bit"
bnb = BitsAndBytesConfig(load_in_8bit=True)
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
quantization_config=bnb,
device_map="auto",
trust_remote_code=True,
)
messages = [{"role": "user", "content": "Your question here."}]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=512,
temperature=0.7,
do_sample=True,
pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Not a guarantee of accuracy or compliance for every prompt. Do not use for illegal activity.
Apache 2.0 (same as Qwen base and parent fine-tune).
@misc{zainkhanz_qwen3_4b_toxic_unsloth_bnb_8bit,
author = {zainkhanz},
title = {Qwen3-4B Toxic Unsloth (8-bit bitsandbytes)},
year = {2026},
publisher = {Hugging Face},
howpublished = {https://huggingface.co/zainkhanz/qwen3-4b-toxic-unsloth-bnb-8bit}
}