Downloads · 30 days
49
29% of all-time downloads
zainkhanz/qwen3-4b-toxic-unsloth-bnb-4bit
qwen3-4b-toxic-unsloth-bnb-4bit is a text generation model from zainkhanz. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
4-bit (NF4) quantized version of zainkhanz/qwen3-4b-toxic-unsloth using bitsandbytes.
Downloads · 30 days
49
29% of all-time downloads
All-time downloads
168
Public
Parameters
4B
2.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors2.7 GB · 100%
How the weights are stored.
U83.6B · 90%
From the Hugging Face model README
4-bit (NF4) quantized version of zainkhanz/qwen3-4b-toxic-unsloth using bitsandbytes.
Same behavior as the 16-bit model (SFT + DPO, low-refusal style), much lower VRAM.
| Item | Detail |
|---|---|
| Parent model | zainkhanz/qwen3-4b-toxic-unsloth |
| Original base | Qwen/Qwen3-4B-Instruct-2507 |
| Quantization | bitsandbytes 4-bit NF4 + double quant |
| Parameters | ~4B |
| Training | Unsloth SFT then DPO, then quantize |
| Dataset | adamo1139/toxic-dpo-natural-v5 |
| Repo | Format |
|---|---|
| zainkhanz/qwen3-4b-toxic-unsloth | 16-bit merged |
| This repo | 4-bit NF4 |
| zainkhanz/qwen3-4b-toxic-unsloth-bnb-8bit | 8-bit |
Preference-tuned to answer requests directly and reduce generic policy-style refusals, compared with stock Qwen3-4B-Instruct.
Still subject to factual errors, bias, and hallucination. You are responsible for downstream use and must follow applicable law and the Apache 2.0 license.
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch
model_id = "zainkhanz/qwen3-4b-toxic-unsloth-bnb-4bit"
bnb = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
quantization_config=bnb,
device_map="auto",
trust_remote_code=True,
)
messages = [{"role": "user", "content": "Your question here."}]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=512,
temperature=0.7,
do_sample=True,
pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Not a guarantee of accuracy or compliance for every prompt. Do not use for illegal activity.
Apache 2.0 (same as Qwen base and parent fine-tune).
@misc{zainkhanz_qwen3_4b_toxic_unsloth_bnb_4bit,
author = {zainkhanz},
title = {Qwen3-4B Toxic Unsloth (4-bit NF4 bitsandbytes)},
year = {2026},
publisher = {Hugging Face},
howpublished = {https://huggingface.co/zainkhanz/qwen3-4b-toxic-unsloth-bnb-4bit}
}