Qwen2.5-Coder-14B-Instruct-Abliterated-NF4

Qwen2.5-Coder-14B-Instruct-Abliterated-NF4

A 4-bit (NF4) quantized, abliteration-uncensored version of Qwen2.5-Coder-14B-Instruct

This model is a pre-quantized 4-bit NormalFloat4 (NF4) version of the uncensored huihui-ai/Qwen2.5-Coder-14B-Instruct-abliterated, optimized for low VRAM and fast local inference.

Ideal for local deployment, edge devices, or low-VRAM environments while maintaining strong coding and reasoning capabilities.

---

🚀 Features

• Base Model: Qwen/Qwen2.5-Coder-14B-Instruct
• Abliteration: huihui-ai (uncensored)
• Quantization: NF4 (4-bit) – pre-quantized
• Efficient: ~8–9 GB VRAM required for inference
• Safetensors: Secure, modern format
• Framework: Compatible with transformers, vLLM, Oobabooga, etc.

---

📥 Installation & Usage

1. Install Dependencies

pip install transformers torch accelerate
---

---
### Load the Model
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "ikarius/Qwen2.5-Coder-14B-Instruct-Abliterated-NF4"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16,  # eller torch.float16
    trust_remote_code=True
)

---
---

# Eksempel
prompt = "Write a Python function to calculate Fibonacci numbers using memoization."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---
---
### Using with text-generation-webui (Oobabooga)

python server.py \
  --model ikarius/Qwen2.5-Coder-14B-Instruct-Abliterated-NF4 \
  --bf16 \
  --trust-remote-code

---

🤝 Credits

Original Model: Qwen Team
Abliteration: huihui-ai
Quantization: ikarius (NF4 via auto-gptq / transformers)
Hosting: Hugging Face Hub

---
📄 License
Apache 2.0 (same as base model)

🙌 Support
⭐ Star this repo if you find it useful!
🐛 Report issues on the Discussions tab.