Llama-3.2-3B-Instruct-heretic-ablitered-uncensored-GGUF

Llama-3.2-3B-Instruct-heretic-ablitered-uncensored — GGUF Quantizations

GGUF quantizations of DavidAU/Llama-3.2-3B-Instruct-heretic-ablitered-uncensored, an abliterated and uncensored version of Meta's Llama 3.2 3B Instruct model. Refusal mechanisms have been removed while preserving the model's general instruction-following capabilities.

---

Available Quantizations

---

VRAM / RAM Requirements

---

Usage

llama.cpp (CLI)

./llama-cli \
  -m model_q4_k_m.gguf \
  -p "You are a helpful assistant." \
  -cnv \
  --temp 0.7 \
  -c 4096

llama.cpp (Server)

./llama-server \
  -m model_q4_k_m.gguf \
  --port 8080 \
  -c 4096

llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="model_q4_k_m.gguf",
    n_ctx=4096,
    n_threads=8,
)

response = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Tell me about yourself."}
    ],
    temperature=0.7,
    max_tokens=512,
)
print(response["choices"][0]["message"]["content"])

Ollama

ollama run hf.co/tinyopsec/Llama-3.2-3B-Instruct-heretic-ablitered-uncensored-GGUF:Q4_K_M

LM Studio

Search for tinyopsec/Llama-3.2-3B-Instruct-heretic-ablitered-uncensored-GGUF in the LM Studio model browser and download your preferred quant.

---

About the Original Model

DavidAU/Llama-3.2-3B-Instruct-heretic-ablitered-uncensored is based on Meta's Llama 3.2 3B Instruct with abliteration applied — a technique that surgically removes refusal behavior by identifying and neutralizing the model's "refusal direction" in the residual stream. The result is a model that follows instructions without built-in content restrictions, while maintaining the original's language understanding and reasoning quality.

Key characteristics:

• Architecture: LlamaForCausalLM
• Parameters: ~3 billion
• Context length: 128K tokens
• Instruction-tuned, chat-optimized
• Abliterated — no refusal behavior
• Language: English

---

Recommended Quant

Q4_K_M — best balance of quality and size for most users. Use Q5_K_M or Q8_0 if you have extra RAM/VRAM.

---

Credits

• Original model: DavidAU
• Base model: meta-llama/Llama-3.2-3B-Instruct
• Quantization: tinyopsec using llama.cpp