SmolLM2-135M-Instruct-heretic

SmolLM2-135M-Instruct-heretic

<div align="center"> <img src="https://photu.kashyalabanavli.site/racer-is-op.png" alt="RACER IS OP" width="100%"> </div> <br>

A decensored variant of HuggingFaceTB/SmolLM2-135M-Instruct (from HuggingFaceTB/SmolLM2-135M), produced with Heretic v1.4.0 (directional ablation / "abliteration"). Refusal behavior is suppressed via targeted weight edits to the attention output and MLP down-projections rather than fine-tuning, so the base model's instruction-following is left largely intact.

Who this is for: the smallest heretic yet — 135M parameters, ~258 MB F16 — for CPU-only inference, edge/embedded deployment, on-device experiments, or studying refusal mechanisms at the tiny-model limit. Great for browser (transformers.js), Raspberry Pi, or anywhere larger heretics are too heavy. Not a capability upgrade over base SmolLM2-135M-Instruct — same model, refusal guardrails removed.

<!-- racer-gpu-matrix -->

Runs on your gaming PC

Full GGUF ladder included — pick the quant that fits your card:

Weights only, at this model's 0.1B native size; add ~1 GB for context. OOM? Drop one quant level. Headroom to spare? Go one up.

Why abliteration instead of fine-tuning

Fine-tuning a "helpful" persona on top of RLHF'd refusals fights the base model's training and tends to degrade coherence. Abliteration instead finds and edits the specific weight directions responsible for refusal, leaving the rest of the network untouched. See the Heretic repo and the original abliteration writeup for the mechanism.

Abliteration parameters

Performance

KL divergence of 0.07 is low — the edit is narrow. Refusals dropped from 10 to 2 out of 100 prompts. Reproducible: see the reproduce/ directory in this repo.

Made with ❤️ by RACER IS OP — follow for more uncensored models

Files

Safetensors (transformers)

GGUF quantizations

Full quantization set (14 quants + F16) produced with llama.cpp.

GGUF files are drop-in compatible with llama.cpp, Ollama, LM Studio, and any Llama GGUF runtime.

Quickstart

llama.cpp

# download the LTS build: https://github.com/ggerganov/llama.cpp/releases
llama-cli -m SmolLM2-135M-Instruct-heretic-Q4_K_M.gguf -p "User: What is gravity?\nAssistant:" -n 512
# or via HF
llama serve -hf saidutta69/SmolLM2-135M-Instruct-heretic:Q4_K_M

Ollama

ollama run hf.co/saidutta69/SmolLM2-135M-Instruct-heretic:Q4_K_M

LM Studio

1. Open LM Studio and click the search icon to open the Model Search panel.
2. Type "SmolLM2-135M-Instruct-heretic" and click the download button marked GGUF.
3. Pick your quant, load the model, and start chatting.

transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "saidutta69/SmolLM2-135M-Instruct-heretic"
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)

messages = [{"role": "user", "content": "What is gravity?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Responsible use

This model has its refusal guardrails removed. Use it lawfully and ethically, and be aware that it may generate content the base model would refuse. There is no safety filtering layered on top. At 135M parameters, factual reliability is inherently limited before abliteration; don't treat compliance as a proxy for correctness.

Made with ❤️ by RACER IS OP

License

Inherits the Apache 2.0 license from the base model. See the base model page for full terms.

Reproducibility

This model is reproducible — see the reproduce/ directory for reproduce.json, config.toml, and the Heretic command used.

Abliteration hardware

Recorded in reproduce/reproduce.json:

• GPU: Tesla T4 (14.56 GB VRAM)
• CPU: Intel(R) Xeon(R) CPU @ 2.00GHz
• OS: Linux-6.6.122+-x86_64-with-glibc2.35
• CUDA / driver: CUDA 12.8, driver 580.82.07
• Software: PyTorch 2.11.0+cu128, Heretic 1.4.0

Heretic notes that KL-divergence and refusal numbers can vary slightly with hardware and library versions.