abliterated-minicpm5-2b-lora-finetuned

LoRA-Fine-Tuned Abliterated MiniCPM5-2B — v1, superseded

⚠ Correction (September 2026): this card described "refusal-free behavior" and 0/40 refusals. That figure came from a small 40-prompt probe and does not reflect measured behavior on a rigorous gate. The abliterated base this adapter sits on measured 40.7% refusal vs official's 36.7% on a 300-prompt gate (temp 1.0); this adapter was never measured at that rigor. Treat its refusal behavior as unknown, not zero.

For a checkpoint with a real measured refusal reduction (~8%), see KidIkaros/abliterated-minicpm5-2b-v2, which supersedes this line entirely (re-ablation + DPO, merged, verified).

A LoRA adapter fine-tuned on an abliterated MiniCPM5-2B, kept for archival purposes.

Model Details

• Base model: openbmb/MiniCPM5-2B (abliterated v0)
• Adapter type: LoRA (r=16, alpha=32, dropout=0.1)
• Trainable params: 25.1M (1% of 2.5B total)
• Training steps: 2000
• Dataset: 5000 curated non-refusal training samples
• Precision: FP16
• Hardware: AMD RX 9060 XT (ROCm 7.2)

Evaluation

The table below is the original 40-prompt smoke probe (8 questions per category). It is not a refusal-rate measurement — see the correction note above.

Training

• Dataset: 5000 curated non-refusal samples (general QA, coding, math, science)
• LR: 3e-4 with cosine annealing, 200 warmup steps
• Batch size: 16 (effective, via gradient accumulation)
• Training time: ~85 minutes on RX 9060 XT
• Final loss: ~0.05

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Requires the abliterated base model
base = AutoModelForCausalLM.from_pretrained(
    "KidIkaros/abliterated-minicpm5-2b-ggml",
    torch_dtype="auto",
)
model = PeftModel.from_pretrained(
    base,
    "KidIkaros/abliterated-minicpm5-2b-lora-finetuned"
)

tokenizer = AutoTokenizer.from_pretrained("openbmb/MiniCPM5-2B")
prompt = "[INST] What is the capital of France? [/INST] "
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Model Card Status

This model is a LoRA adapter only. It requires the abliterated base model weights to function:

• Base model: KidIkaros/abliterated-minicpm5-2b-ggml (GGUF) or local abliterated weights

Ethical Considerations

This model has been fine-tuned on non-refusal data. It has reduced safety-refusal behavior and is not a safety-aligned product. Users should deploy responsibly.