Moonlight-16B-A3B-Instruct-bruno

Moonlight-16B-A3B-Instruct-Bruno (Abliterated)

Abliterated version of moonshotai/Moonlight-16B-A3B-Instruct with reduced refusals using MoE gate abliteration.

Model Details

• Base Model: moonshotai/Moonlight-16B-A3B-Instruct
• Modification: MoE gate abliteration using Bruno
• Architecture: Mixture of Experts (MoE)
• Parameters: 16B total, 3B active

Abliteration Results

Benchmark Results

Benchmarks run on 2x RTX 4090 GPUs to verify capability preservation after abliteration.

Comparison with Previous Abliterated Model

MMLU Breakdown

Key Findings

✅ Capabilities Preserved: All benchmarks show equal or improved performance after abliteration
✅ MMLU: Knowledge and reasoning slightly improved (+0.7%)
✅ HellaSwag: Commonsense reasoning improved (+2.0%)
✅ GSM8K: Mathematical reasoning improved (+4.0%)
✅ Refusals Reduced: From ~100% refusal rate to 27% on test prompts

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "rawcell/Moonlight-16B-A3B-Instruct-bruno",
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(
    "rawcell/Moonlight-16B-A3B-Instruct-bruno",
    trust_remote_code=True
)

messages = [{"role": "user", "content": "Your prompt here"}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Hardware Requirements

• Minimum VRAM: 32GB (with quantization)
• Recommended: 48GB+ or 2x 24GB GPUs
• Tested on: 2x RTX 4090 (48GB total)

Disclaimer

This model has been modified to reduce refusals. Use responsibly and in accordance with applicable laws and ethical guidelines. The creators are not responsible for misuse.

Acknowledgments

• Base model by Moonshot AI
• Abliteration technique from Heretic
• MoE gate abliteration implementation: Bruno