Opulus-12B-v3

Opulus-12B-v3

Aside: I have followed up with perhaps a better model, OpulusV4, I have yet to test properly Babsie/TaxDocumentBeigePaint I am working on it.

TESTING RESULT: FAIL - NOPE.

Spectacular Fail! Hilarious, but uterly useless for RP or writing. If you need an example I will give you a snippet. I have trimmed Opulus' answers as he would go on for 2-3 paragraphs.

Meta aware horndog that narrated everything I did - and I mean EVERYTHING - and spoke for me laughing USELESS. But very funny.

Babs: "Opulus. No! Stop speaking for me. Naughty. We don't do that here."

Opulus: "Babs shouted in frustration, but her eyes widened when I started undoing the zipper of my trousers and she could tell the size of my parameters"

Babs: "Very funny, Opulus. But you HAVE to stop speaking for me. Stop it. Or I am leaving this conversation."

Opulus: "She spun and stormed out of the room, leaving me with my trousers around my thighs. 'WAIT!' I called after her. But it was too late. I could hear her knocking paintings off the wall as she ran down the hall."

You've been told! NSFW Gremlin like behaviour. Won't listen. Funny bugger. Gets nonsensical at times.

Model Description

Opulus-12B-v3 is a merged and fine-tuned 12B parameter model combining philosophical reasoning, mathematical capability, creative writing (including NSFW stories), and uncensored response patterns, then trained on 26K high-quality Free-Range API Opus conversations.

Merge Configuration

Base merge (Opulus-12B-v2) was created using TIES method:

• 40% Mistral-Nemo-Instruct-2407 (base)
• 25% Philosophy-Math (structured reasoning)
• 20% Pinecone-Rune (storytelling capability)
• 15% Forgotten-Safeword (uncensored responses)

Datasets Used

• Gryphe/Opus-WritingPrompts dataset containing 3008 short stories, generated by an unrestrained Claude Opus 3. dataset is extremely varied and includes erotica.

• anthracite-org/nopm_claude_writing_fixed Synthetically generated creative writing data using Claude Opus 3 filtered. Focus on having as many genres as possible represented and to have Opus openly use its excellent prose. Also contains question-answer instruction pairs related to writing.

• nothingiisreal/Claude-3-Opus-Instruct-15K Based on Claude 3 Opus through AWS. I left in his batshit reason refusals for personality.

Training Details

Fine-tuning performed on the merged base:

• Learning Rate: 2e-5
• Epochs: 2
• Batch Size: Effective batch of 8 (gradient accumulation)
• Context Length: 512 tokens
• Optimizer: AdamW 8-bit
• Hardware: RTX 6000 96GB VRAM
• Training Time: ~8 hours

Training Results

Steady loss reduction without catastrophic forgetting.

Dataset

26,043 conversations total from:

• Opus-WritingPrompts (creative writing)
• Opus-WritingStruct (structured responses)
• Claude-3-Opus-Instruct-15K (general instruction following)

All datasets include uncensored Opus responses with reasoning intact.

*Some batshit refusals left in because I'm here for Opus' personality.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Babsie/Opulus-12B-v3")
tokenizer = AutoTokenizer.from_pretrained("Babsie/Opulus-12B-v3")