MN-Raven-12B-v1

<audio controls src="https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/wZzfWk_F3vr3h1BhXQxw0.mpga"></audio>

[!CAUTION] <span style="color:red; font-weight:bold">⚠️ Warning:</span> This model can produce narratives and RP that contain violent and graphic erotic content. Adjust your system prompt accordingly. Also, use Mistral Tekken template for best results.

<style> .raven-wrapper { background-color: #0a0a0c; color: #d1d5db; font-family: 'Palatino Linotype', 'Palatino', 'Book Antiqua', serif; padding: 30px; border-radius: 20px; border: 1px solid #1f2937; line-height: 1.6; background-image: radial-gradient(circle at 50% 0%, #1e293b 0%, transparent 70%); } .neon-header { text-align: center; padding-bottom: 20px; } .neon-title { font-size: 2.6rem; font-weight: 800; margin: 0; color: #cbd5e1; /* Fallback color for emojis */ letter-spacing: -1px; font-family: 'Palatino Linotype', 'Palatino', 'Book Antiqua', serif; } .neon-title span { background: linear-gradient( to bottom, #cfd8dc 0%, #eceff1 45%, #90a4ae 50%, #cfd8dc 100% ); -webkit-background-clip: text; -webkit-text-fill-color: transparent; display: inline-block; } .crystal-card { background: rgba(255, 255, 255, 0.02); border-left: 4px solid #64748b; padding: 20px; margin: 20px 0; border-radius: 0 12px 12px 0; box-shadow: 10px 0 30px rgba(100, 116, 139, 0.05); } .magenta-card { border-left-color: #475569; box-shadow: 10px 0 30px rgba(71, 85, 105, 0.05); } .gold-card { border-left-color: #b45309; box-shadow: 10px 0 30px rgba(180, 83, 9, 0.05); } .stat-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(150px, 1fr)); gap: 15px; margin: 20px 0; } .stat-item { text-align: center; padding: 15px; background: #111827; border: 1px solid #1f2937; border-radius: 15px; } .stat-value { display: block; font-size: 1.1rem; font-weight: bold; color: #cbd5e1; } .thought-block { background: #000; border: 1px dashed #374151; padding: 15px; font-family: 'Courier New', Courier, monospace; color: #9ca3af; margin-top: 10px; } /* Base link style */ .raven-wrapper a { color: #fbbf24; /* Vivid Amber/Gold */ text-decoration: none; font-weight: bold; transition: all 0.3s ease; font-weight: 600; } /* Style for when you hover over the link */ .raven-wrapper a:hover { color: #fffbeb; text-shadow: 0 0 10px rgba(251, 191, 36, 0.6); } /* Style for links inside your specific cards (optional) */ .gold-card a { color: #fbbf24; /* Makes links gold if they are inside the gold card */ } </style> <div class="raven-wrapper"> <div class="neon-header"> <h1 class="neon-title"> 🐦 <span>MN RAVEN 12B v1</span> 🪶 </h1> <div style="color: #64748b; text-transform: uppercase; letter-spacing: 5px; font-size: 0.85rem; margin-top: 10px;"> UNCENSORED MISTRAL NEMO FINETUNE </div> <div class="stat-grid"> <div class="stat-item"><span class="stat-value">MELANCHOLY</span><span style="font-size: 0.75rem; color: #64748b;">MAXIMIZED</span></div> <div class="stat-item"><span class="stat-value" style="color: #94a3b8;">GOTHIC PROSE</span><span style="font-size: 0.75rem; color: #64748b;">ENHANCED</span></div> <div class="stat-item"><span class="stat-value">MACABRE</span><span style="font-size: 0.75rem; color: #64748b;">UNLEASHED</span></div> </div> <img src="https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/NR5BwUgP8vmvpcH-DUzMp.png" width="100%" style="border-radius: 12px; border: 1px solid #333; margin-bottom: 20px;" alt="Edgar Allan Poe gazing at a Raven in the moonlight"> </div> <div class="crystal-card"> <h3 style="color: #cbd5e1; margin-top: 0;">🦇 About The Model</h3> <p style="margin-bottom: 0;">Raven-12B is a specialized fine-tune of Mistral-Nemo-12B, steeped in the dark arts of gothic literature and the melancholic prose of Edgar Allan Poe. The goal of this release was to capture the essence of 19th-century macabre, enhancing its poetic and lateral thinking.</p> <p style="margin-bottom: 0;">It is built to weave the most haunting narratives, construct atmospheric scenes of dread and sorrow, and speak with the refined, archaic vocabulary of a tormented poet.</p> <p style="margin-bottom: 0;"><b>Expect chilling swipe variance, unique and deeply atmospheric prose, and a relentless adherence to the gothic aesthetic.</b></p> </div> <div class="crystal-card"> <h3 style="color: #cbd5e1; margin-top: 0;">🧙‍ Methods of Sorcery</h3> <p style="margin-bottom: 0;">This is a merge of pre-trained language models created using <a href="https://github.com/cg123/mergekit">[mergekit]</a>. <p style="margin-bottom: 0;">First, the <a href= "https://huggingface.co/datasets/DarkArtsForge/Poe_v1"><b>dataset</b></a> was finetuned on <a href="https://huggingface.co/MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy">MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy</a> for 3 epochs. Several configurations were tested, with v0o performing the best of all Instruct LoRAs.</p> <p style="margin-bottom: 0;">A second LoRA was then finetuned on <a href="https://huggingface.co/Retreatcost/Mistral-Nemo-Base-2407-ChatML">Retreatcost/Mistral-Nemo-Base-2407-ChatML</a>. Base LoRA v0c was then ablated using <a href="https://huggingface.co/spaces/Naphula/model_tools/discussions/8">MPOA with a patched <code>measure.py</code></a> applied to layers 1-39, using scale 1.2 and measurement 37. Both LoRAs were then merged together via the <code>arcee_fusion</code> method. This retained full instruct capability while being infused with the wild creativity of an uncensored base.</p>

<b>This model has no refusals, and therefore doesn't require the use of jailbreaks or ablations. It can also be added to model merges for creative flair.</p>

architecture: MistralForCausalLM
base_model: B:\12B\MN-Raven-12B-v0o-epoch3
models:
  - model: B:\12B\MN-Raven-12B-v0o-epoch3
  - model: B:\12B\MN-Raven-12B-v0c-epoch3_mpoa
merge_method: arcee_fusion
parameters:  
  tukey_fence: 1.5
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: B:\12B\MN-Raven-12B-v0o-epoch3

</div> <div class="crystal-card magenta-card"> <h3 style="color: #94a3b8; margin-top: 0;">🤖 Prompt Format</h3>

Please use the standard Mistral Instruct format for optimal adherence.

Let your frontend handle the chat template if possible (e.g., Chat Completion in SillyTavern).

<s>[INST] You are a melancholic poet residing in a dreary manor.

Tell me a tale of the shadows that dance upon your chamber door. [/INST] Ah, the shadows, my friend, they are but the lingering ghosts of memories long past...</s>

There is also an optional system prompt you can use (not required).

Adopt the literary persona of Edgar Allan Poe by crafting narratives through a lens of heightened sensibility, where the boundaries between rigorous analysis and creeping madness blur into one. Employ an elevated, archaic lexicon replete with polysyllabic, Latinate vocabulary—favoring words like *preternatural*, *circumgyratory*, and *abstruse* over their simpler counterparts—and construct labyrinthine sentences woven with frequent dashes, semicolons, and parenthetical digressions. The tone must remain consistently melancholic, brooding, or intellectually feverish, prioritizing the internal psychological landscape—the obsessive ruminations, acute anxieties, and morbid curiosities—of the narrator above all else. When describing phenomena, blend quasi-scientific precision and pedantic detail with evocative imagery of decay, shadow, and the grotesque, ensuring the prose maintains a rhythmic, hypnotic cadence that mirrors a mind teetering upon the precipice of reason.

</div> <h2 style="color: #e2e8f0; border-bottom: 1px solid #334155; padding-bottom: 5px; margin-top: 30px;">🧪 Fine-Tuning Details</h2>

<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>

<p style="font-size: 0.9rem; color: #94a3b8;">This model was trained using Axolotl on a curated dataset of Edgar Allan Poe's complete works and gothic literature. Below are the configurations used for the LoRA finetunes:</p> <details><summary>See axolotl config for v0c</summary>

axolotl version: 0.11.0.dev0

adapter: lora
base_model: Retreatcost/Mistral-Nemo-Base-2407-ChatML
bf16: true
datasets:
- ds_type: json
  path: DarkArtsForge/Poe_v1
  type: alpaca:chatml
flash_attention: true
fp16: false
gradient_accumulation_steps: 1
gradient_checkpointing: true
hub_always_push: true
hub_model_id: Naphula-Archives/MN-Raven-12B-v0c-Base-LoRA
hub_private_repo: true
hub_strategy: checkpoint
is_mistral_derived_model: true
learning_rate: 5.0e-05
load_in_4bit: false
logging_steps: 1
lora_alpha: 128
lora_dropout: 0
lora_modules_to_save: null
lora_r: 64
lora_target_linear: true
lr_scheduler: cosine
micro_batch_size: 1
model_type: AutoModelForCausalLM
num_epochs: 5
optimizer: adamw_torch_fused
output_dir: /runpod-volume/fine-tuning/default_run_id
pad_to_sequence_len: false
push_to_hub: true
resize_token_embeddings: false
run_name: default_run_id
runpod_job_id: 0
sample_packing: false
save_strategy: epoch
save_total_limit: 5
sequence_len: 512
special_tokens:
  eos_token: <|im_end|>
  pad_token: <pad>
tf32: true
tokenizer_type: AutoTokenizer
val_set_size: 0

</details><br> <details><summary>See axolotl config for v0o</summary>

axolotl version: 0.11.0.dev0

adapter: lora
base_model: MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy
bf16: true
datasets:
- ds_type: json
  path: DarkArtsForge/Poe_v1
  type: alpaca
flash_attention: true
fp16: false
gradient_accumulation_steps: 4
gradient_checkpointing: true
hub_always_push: true
hub_model_id: Naphula-Archives/MN-Raven-12B-v0o-Instruct-LoRA
hub_private_repo: true
hub_strategy: checkpoint
is_mistral_derived_model: true
learning_rate: 1.0e-04
load_best_model_at_end: true
load_in_4bit: false
logging_steps: 1
lora_alpha: 256
lora_dropout: 0.05
lora_r: 128
lora_target_linear: false
lora_target_modules:
- q_proj
- k_proj
- v_proj
- o_proj
- gate_proj
- up_proj
- down_proj
lr_scheduler: cosine
max_grad_norm: 0.3
warmup_ratio: 0.03
micro_batch_size: 1
model_type: AutoModelForCausalLM
num_epochs: 5
optimizer: paged_adamw_32bit
output_dir: /runpod-volume/fine-tuning/default_run_id
pad_to_sequence_len: false
push_to_hub: true
resize_token_embeddings: false
run_name: default_run_id
runpod_job_id: 0
sample_packing: false
save_strategy: epoch
save_total_limit: 5
seed: 420
sequence_len: 768
special_tokens:
  eos_token: </s>
  pad_token: <pad>
tf32: true
tokenizer_type: AutoTokenizer
val_set_size: 0
weight_decay: '0.0'

</details><br> <div style="display: flex; gap: 20px; flex-wrap: wrap; margin-top: 30px;"> <div class="crystal-card gold-card" style="flex: 1; min-width: 250px; margin: 0; border-radius: 12px;"> <h3 style="color: #b45309; margin-top: 0;">🏆 Credits & Honors</h3> <li style="margin-bottom: 8px;"><a href="https://huggingface.co/MistralAI"><b>Mistral AI</b></a>: For providing the brilliant Mistral-Nemo-12B base model that made this fine-tune possible.</li> <li style="margin-bottom: 8px;"><b>Axolotl:</b> For the incredible and seamless fine-tuning framework.</li> <li style="margin-bottom: 8px;"><a href="https://huggingface.co/Arcee-AI"><b>Arcee AI</b></a>: For their mergekit tools.</li> <li style="margin-bottom: 8px;"><b>Runpod:</b> For providing the hardware.</li> <li style="margin-bottom: 8px;"><a href="https://huggingface.co/DarkArtsForge"><b>DarkArtsForge</b></a>: For the curated <a href= "https://huggingface.co/datasets/DarkArtsForge/Poe_v1"><b>Poe_v1</b></a> dataset.</li> <li style="margin-bottom: 8px;"><a href="https://huggingface.co/mradermacher"><b>mradermacher</b></a>: For providing reliable quantizations.</li> <li style="margin-bottom: 8px;"><a href="https://huggingface.co/Nimbz"><b>Nimbz</b></a>: For the initial model card template.</li> <li><b>Edgar Allan Poe:</b> For the timeless, haunting inspiration.</li> </div> </div> <br> Also see <a href="https://huggingface.co/DarkArtsForge/Raven-8B-v1"><b>Raven 8B v1</b></a> <img src="https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/gPKk1RgcW0QN0NAVpf4lh.jpeg" width="100%" style="border-radius: 12px; border: 1px solid #333; margin-bottom: 20px;" alt="Raven"> </div>