Eximius_Persona_5B-GGUF

QuantFactory/Eximius_Persona_5B-GGUF

This is quantized version of SicariusSicariiStuff/Eximius_Persona_5B created using llama.cpp

Original Model Card

<div align="center"> <b style="font-size: 40px;">Eximius_Persona_5B</b> </div> <img src="https://huggingface.co/SicariusSicariiStuff/Eximius_Persona_5B/resolve/main/Images/Eximius_Persona_5B.png" alt="Eximius_Persona_5B" style="width: 70%; min-width: 500px; display: block; margin: auto;">

---

<a href="https://huggingface.co/SicariusSicariiStuff/Eximius_Persona_5B#tldr" style="color: purple; font-weight: bold; font-size: 48px; text-decoration: none; display: block; text-align: center;">Click here for TL;DR</a>

---

I wanted to create a model with an exceptional capacity for using varied speech patterns and fresh role-play takes. The model had to have a unique personality, not on a surface level but on the inside, for real. Unfortunately, SFT alone just didn't cut it. And I had only 16GB of VRAM at the time. Oh, and I wanted it to be small enough to be viable for phones and to be able to give a fight to larger models while at it. If only there was a magical way to do it.

Merges. Merges are quite unique. In the early days, they were considered "fake." Clearly, there's no such thing as merges. Where are the papers? No papers? Then it's clearly impossible. "Mathematically impossible." Simply preposterous. To mix layers and hope for a coherent output? What nonsense!

And yet, they were real. <a href="https://huggingface.co/Undi95">Undi95</a> made some of the earliest merges I can remember, and the "LLAMA2 Era" was truly amazing and innovative thanks to them. Cool stuff like <a href="https://huggingface.co/KoboldAI/LLaMA2-13B-TiefighterLR">Tiefighter</a> was being made, and eventually the time tested <a href="https://huggingface.co/sophosympatheia/Midnight-Miqu-70B-v1.5">Midnight-Miqu-70B (v1.5 is my personal favorite)</a>.

Merges are an interesting thing, as they affect LLMs in a way that is currently impossible to reproduce using SFT (or any 'SOTA' technique). One of the plagues we have today, while we have orders of magnitude smarter LLMs, is GPTisms and predictability. Merges can potentially 'solve' that. How? In short, if you physically tear neurons (passthrough brain surgery) while you somehow manage to keep the model coherent enough, and if you're lucky, it can even follows instructions- then magical stuff begins to happen.

Magic, because it's not an exact science, there's some art to it, as it is done with a lot of intuition. GPTisms are patterns that the model really really "wants" to follow, it's quite hard to dissuade it. But if you yeet a couple of layers and rearrange them, boy does it get hard to spew those shivers down the spine... and instead the model starts spewing stuff that it was never intended to. It breaks its patterns and introduces some healthy chaos into the mix.

This model, Eximius_Persona_5B, is the result of multiple merges, that have been tuned, then merged again, then... for many times and iterations. The base was LLAMA 3.2 3B and I focused on achieving the following 4 traits, in that specific order:

• Varied speech patterns

• Roleplay ability

• Long context coherency

• Instruction following

For me, getting varied speech patterns was more important than instruction following, for instruction following we got API models, or LLAMA 3.3. Many models are excellent assistants, yet they all sound pretty much the same.

I also wanted to make use of my 4090m 16GB while my workstation crunches Phi-4' brain. Making a nice 5B model aligns with my goal of making AI accessible and fun for everyone, and hence Eximius_Persona_5B was born. Let this also be a call to action for more people to make AI models, you don't have to have multiple GPUs or spend a fortune on the cloud (although that definitely opens up options), you can do plenty with a mere 16GB of VRAM. And in case 16GB seems out of reach too, I should mention that Google Collab gives access to a free T4.

I uploaded a more funky, less stable, and thiccer version of Eximius_Persona to my prototyping org here:

Eximius_Persona with 84 Layers from various checkpoints

(from some early tests, occasionally it outputs stories that fool GPTZERO that it was written by a human- 60% human, 40% AI with a lucky roll)

<details> <summary><b>See example:</b></summary> <img src="https://huggingface.co/SicariusSicariiStuff/Eximius_Persona_5B/resolve/main/Images/Eximius_Persona_5B_GPTZERO.png" alt="GPTZERO Example" style="width: 100%; min-width: 600px; display: block; margin: auto;"> </details>

---

TL;DR

• Fun & Fresh Roleplay flavour.
• Interesting speech patterns in creative writing.
• Good long context coherency in Roleplay.
• Occasionally outputs quite human like stories.
• 50 Layers LLAMA 3.2, fully coherent.
• Strong performance in general for a 5B model.

Important: Make sure to use the correct settings!

Assistant settings

Roleplay settings

---

Eximius_Persona_5B is available at the following quantizations:

• Original: FP16
• GGUF: Static Quants | iMatrix_GGUF
• EXL2: 3.5 bpw | 4.0 bpw | 5.0 bpw | 6.0 bpw | 7.0 bpw | 8.0 bpw
• Specialized: FP8

---

Model Details

• Intended use: Role-Play, Creative Writing, General Tasks.

• Censorship level: <b>Medium</b>

• 5 / 10 (10 completely uncensored)

UGI score:

<img src="https://huggingface.co/SicariusSicariiStuff/Eximius_Persona_5B/resolve/main/Images/Eximius_Persona_5B_UGI.png" alt="UGI Score" style="width: 100%; min-width: 700px; display: block;">

Don't use it for coding :)

---

Model instruction template: Llama-3-Instruct

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>

{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{output}<|eot_id|>

---

<h2 style="color: darkorange; font-weight: bold; font-size: 55px; text-align: center;">Roleplay format: Classic Internet RP</h2>

*action* speech *narration*

The model is pretty smart, so it might handle other formats as well, but it was trained and tested specifically with the classic internet RP style in mind.

Recommended settings for assistant mode

<details> <summary>Full generation settings: <b>Debug Deterministic</b>.</summary> <img src="https://huggingface.co/SicariusSicariiStuff/Dusk_Rainbow/resolve/main/Presets/Debug-deterministic.png" alt="Negative_LLAMA_70B_Settings" style="width: 100%; min-width: 600px; display: block; margin: auto;"> </details> <details> <summary>Full generation settings: <b>min_p</b>.</summary> <img src="https://huggingface.co/SicariusSicariiStuff/Dusk_Rainbow/resolve/main/Presets/min_p.png" alt="Negative_LLAMA_70B_Settings" style="width: 100%; min-width: 600px; display: block; margin: auto;"> </details>

---

Recommended settings for Roleplay mode

<details> <summary><b>Roleplay settings:</b>.</summary> A good repetition_penalty range is <b>between 1.12 - 1.15</b>, feel free to experiment.

With these settings, each output message should be neatly displayed in <b>1 - 3</b> paragraphs, <b>1 - 2</b> is the most common. A single paragraph will be output as a response to a simple message ("What was your name again?").

<b>min_P</b> for RP works too but is more likely to put everything under one large paragraph, instead of a neatly formatted short one. Feel free to switch in between.

<b>(Open the image in a new window to better see the full details)</b> <img src="https://huggingface.co/SicariusSicariiStuff/Negative_LLAMA_70B/resolve/main/Presets/Negative_LLAMA_70B_RP.png" alt="Negative_LLAMA_70B_Settings" style="width: 100%; min-width: 600px; display: block; margin: auto;">

temperature:  0.8
top_p:  0.95
top_k:  25
typical_p:  1
min_p:  0
repetition_penalty: 1.12
repetition_penalty_range: 1024

</details>

---

Other recommended generation Presets:

<details> <summary><b>Midnight Enigma</b></summary>

max_new_tokens: 512
temperature: 0.98
top_p: 0.37
top_k: 100
typical_p: 1
min_p: 0
repetition_penalty: 1.18
do_sample: True

</details> <details> <summary><b>Divine Intellect</b></summary>

max_new_tokens: 512
temperature: 1.31
top_p: 0.14
top_k: 49
typical_p: 1
min_p: 0
repetition_penalty: 1.17
do_sample: True

</details> <details> <summary><b>simple-1</b></summary>

max_new_tokens: 512
temperature: 0.7
top_p: 0.9
top_k: 20
typical_p: 1
min_p: 0
repetition_penalty: 1.15
do_sample: True

</details>

---

<h2 style="color: green; font-weight: bold; font-size: 65px; text-align: center;">Your support = more models</h2> <a href="https://ko-fi.com/sicarius" style="color: pink; font-weight: bold; font-size: 48px; text-decoration: none; display: block; text-align: center;">My Ko-fi page (Click here)</a>

---

Benchmarks

---

Other stuff

• SLOP_Detector Nuke GPTisms, with SLOP detector.
• LLAMA-3_8B_Unaligned The grand project that started it all.
• Blog and updates (Archived) Some updates, some rambles, sort of a mix between a diary and a blog.