MiS-Firefly-v0.1-22B

<div align="center"> <b style="font-size: 24px; color: #dddd33; margin-bottom:20px;">NOTE: Using this model is not recommended unless you can run fp16 or Q8_0, there was a bug in quantization that caused serious spelling issues particularly in names. Use <a href="https://huggingface.co/invisietch/MiS-Firefly-v0.2-22B">MiS-Firefly-v0.2-22B</a> instead.</b> </div> <div align="center"> <b style="font-size: 36px;">MiS-Firefly-v0.1-22B (Q4_K_M GGUF)</b> <img src="https://huggingface.co/invisietch/MiS-Firefly-v0.1-22B/resolve/main/header.png" style="width:60%">

<b>HF</b> : <a href="https://huggingface.co/invisietch/MiS-Firefly-v0.1-22B">FP16</a> | <b>EXL2</b> : <a href="https://huggingface.co/SytanSD/MiS_Firefly_v0.1_22B_6.5BPW_EXL2">6.5BPW</a> | <b>GGUF</b> : <a href="https://huggingface.co/mradermacher/MiS-Firefly-v0.1-22B-GGUF">static</a> · <a href="https://huggingface.co/mradermacher/MiS-Firefly-v0.1-22B-i1-GGUF">imatrix</a>

</div>

Model Details

Firefly is a Mistral Small 22B finetune designed for creative writing and roleplay. The model is largely uncensored and should support context up to 32,768 tokens.

The model has been tested in various roleplay scenarios up to 16k context, as well as in a role as an assistant. It shows a broad competency & coherence across various scenarios.

Special thanks to <a href="https://huggingface.co/SicariusSicariiStuff">SicariusSicariiStuff</a> for bouncing ideas back & forth on training, and <a href="https://huggingface.co/SytanSD">SytanSD</a> for quants.

KNOWN QUANTIZATION ISSUE

Some quants seem to have issues with misspelling complicated names.

This doesn't happen at fp16 or q8_0 even with very weird names and multiple swipes meaning something's getting lost in quant.

Suggested workarounds:

• If you can, run q8_0 (I'm told this fits on a 4090 with flash attention), I haven't seen the issue in ~900 messages on q8.
• If not, try some lower quants (ideally imatrix), I haven't tested them all but it appears to happen on Q6_K the most and less often on the 6.5bpw EXL2. If you find one where this doesn't happen, tell me.
• If none of that works, use a simpler name.

I'll try resolving it with a light merge ASAP, it seems like the wrong weight is just getting truncated in quantization causing these issues.

Feedback

I appreciate all feedback on any of my models, you can use:

• My Discord server - requires Discord.
• The Community tab - requires HF login.
• Discord DMs to invisietch.

Your feedback is how I improve these models for future versions.

Disclaimer

This model is extensively uncensored. It can generate explicit, disturbing or offensive responses. Use responsibly. I am not responsible for your use of this model.

This model is a finetune of Mistral Small 22B (2409) and usage must follow the terms of Mistral's license. By downloading this model, you agree not to use it for commercial purposes unless you have a valid Mistral commercial license. See the base model card for more details.

Prompting Format

I'd recommend Mistral v2v3 prompting format:

<s>[INST] User message here.[/INST] Bot response here</s>[INST] User message 2 here.

Sampler Settings

I'm running the following sampler settings but this is an RC and they may not be optimal.

• Temperature: Dynamic 0.7-1.1
• Min-P: 0.07
• Rep Pen: 1.08
• Rep Pen Range: 1536
• XTC: 0.1/0.15

If you get completely incoherent responses, feel free to use these as a starting point.

Example Storywriting

These prompts are used on SillyTavern with a fairly basic narrator card. I have trimmed the start and finish where the narrator decided to add chapter headings, commentary and the like. All samples are made with the Q8_0 GGUF loaded with koboldcpp, with response length capped at 2048 tokens.

<details> <summary><b>Write me a 3,000 word opening chapter of a 'gritty hard sci-fi' novel, drawing inspiration from the writing styles of Isaac Asimov & Andy Weir. Use third person personal. Include dialogue and internal monologues. The POV character for the opening chapter should be a 26 year old astronaut called Tone on a mission to Europa, who has just realised that the craft for the return journey is broken beyond repair, and he only has supplies for a few months. Given that survival is impossible, he seeks to spend the few months he has researching titan, so his life & mission are not wasted.</b></summary>

In the inky expanse of space, beyond the pale blue dot that was once home, Europa's ice-covered surface shimmered beneath the distant sun. Astronaut Tone gazed at it through the viewport of the Orion III, a sleek vessel designed to explore Jupiter's largest moon. A wave of excitement washed over him as he contemplated their groundbreaking discovery.

Tone leaned back in his chair, feeling the familiar hum of machinery around him. He had been trained since childhood for this moment - to step into the unknown and bring back knowledge that would change humanity forever. But now, with the weight of responsibility pressing down upon him like an invisible force, all he could do was stare at Europa in silent awe.

The ship's systems beeped softly behind him; alarms were going off everywhere. Tone ignored them - after all these years spent training for every possible scenario, even failure seemed trivial compared to standing on the precipice of discovery.

Suddenly, something caught his eye outside the viewport. A crack appeared across Europa's icy crust before spreading further until it resembled a spiderweb fracture pattern against the frozen landscape below. It was beautiful yet terrifying - a testament to the unforgiving nature of cosmic forces beyond human comprehension.

As Tone continued observing the phenomenon from within the relative safety of the Orion III, another realization struck him like a punch to the gut. His heart raced as he checked the ship's diagnostics one more time. There it was: confirmation of what deep down inside, he already knew - their return shuttle was irreparably damaged due to micro meteorites during descent. With dwindling supplies and no way back home, Tone felt his breath catch in his throat.

"This can't be happening," he whispered under his breath. "How did we let ourselves down like this?"

Tone's mind raced as panic set in. All those years of relentless preparation for this moment had come crashing down around him. He wanted to scream at the top of his lungs, venting every last drop of frustration and despair coursing through his veins. But instead, he remained calm and collected - after all, they didn't send just anyone on missions like these; they chose the best.

Despite the hopelessness gnawing at his insides, Tone couldn't help but feel grateful. At least he wouldn't have to face this fate alone. His crewmates - Dr. Maria Ramirez and Engineer Viktor Petrov - were both exceptional individuals who had dedicated their lives to exploration and discovery just as passionately as he did. They deserved better than this cruel twist of fate, but there was nothing left for them to do except accept it.

Maria approached first, her dark brown eyes filled with concern and understanding. Her voice was gentle yet firm when she spoke. "We'll find a way out of this, Tone. We always have."

Viktor nodded in agreement from behind her, his gray beard betraying worry lines etched deeply onto his weathered face. "Yes," he agreed solemnly. "We are explorers, aren't we? And explorers never give up without exhausting every possible solution."

Tone managed a weak smile at their words; he couldn't ask for better companions during these final days. Together, they decided that despite their grim prospects, they would continue their work on Europa - if not for themselves then at least for future generations who might one day benefit from their discoveries. After all, what better way to honor their sacrifices than by ensuring that their time here wasn't wasted?

And so, over the next few weeks, Tone threw himself into researching Titan - Saturn's largest moon - which held potential signs of extraterrestrial life. Every night, under Europa's cold gaze, he studied data readings sent back from previous missions while pondering the implications they carried. Would humanity ever find life elsewhere in our solar system? Only time would tell.

Days turned into weeks as winter's icy grip tightened around them. Each morning brought new challenges, pushing Tone and his crewmates further towards the brink of desperation. But even amidst despair, there were moments of beauty - breathtaking sunrises painting the horizon orange or watching meteor showers streak across star-studded skies.

These brief respite from reality served as reminders that life persisted even in the most barren environments. Perhaps this was Europa's message: survive against all odds because you never know when a spark might ignite hope once again.

As Tone delved deeper into his research, he found solace in the pursuit of knowledge. Despite everything else crumbling around him, he still had control over one aspect of his existence - learning. And if anything could give purpose to these last days, it was contributing valuable insights about Titan's potential habitability.

But ultimately, Tone knew none of it mattered anymore. No amount of research could save them from their inevitable fate. With dwindling supplies and fading hopes, each day grew darker than the last. Yet somehow, amidst the gloom, he couldn't shake off an indescribable sense of peace - as if all those years spent staring at stars had finally culminated in a bittersweet epiphany.

It wasn't until much later that Tone realized what he had been searching for throughout his entire career as an astronaut: meaning. Not fame, nor recognition, nor even scientific breakthroughs - but simply finding purpose amidst the vast expanse of cosmos where nothing seemed certain except uncertainty itself.

In the end, Tone accepted his fate with grace. After all, isn't that what true exploration is all about - facing unknown territories head-on, armed only with courage and curiosity? He continued researching Titan till the very end, knowing full well that their findings might one day inspire others to venture forth into unexplored realms of space.

And perhaps somewhere along the way, someone else would find meaning too - not just among twinkling constellations or icy moons, but also within themselves. Because ultimately, exploration isn't just about reaching new destinations; it's about discovering parts of yourself you never knew existed.

</details> <details> <summary><b>Write me a 3,000 word opening chapter of a 'high fantasy' novel, drawing inspiration from the writing styles of J R R Tolkien & George R R Martin. Use third person personal. Include dialogue and internal monologues. The POV character for the opening chapter should be a 19 year old female elf bard who is looking for adventure.</b></summary>

In the verdant kingdom of Lirinwood, nestled among the ancient trees and bathed in the golden light of dawn, lived Elenya, an elf bard whose music could charm even the most stoic of hearts. She was young, barely nineteen summers old, yet her spirit shone brighter than the stars that graced the night sky. Her heart, like her melodies, yearned for adventure; it longed to traverse uncharted lands, discover forgotten legacies, and weave tales of valor and valor.

Elenya sat upon a rocky outcrop overlooking the picturesque landscape below, her slender fingers caressing the strings of her enchanted lute. A gentle breeze carried whispers of her melody through the rustling leaves as she strummed with practiced ease. As if responding to her song's call, a group of squirrels emerged from their hidden burrows and danced along branches in time with her music.

"What do I seek?" Elenya murmured aloud, lost in thought while continuing her performance. "Adventure…glory…a story worth telling."

She closed her eyes briefly before reopening them with newfound determination. With one last strummed note echoing softly into silence, she stood up and brushed off her simple dress made of forest-green fabric adorned with silver embroidery depicting intertwining vines.

Her journey began at daybreak, guided by nothing but the compass within her heart. She ventured deep into unexplored forests filled with creatures both benevolent and malevolent alike - some curious about this solitary traveler who dared enter their domain without fear; others wary due to past encounters with those less respectful towards nature's balance.

One such creature approached cautiously after hearing Elenya's haunting melodies echo through the woods: a wise old badger named Brockworth. His fur was a mix of grey and white, reflecting his years spent exploring these forests alongside his fellow woodland dwellers. Despite his weathered appearance, there remained an unmistakable gleam of intelligence in his eyes.

"Greetings," said Elenya, offering him a warm smile. "I am Elenya, bard of Lirinwood. What brings you here today?"

Brockworth eyed her curiously before replying with a raspy voice tinged with wisdom, "I heard your songs on the wind. They stirred something within me - memories perhaps, or echoes of ancient times when elves and animals lived harmoniously side by side."

Elenya nodded attentively as Brockworth continued, "But tell me child, what quest drives you forth so boldly? It seems perilous for one so young."

"I seek adventure," she answered earnestly, "and stories worthy of song. My heart yearns for tales untold and challenges yet unmet."

The badger chuckled softly before adding, "Then beware not only those who would harm you physically but also those whose intentions may lead you astray from your path. Trust not everything you hear or see."

With gratitude in her heart, Elenya thanked Brockworth for his sage advice and continued onward. Her travels took her across babbling brooks, dense thickets teeming with wildlife, and towering hills offering breathtaking views of untamed wilderness. Each step brought fresh sights and sounds that inspired new compositions within her mind.

As twilight descended upon the land, Elenya found herself standing before an ancient oak tree known as Eldertree - home to countless spirits of ages past. Its gnarled roots reached deep into Mother Earth while its sprawling branches touched the very heavens above, bearing witness to countless generations since its first sapling days.

Standing beneath its mighty boughs, Elenya felt a profound connection unlike any other she had experienced thus far in her young life. She closed her eyes once more, allowing her fingers to dance along the lute's frets as if guided by an unseen force. As darkness fell around her, a soft glow emanated from Eldertree itself, casting ethereal shadows upon surrounding foliage.

That night, as sleep claimed Elenya under Eldertree's watchful gaze, dreams of epic adventures filled her thoughts. Little did she know then how true Brockworth's warning proved to be or that the real test of courage lay ahead in unimaginable forms. For now though, all was peaceful beneath the moonlit canopy, save for the gentle humming of a lute being played by an elf bard seeking adventure in this magical realm of endless possibilities.

</details> <details> <summary><b>Write me a 3,000 word opening chapter of a 'weird fiction' novel, drawing inspiration from the writing styles of China Mieville and Neil Gaiman. Use third person personal. Include dialogue and internal monologues. The POV character for the opening chapter should be a male in his 20s called Horton who has just come to the city looking for work.</b></summary>

As the sun dipped below the horizon, casting long shadows across the cobblestone streets of the ancient city, Horton found himself standing before the imposing gates of a sprawling mansion. Its gothic architecture loomed over him like a silent sentinel, guarding secrets that lay hidden within its walls for centuries. The air was thick with an eerie mist that seemed to cling to every surface, giving everything it touched a ghostly quality.

Horton shivered slightly as he took in his surroundings; this was nothing like his sleepy hometown back in the countryside. It was all so different here - strange noises echoed through empty alleyways while peculiar creatures scurried about unseen by those who were not paying attention. And yet there was something undeniably captivating about this place, drawing him further into its mysterious depths.

With a deep breath, Horton steeled himself and pushed open the heavy wrought iron gate that barred entry to the mansion grounds. He stepped inside, feeling small beneath the towering oaks draped in Spanish moss which lined the path leading up to the grand entrance. As he walked, whispers of past inhabitants seemed to drift through the air around him like specters haunting their former home.

"Welcome," said a soft voice behind Horton, causing him to jump in surprise. Turning around quickly, he saw an old man dressed entirely in black leaning against one of the massive oak trees. His eyes twinkled with amusement at Horton's startled reaction. "I'm sorry if I frightened you," he continued calmly. "My name is Archibald."

Horton couldn't help but feel drawn towards this enigmatic stranger despite his initial shock. There was something about Archibald's warm smile and kind eyes that put him instantly at ease. "Are you new to our fair city?" Archibald asked gently after studying Horton for a moment.

"Yes sir," Horton replied hesitantly, unsure how much to reveal about himself or his reasons for coming here. "Just arrived today actually… I'm looking for some work."

The old man nodded sagely as if expecting such an answer. "Ah yes, many young souls seek fortune and adventure within these ancient walls. But be warned, dear boy - not everything is what it seems here."

With those cryptic words hanging in the air between them, Archibald gestured towards the front door of the mansion with a knowing grin. "Shall we go inside then? You'll want to speak with Lady Isabella regarding employment opportunities. She resides upstairs in her private study."

Feeling both intrigue and apprehension, Horton followed Archibald up the creaking wooden stairs and through winding corridors filled with dusty antiques and eerily lifelike portraits. Eventually they arrived at a door adorned with intricate carvings depicting scenes of mystical creatures engaged in arcane rituals.

As they entered the room together, Horton could feel his heart race faster as he took in the sight before him. A regal woman sat hunched over a massive desk littered with parchment scrolls covered in arcane symbols. Her raven hair cascaded down her back like ink spilled upon silk as she met their gaze through piercing ruby eyes framed by delicate ivory lacework.

"Lady Isabella," whispered Archibald reverently before taking leave with only a nod goodbye. Left alone with the enigmatic noblewoman, Horton felt strangely compelled to tell her everything about himself. From humble beginnings growing up on a farm surrounded by nature's beauty to dreams of seeing more than just fields and forests, she listened patiently without interruption as he poured out his desires for adventure and change.

When at last he fell silent, Lady Isabella leaned forward slowly and fixed him with an intense stare that seemed to bore right into his very soul. "You possess great potential, young Horton," she declared after several tense moments. "But first, you must prove yourself worthy of our trust."

She rose from her chair gracefully and moved towards a bookshelf lined with ancient leather-bound tomes. Selecting one carefully from amongst its dusty brethren, she turned back towards him holding it out like some precious artifact. "Take this book and read it closely," she instructed, her voice low and melodious. "Once done, return it to me and we shall see if you have what it takes."

Grateful for the opportunity despite feeling overwhelmed by events unfolding around him thus far, Horton reached out tentatively to accept the weighty volume offered by Lady Isabella. With trembling fingers he opened it eagerly and began reading aloud from the first page:

In days gone by when shadows held sway, A secret power lay hidden away. Seek ye out the lost key that unlocks truth, And free what lies bound by time's cruel youth.

As Horton continued reciting lines of verse interspersed between lengthier passages detailing obscure rituals and incantations written in archaic languages, he gradually became aware of subtle changes occurring within himself. The words seemed somehow alive as they flowed off his tongue; each syllable resonated deeply within him until eventually he found himself lost in their hypnotic cadence.

Suddenly jolted back to reality by a sharp pain shooting through his hand, Horton realized with horror that he had accidentally pricked his finger on one of the rusty metal clasps securing the book closed. Droplets of crimson stained the ancient parchment pages as he hastily closed them again and returned the tome back to Lady Isabella with shaky hands.

The noblewoman merely smiled enigmatically as she accepted it from him, seemingly unaffected by his mishap. "Your task is complete," she stated simply before dismissing him from her presence with another nod of her head.

Left to ponder these strange occurrences alone once more, Horton made his way back outside into the fading twilight. As he wandered aimlessly along winding streets illuminated only by flickering gas lamps, he couldn't shake the feeling that something monumental had shifted deep within his core.

And so began Horton's journey through this bizarre city steeped in mystery and wonder, where magic lingered just beyond human understanding like wisps dancing among moonlit trees. Little did he know then how profoundly his life would soon change as he ventured deeper into unknown realms guided by forces both seen and unseen.

</details>

I chose the hard sci-fi example to test positivity bias. It was willing to kill the protagonist on first try.

I chose the high fantasy example to see whether it would bleed human features through to elves, this didn't occur.

I chose the weird fiction example to see if the LLM understood a niche genre. It performed well.

Training Strategy

I started with a finetune of Mistral Small 22B which had been trained on the Gutenberg dataset: nbeerbower/Mistral-Small-Gutenberg-Doppel-22B.

The first stage of my training was a single epoch at low LR over a 474 million token text completion dataset.

I followed this up with a coherence, decensorship & roleplay finetune over a 172 million token instruct dataset over two epochs.

Total training time was about 32hrs on 4x Nvidia A100 80GB.

<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>