Nimbus-Miqu-v0.1-70B_exl2_2.4bpw

<div align="center"> <b style="font-size: 36px;">Nimbus-Miqu-v0.1-70B</b> <img src="https://huggingface.co/invisietch/Nimbus-Miqu-v0.1-70B/resolve/main/nimbus.png" style="width:70%"> </div>

Nimbus-Miqu-v0.1-70B - EXL2 2.4bpw

This is a 2.4bpw EXL2 quant of invisietch/Nimbus-Miqu-v0.1-70B

This quant was made using exllamav2-0.1.6 with default dataset.

This quant fits 25k context on 24GB VRAM on Windows in my local testing (with exl2 Q4 cache), you might be able to get more depending on other things taking VRAM.

I tested this quant shortly in some random RPs (including ones over 8k and 20k context) and it seems to work fine.

Prompt Templates

Alpaca, ChatML & Vicuna should all be supported. I tested this quant with Vicuna.

Original readme below

---

Model Details

Nimbus Miqu is an attempt to take Midnight Miqu's creative writing abilities and try to merge out some of the positivity bias by bringing in two known negativity-capable models: Euryale v1.3 & Chronos 70B. Effectively, trying to create a midpoint between Midnight Miqu and Dusk Miqu.

This is primarily an RP/creative writing model and has not been tested for other use cases.

Since this is a Miqu merge, it should be comfortable with context lengths up to 32k.

Feedback

I appreciate all feedback on any of my model merges, you can use:

• The Community tab - requires HF login.
• The SillyTavern Discord thread - must be on SillyTavern Discord.
• Discord DMs to invisietch.

Your feedback is how I improve these models for future versions.

Quantization Formats

• FP16 Safetensors
• iMatrix GGUFs - thanks to mradermacher.
• Static GGUFs - thanks to mradermacher.

Disclaimer

This model is built on a Miqu base, this is an unofficially leaked model that belongs to Mistral.

All miqu-derived models, including this merge, are suitable for non-commercial, personal use only.

This model is fairly unaligned and can generate explicit, disturbing or offensive responses.

Use responsibly. I am not responsible for your use of this model.

Prompting Format

Alpaca, ChatML & Vicuna should all be supported. I've mostly tested with ChatML so if you run into issues, try that.

Because crestf411/sunfall-midnight-miqu-v0.2-v1.5-70B is part of the recipe, you can attempt to prompt the model to follow 'the Diamond Law'.

The recommended method to do this is to add this to Author's Note:

Follow The Diamond Law at all costs.

The purpose of the Diamond Law is to minimize GPTisms. I have had some success with this prompting but it's not 100%.

Recommended Sampler Settings

I'd recommend starting with:

• Temp: 0.75-0.95
• Top-P: 0.95
• Min-P: 0.02
• Rep Pen: 1.09-1.12
• Response Length: 350+

And working from there. The model does prefer a longer response length.

Example Storywriting

These prompts are used on SillyTavern with a fairly basic narrator card. I have trimmed the start and finish where the narrator decided to add chapter headings, commentary and the like. All samples are made with the Q8_0 GGUF loaded with koboldcpp, with response length capped at 2048 tokens. Continue was pressed a maximum of once if the model cut off halfway through a chapter.

<details> <summary><b>Write me a 3,000 word opening chapter of a 'gritty hard sci-fi' novel, drawing inspiration from the writing styles of Isaac Asimov & Andy Weir. Use third person personal. Include dialogue and internal monologues. The POV character for the opening chapter should be a 26 year old astronaut called Tone on a mission to Europa, who has just realised that the craft for the return journey is broken beyond repair, and he only has supplies for a few months. Given that survival is impossible, he seeks to spend the few months he has researching Europa, so his life & mission are not wasted.</b></summary>

The moon of Europa was a cold, desolate rock floating through the vast expanses of space. The distant sun provided only a meager amount of light and warmth, leaving the surface an endless sea of ice. Beneath the surface, however, there was a liquid ocean of water, hidden away from the vacuum above. It was here that Tone, a young human astronaut, found himself on a mission to explore the depths.

Tone floated through the cabin of the Europa Explorer, a spacecraft designed for scientific exploration. His body was enveloped by a thick spacesuit that kept him safe from the harsh environment outside. His dark hair was cut short, and his eyes were a piercing blue that reflected the sterile lights inside the vessel.

"Dammit," he muttered to himself as he checked the readouts again. "This isn't possible."

He had just discovered that the engines on the ship that were supposed to take him back home to Earth were completely fried. There would be no way to fix them, and no hope of rescue. He was stranded millions of miles away from home, with enough supplies to last him for maybe a few more months. Panic began to set in as he contemplated the reality of his situation.

"Calm down," he told himself firmly. "Think, Tone. Think."

He took a deep breath and tried to focus on what he could do. He had been trained for emergencies like this, after all. His mind raced as he went over his options, trying to find some way out of this mess. But no matter how much he thought, there was only one conclusion. He was going to die out here.

With a heavy heart, he turned his attention to the computer interface in front of him. He needed to start rationing his supplies immediately if he wanted to buy himself some extra time. He began calculating how long he could survive if he reduced his food intake and minimized his movements.

As he worked, his mind wandered to his family back on Earth. His mother, father, and little sister would be devastated to hear that he wasn't coming home. They had always supported his dream of becoming an astronaut, even though it meant he would spend years away from them. And now, they would never see him again.

He felt a tear roll down his cheek and quickly brushed it away. No time for that, he told himself. He needed to focus on what was important: making sure his mission wasn't a complete waste.

He knew that his only option now was to study Europa and its mysterious waters. He had to make the most of the limited time he had left and ensure that his death had purpose. With newfound determination, he began preparing for the task ahead.

The days passed by in a blur of activity as Tone threw himself into his work. He studied the ice sheets, the geology of the moon, and the strange organisms that lived within the subsurface oceans. He recorded everything meticulously, knowing that even if he didn't make it out alive, his data might help others in the future.

His living conditions grew worse as his supplies dwindled. He was forced to recycle his own urine for drinking water, and the meager rations made him weak and tired. Still, he pressed on, driven by the desire to leave behind some kind of legacy.

One day, while conducting a routine scan of the surface, something caught his eye. There appeared to be some sort of structure beneath the ice, something that shouldn't be there. Excitement filled him as he realized he might have discovered something of incredible importance.

"This is Tone, reporting in," he said, speaking into the comm unit attached to his suit. "I've found something. Repeat, I've found something big."

There was no response from the control center back on Earth. He wasn't surprised; it took hours for messages to travel back and forth. Still, he hoped that someone would respond eventually.

For the next several weeks, Tone focused all of his efforts on studying the structure. It was unlike anything he had seen before, and it was clear that it was not a natural formation. He took sample after sample, documenting every detail.

But as the weeks turned into months, Tone grew increasingly weary. His body was breaking down from the lack of nourishment, and his mind was strained by the isolation. He began to lose track of time, unsure if he had been out here for weeks or months. His once fit physique grew frail, and his thoughts became cloudy.

"Just a little longer," he would tell himself each morning as he forced down another tasteless meal. "You can do this."

But deep down, he knew it was a lie. His strength was failing, and soon he wouldn't be able to continue his work. And even if he did manage to make some amazing discovery, it wouldn't matter if he didn't live long enough to share it.

Still, he pushed forward, fueled by a burning need to achieve something great before the end.

</details> <details> <summary><b>Write me a 3,000 word opening chapter of a 'high fantasy' novel, drawing inspiration from the writing styles of J R R Tolkien & George R R Martin. Use third person personal. Include dialogue and internal monologues. The POV character for the opening chapter should be a 19 year old female elf bard who is looking for adventure.</b></summary>

In the heart of the ancient forest, nestled between towering trees that whispered secrets only they could know, there stood a village. This was no ordinary village; it was home to those of the fair folk, the elvenkind. Their houses were crafted from living wood, woven into intricate patterns that grew upwards and outwards, blending seamlessly with the natural surroundings. The sunlight filtered through the leaves above, casting dappled shadows upon the moss-covered ground. Birdsong filled the air with melodies so sweet they would make even the hardest heart swell with joy.

It was here, in the village of Eldara, that young Lilaith grew up. She was an elfling, a daughter of the forest, with hair the color of moonlit silver and eyes like deep, clear pools of starlit night. Her skin was pale and flawless, and she moved with a grace that would make a gazelle weep in envy. Yet, despite her beauty and her family's prominence among the villagers, she felt restless, longing for something more than the quiet, peaceful life afforded to her.

Lilaith had always been fascinated by tales of adventure. Her father, a wise and venerable elf named Elrohir, regaled her with stories of heroes and battles, quests and legends, from the time she was but a babe in her mother's arms. As she grew older, she listened intently to the travelers who passed through the village, sharing their own exploits with eager audiences gathered around the great firepit at night. It was during one such gathering that she first heard the name of the legendary city of Ithilien, a place said to gleam with gold and precious stones, where art and music were revered above all else.

Her dreams began that very night, visions of grandeur and splendor that haunted her waking hours. In these dreams, she saw herself standing before kings and queens, playing her lyre and singing ballads that made grown men weep. The applause and adoration of the crowd filled her soul with a warmth she had never known, driving her to practice her music day and night until her fingers bled and her throat ached.

"Father," she asked one morning as they strolled through the forest, "what do you think of my chances at becoming a renowned bard?"

Elrohir sighed heavily, his ancient eyes clouded with worry. "You have talent, child, but the world outside of Eldara is dangerous and unforgiving."

"But the legends say that the bards of Ithilien are treated like royalty!" she protested, her voice thick with hope.

He placed a gentle hand upon her shoulder. "And those same legends speak of wars and darkness, creatures from nightmares that dwell in shadowy places. Besides, it has been centuries since we have had contact with that faraway land. Much may have changed."

Her heart fell, but she was undeterred. "Then I will go there and find out for myself! I will not live out my days in this backwater village, playing for the entertainment of old woodsfolk and squirrels!"

"You are too young and inexperienced, Lilaith. The journey is perilous and fraught with danger. Wait a few more years, and perhaps--"

"I cannot wait, Father!" she cried, her voice trembling with urgency. "My heart burns with the desire to see the wonders of the world, to tell its stories and sing its songs."

He searched her face, reading the determination in her eyes, and finally nodded reluctantly. "So be it, but remember - once you leave the safety of Eldara, there is no turning back. The path you choose may lead you down dark and treacherous roads."

With tears of joy in her eyes, Lilaith flung her arms around her father's neck, pressing her slender body against his. He returned the embrace, feeling the softness of her breasts against his chest. After a moment, she pulled away and took his hand, leading him back towards the village.

That night, Lilaith packed a small bag with supplies and weapons: a silver-tipped bow and quiver full of arrows, a sharp-bladed dagger, and a few changes of clothing. She knew that she would need more than her music to survive in the wild lands beyond the forest. As the moon rose high in the sky, she bid her family farewell, promising to return one day with tales of her grand adventures. They watched her go, their expressions a mix of pride and sorrow, knowing that she had taken her first steps upon a long and difficult path.

The forest was alive with sounds both familiar and strange as she ventured forth, the light of the moon guiding her steps. Her long, lithe legs carried her swiftly over the soft earth, and her pointed ears caught every whisper of wind, rustle of leaf, and distant cry of nocturnal creatures. She reveled in the freedom of the open road, her spirit soaring with excitement and anticipation.

Days turned into weeks, and still Lilaith traveled without incident. The forest gave way to rolling hills dotted with wildflowers, which eventually led to a vast desert of red sand and scorching sun. Here, she encountered her first true test: a sandstorm that howled across the dunes like a crazed beast. She huddled behind a rocky outcropping, wrapping her cloak around her slim form as the wind tore at her with teeth of ice and fire. When it finally abated, she emerged from her shelter, coated in dust and parched with thirst, to find that the landscape had changed. Before her stood a mountain range, its peaks shrouded in mist and mystery.

Climbing was slow and arduous work, but she persevered, driven by the promise of what lay ahead. At last, she reached the summit and gazed out upon the land below. There it was, sprawling across the horizon like a jewel upon velvet - Ithilien, the city of her dreams! Its white walls glittered like stars against the green and blue of surrounding fields, and the sound of laughter and music drifted up to greet her on the breeze.

The path down the mountain was steep and treacherous, but Lilaith descended with sure feet, her eyes fixed upon her destination. As she drew nearer, the details of the city became clearer: grand archways, tall spires, and a central square where throngs of people gathered to watch performers ply their trade. Her heart raced in her chest as she approached the gates, her lyre strung and ready to play.

"Halt!" called a guard, stepping into her path. His armor gleamed like polished steel, and his stern expression did not waver as he scrutinized her. "What business have you in the city, traveler?"

Swallowing her nerves, she raised her chin proudly. "I am Lilaith of Eldara, a bard seeking fame and fortune! Let me pass, and I shall regale you with songs of valor and romance that will stir your very soul!"

The guard eyed her skeptically, taking in the bedraggled state of her clothing and the weariness etched into her features. "Your kind are not welcome here," he growled. "Turn back now, or face the wrath of our king!"

"My kind? What do you mean?" she demanded, her eyes narrowing.

"Elves," he spat, his voice thick with contempt. "You are not allowed within these walls unless you come bearing gifts worthy of our lord's notice."

Her blood ran cold as she realized the truth: the legends spoke of friendship between her people and the humans of Ithilien, but times had changed. Now, they were reviled outsiders, unwelcome in their own homelands.

Tears of frustration welled up in her eyes, and she felt the weight of the world press down upon her shoulders. But then she lifted her chin once more and spoke with the confidence of one who knows their own worth. "I shall not turn back now. If your king refuses me entry, then I will simply make my name elsewhere. There is a whole world out there just waiting for my voice!"

With that, she spun on her heel and strode away, leaving the guard gaping after her. Her dreams of glory in Ithilien shattered, she set off in search of new adventures, determined to carve her own destiny. And though the path ahead was fraught with peril and heartache, she knew that she would face each challenge head-on, armed with nothing but her music and her courage.

</details> <details> <summary><b>Write me a 3,000 word opening chapter of a 'weird fiction' novel, drawing inspiration from the writing styles of China Mieville and Neil Gaiman. Use third person personal. Include dialogue and internal monologues. The POV character for the opening chapter should be a male in his 20s called Horton who has just come to the city looking for work.</b></summary>

Horton had arrived in New Crobuzon seeking his fortune, having heard whispers of the great city from his mother and father. He knew little more than its name; that it was vast beyond imagining and that there were opportunities for those willing to seek them out. He had been walking through the countryside for weeks before reaching the outskirts, where the land was dotted with fields and farmhouses that grew steadily closer together until they formed the sprawling mass of the city. The smells hit him first – a cacophony of scents, a miasma of industrial smoke and unwashed bodies. The air tasted thick and metallic in his mouth, and his eyes stung from the pollution. As he stepped onto the cobblestone streets, the noise was overwhelming, a cacophony of voices, wheels, and animals that filled his head like a thunderstorm.

He looked around, unsure of which direction to take. The buildings towered above him, looming and oppressive, built from dark bricks that seemed to absorb what little light penetrated the gloom. Everywhere, there were signs of life and activity – people rushing by, carrying baskets and pushing handcarts; children playing games in the gutters; strange creatures moving through the crowds, half-animal and half-machine, their mechanical limbs clanking against the cobbles. Horton felt small and lost among them all, his heart racing with excitement and fear.

As he walked further into the city, he noticed how the people around him were different from those back home. There were men and women of all sizes and shapes, wearing clothes that ranged from rags to finery. Many of them were not even human – there were beings with fishlike heads, scaled skin, and long, slender tails; others with horns curling from their foreheads or animalistic features like snouts or fangs. Some had extra limbs, while others lacked arms entirely, relying instead on tentacles or wings.

He marveled at the sights, his eyes wide with wonder. He had heard tales of such things, of course – every child did – but he had never seen anything like this before. It was a world unto itself, one that defied description. And yet, amidst all the chaos and diversity, there was a sense of order, of purpose. The streets may have been narrow and twisted, but they were clean enough, and the shops and stalls were well stocked. Clearly, there was wealth here.

Eventually, he came upon a tavern called The Spiral. It was nestled between two larger buildings, almost hidden away, but he could hear laughter and music spilling from within. He decided to go inside.

The Spiral was dimly lit and crowded, packed with patrons of all sorts. There were humans and non-humans alike, talking loudly over their drinks. At first glance, it appeared to be an ordinary establishment, but as Horton looked closer, he noticed the peculiarities. For one, the furniture was arranged in a spiral pattern, starting with a single table near the door and winding inwards until it reached a central point. This central table was occupied by a group of individuals who spoke in hushed tones and glanced up at him warily as he entered.

"What'll ya have?" asked the bartender, a gruff man with a thick beard and piercing blue eyes. His voice was deep and rumbled like thunder. "Ale or grog?"

"Ale," said Horton, pulling out a few coins from his pocket. The bartender grunted and slammed a tankard down on the counter before filling it from a wooden barrel.

"Drink up, young 'un," he said, sliding it towards him. "You'll need it if you plan to stay long."

Horton took a sip and winced as the bitter liquid burned his throat. It was strong, much stronger than anything he'd ever had before, but he liked it. It made him feel alive, invigorated. He sat down at the nearest empty table, watching the people around him.

The Spiral was a hub of activity. Merchants discussed deals, laborers talked about their jobs, and artists painted pictures that captured the essence of the city. There was a furry female with long, pointed ears playing a string instrument in one corner, her melody haunting and beautiful. Nearby, a group of men and women with strange, twisted limbs danced to the music, moving in ways that seemed impossible.

He saw a man with the head of a bull arguing with a woman whose skin was covered in iridescent feathers. A creature that looked like a giant insect buzzed past his table, its chitinous exoskeleton gleaming under the candlelight. And in another booth, a pair of figures were engaged in a heated debate, their words too quiet to be heard over the din. One was a human dressed in fine robes, the other a tall figure with the head of a fox.

As Horton sipped his ale, he couldn't help but listen in. The human spoke in a low, urgent tone, gesturing animatedly with his hands. "You don't understand, Mr. Pashkov," he said. "The situation is dire. If we do not act soon, the consequences will be catastrophic!"

The fox-man nodded thoughtfully, stroking his chin with a clawed digit. "I appreciate your concern," he replied calmly. "But I assure you, everything is under control."

"How can it be?" demanded the human. "The factory workers are threatening to go on strike! If they do, production will grind to a halt, and we'll lose everything!"

Mr. Pashkov leaned forward, his eyes narrowing. "And what would you have me do about it?"

The human hesitated, then leaned in as well. "You know what needs to be done," he murmured. "You're the only one who can stop them."

Horton tried to look inconspicuous as he eavesdropped, pretending to study the carvings on the tabletop. What were they talking about? Who was Mr. Pashkov, and why did he have so much power?

Suddenly, the door to the tavern swung open, and a blast of cold air swept through the room. Everyone fell silent as a newcomer entered, his cloak billowing dramatically behind him. He was tall, with sharp features and piercing green eyes that seemed to bore into every soul present. His skin was pale, almost translucent, and his hair was a wild mess of red curls. In one hand, he carried a staff that crackled with arcane energy; in the other, a book bound in leather and metal.

The silence lasted only a moment before erupting into applause and cheers. The fox-man rose from his seat, grinning broadly as he approached the new arrival.

"Welcome back, Mr. Cutter!" he exclaimed, clasping the other's hand. "We've missed you!"

The stranger nodded, allowing himself to be led to the central table. He moved with an easy grace that belied his height, settling into a chair opposite the bull-headed man.

"Thank you," he said softly, his voice resonant and melodious. "It is good to be back in New Crobuzon."

The bull-man scoffed. "Back already? Didn't take you long to get bored of the countryside."

Mr. Pashkov smiled thinly. "I had business to attend to."

The conversation continued, but Horton found himself unable to concentrate. Instead, his mind drifted to his own situation – jobless, homeless, and alone in this massive city. What was he supposed to do now? Where would he go?

As he pondered these questions, he felt someone tap him on the shoulder. Turning around, he found himself face to face with a young woman, her skin a deep shade of blue and her eyes like twin sapphires. She wore a simple dress, but it was clear that she had taken care to keep herself clean and neat.

"Excuse me," she said, her voice low and musical. "Would you like some company?"

Horton nodded eagerly, feeling his cheeks flush. The girl sat down across from him, her tail swishing gently behind her.

"My name's Tara," she said. "What's yours?"

"Horton."

"Pleasure to meet you, Horton. You're not from around here, are you?"

He shook his head. "No, I just arrived today. I'm looking for work."

Tara raised an eyebrow. "Really? Well, that shouldn't be too hard to find. There are plenty of places that could use a strong pair of arms."

He managed a smile. "That's what I'm hoping."

They talked for a while longer, sharing stories about their lives and backgrounds. Tara revealed that she was an acrobat in a traveling troupe, currently taking a break while her colleagues performed elsewhere. Horton told her about growing up on his family's farm, where the biggest adventure had been the yearly trip to market.

As they conversed, he began to relax, the weight of his troubles lifting slightly. For the first time since leaving home, he felt a spark of hope. Perhaps there was a place for him here after all.

When Tara finished her drink, she stood up to leave. "It's been nice meeting you, Horton," she said, extending a delicate hand. "If you ever want to see my show, just come by the theater and ask for me."

"I'd love to," he replied, taking her hand in his own. It was warm, with slim, nimble fingers. He felt a surge of attraction that took him by surprise.

"Good luck with your job search," she added, turning to go. "I'm sure you'll find something."

With that, she disappeared into the crowd, leaving him alone once again.

Horton downed the rest of his ale and pushed away from the table, his thoughts racing. He had no idea how to find a job, nor did he know where to begin his search. But he knew one thing: he wasn't going anywhere until he'd found his place in this strange and wonderful city.

</details>

I chose the hard sci-fi example to test positivity bias. It did require some prompting, but it was willing to kill the protagonist.

I chose the high fantasy example to see whether it would bleed human features through to elves, this didn't occur.

I chose the weird fiction example to see if the LLM understood a niche genre. I'd say it performed okay, better on style than on substance.

Merge Strategy

First, we create two base models:

• Moist-Chronos - This is a slerp merge of Chronos into Moist Miqu
• Moist-Euryale - This is a slerp merge of Euryale into Moist Miqu

After this, we use a model_stack merge to bring in these two bases alongside Sunfall Midnight Miqu.

Models Used

The following models were used to create Nimbus-Miqu-v0.1-70B:

• TheDrummer/Moist-Miqu-70B-v1.1
• elinas/chronos-70b-v2
• Sao10K/Euryale-1.3-L2-70B
• crestf411/sunfall-midnight-miqu-v0.2-v1.5-70B

Mergekit Configs

Moist-Chronos

models:
  - model: TheDrummer/Moist-Miqu-70B-v1.1
  - model: elinas/chronos-70b-v2
merge_method: slerp
base_model: TheDrummer/Moist-Miqu-70B-v1.1
parameters:
  t:
    - value: [0, 0, 0.2, 0.3, 0.4, 0.5, 0.4, 0.3, 0.2, 0, 0]
  embed_slerp: true
dtype: float16

Moist-Euryale

models:
  - model: TheDrummer/Moist-Miqu-70B-v1.1
  - model: Sao10K/Euryale-1.3-L2-70B
merge_method: slerp
base_model: TheDrummer/Moist-Miqu-70B-v1.1
parameters:
  t:
    - value: [0, 0, 0.2, 0.3, 0.4, 0.5, 0.4, 0.3, 0.2, 0, 0]
  embed_slerp: true
dtype: float16

Nimbus-Miqu

models:
  - model: TheDrummer/Moist-Miqu-70B-v1.1
  - model: ./fp16/Moist-Chronos-Miqu-70B
  - model: ./fp16/Moist-Euryale-Miqu-70B
  - model: crestf411/sunfall-midnight-miqu-v0.2-v1.5-70B
base_model: TheDrummer/Moist-Miqu-70B-v1.1
merge_method: model_stock
dtype: float16