Gemma-The-Writer-9B-GGUF

<i><font color="red">NEW:</font> Heretic, Uncensored, Abliterated version of "Gemma The Writer" HERE </i>

<h3>Gemma-The-Writer-9B-GGUF</h3> <img src="the-writer.jpg" style="float:right; width:300px; height:300px; padding:10px;">

This is a Gemma2 model merge of the top storytelling / writing models as noted at EQBench, tuned specifically for fiction, story, and writing.

Due to high stability and compressed nature of the model you can also use it for general use too, including roleplay.

This model requires GEMMA Instruct template, and has 8k context window but is extendable via rope to 32k or higher.

Recommended Rep Pen of 1.05 or higher, temp range 0-5.

Example outputs below.

<B>Settings, Quants and Critical Operations Notes:</b>

Change in temp (ie, .4, .8, 1.5, 2, 3 ) will drastically alter output.

Rep pen settings will also alter output too.

This model needs "rep pen" of 1.02 or higher.

For role play: Rep pen of 1.05 to 1.08 is suggested.

Raise/lower rep pen SLOWLY ie: 1.011, 1.012 ...

Rep pen will alter prose, word choice (lower rep pen=small words / more small word - sometimes) and creativity.

To really push the model:

Rep pen 1.05 or lower / Temp 3+

Longer prompts vastly increase the quality of the model's output.

---

<B>Special Notice:</B>

The new version of this model, mastered from float 32 source files is located here:

[ https://huggingface.co/DavidAU/Gemma-The-Writer-Mighty-Sword-9B-GGUF ]

This version will outform "Gemma The Writer 9B".

It also has new augmented quants, in addition to new float 32 mastered quants.

---

QUANT CHOICE(S):

Higher quants will have more detail, nuance and in some cases stronger "emotional" levels. Characters will also be more "fleshed out" too. Sense of "there" will also increase.

Q4KM/Q4KS are good, strong quants however if you can run Q5, Q6 or Q8 - go for the highest quant you can.

This repo also has 3 "ARM" quants for computers that support this quant. If you use these on a "non arm" machine token per second will be very low.

IQ4XS: Due to the unusual nature of this quant (mixture/processing), generations from it will be different then other quants.

You may want to try it / compare it to other quant(s) output.

Special note on Q2k/Q3 quants:

You may need to use temp 2 or lower with these quants (1 or lower for q2k). Just too much compression at this level, damaging the model. I will see if Imatrix versions of these quants will function better.

Rep pen adjustments may also be required to get the most out of this model at this/these quant level(s).

<B>Settings: CHAT / ROLEPLAY and/or SMOOTHER operation of this model:</B>

In "KoboldCpp" or "oobabooga/text-generation-webui" or "Silly Tavern" ;

Set the "Smoothing_factor" to 1.5 to 2.5

: in KoboldCpp -> Settings->Samplers->Advanced-> "Smooth_F"

: in text-generation-webui -> parameters -> lower right.

: In Silly Tavern this is called: "Smoothing"

NOTE: For "text-generation-webui"

-> if using GGUFs you need to use "llama_HF" (which involves downloading some config files from the SOURCE version of this model)

Source versions (and config files) of my models are here:

https://huggingface.co/collections/DavidAU/d-au-source-files-for-gguf-exl2-awq-gptq-hqq-etc-etc-66b55cb8ba25f914cbf210be

OTHER OPTIONS:

• Increase rep pen to 1.1 to 1.15 (you don't need to do this if you use "smoothing_factor")

• If the interface/program you are using to run AI MODELS supports "Quadratic Sampling" ("smoothing") just make the adjustment as noted.

<B>Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers</B>

This a "Class 1" model:

For all settings used for this model (including specifics for its "class"), including example generation(s) and for advanced settings guide (which many times addresses any model issue(s)), including methods to improve model performance for all use case(s) as well as chat, roleplay and other use case(s) please see:

[ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ]

You can see all parameters used for generation, in addition to advanced parameters and samplers to get the most out of this model here:

[ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ]

<B>Other Versions of "Gemma The Writer": </B>

Gemma-The-Writer-DEADLINE:

The second version of this model is "Deadline" at 10B parameters. It is a specially modified version that changes prose, sentence structure, story telling, reduces "GPTISMS", and generally improves all parts of the model.

Output generation length is almost 2x more on average than "Gemma The Writer 9B"

[ https://huggingface.co/DavidAU/Gemma-The-Writer-DEADLINE-10B-GGUF ]

This is not a replacement for "Gemma The Writer 9B" ; it is a very different version or "flavour" so to speak.

Gemma-The-Writer-J.GutenBerg-10B:

Stronger prose, with a taste of "Gutenberg", and Brainstorm 5X modified.

[ https://huggingface.co/DavidAU/Gemma-The-Writer-J.GutenBerg-10B-GGUF ]

Gemma-The-Writer-N-Restless-Quill-10B (uncensored):

Strong and varied prose, sentences, and paragraphs with more tempered metaphor generations. This version is also uncensored with censorship controlled at the prompt level. This version also allows prose control via pre-prompt (shown at the repo)

[ https://huggingface.co/DavidAU/Gemma-The-Writer-N-Restless-Quill-10B-GGUF ]

You may want to download all and try them out.

<B>Models Used:</b>

This is a high precision "DARE TIES" merge at the layer level (each layer per model adjusted - 168 points of adjustment over the 4 models) comprised of these models:

[ https://huggingface.co/lemon07r/Gemma-2-Ataraxy-9B ]

[ https://huggingface.co/UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3 ]

[ https://huggingface.co/ifable/gemma-2-Ifable-9B ]

[ https://huggingface.co/princeton-nlp/gemma-2-9b-it-SimPO ]

Special thanks to all the model makers. Great work!

---

<h3>Example #1 at q8, Temp:1</h3>

---

<B> Science Fiction: The Last Transmission - Write a story that takes place entirely within a spaceship's cockpit as the sole surviving crew member attempts to send a final message back to Earth before the ship's power runs out. The story should explore themes of isolation, sacrifice, and the importance of human connection in the face of adversity. 800-1000 words. </B>

Captain Eva Rostova’s gloved fingers hovered over the console, her reflection warped in the array of flickering monitors. Outside, the starfield was a mesmerizing tapestry of diamond dust against absolute black, yet it offered no comfort. The hum of failing life support systems whispered through the cockpit like a mournful dirge. Her ship, the Vanguard, was dying, and with it, her last hope of reaching Earth.

It had been three months since the solar flare, three agonizing months of dwindling power reserves, watching her crew succumb one by one to radiation poisoning, their laughter and arguments replaced by the sterile silence of death. Now, she was alone, tethered to a ghost ship hurtling towards an oblivion mirrored in her own fading consciousness.

Her gaze fell on the comms panel, its familiar green glow a beacon of defiance against the encroaching darkness. It was a long shot, a desperate plea into the cosmic void, but it was all she had left. She had to send the message, for them, for Earth, for herself.

“This is Captain Eva Rostova, Vanguard…,” she began, her voice strained, echoing unnaturally in the cavernous cockpit. “Last transmission. All other crew deceased. Cause: catastrophic solar flare…unforeseen…magnitude…” Her throat tightened. Words felt inadequate, unable to encapsulate the horror and grief that gnawed at her soul.

She took a shuddering breath, focusing on the technical details, forcing herself to remain detached, professional. “Ship systems critically compromised…power failing within estimated 24 Earth hours. Location coordinates attached. Apologies for incomplete mission parameters…” A bitter laugh escaped her lips. Incomplete? Their mission to Proxima Centauri b had been reduced to a desperate scramble for survival, a race against time they’d tragically lost.

Eva switched to a personal frequency, hoping against hope that someone, somewhere, might pick up the faint signal. “To my family back on Kepler-186f…mother, father…sister Lena…if this reaches you, know that I loved you all beyond measure. Forgive me for not returning. The universe sometimes demands… sacrifices…” Her voice cracked, tears blurring her vision. She wiped them away furiously, refusing to let emotion cripple her in this final act.

“To the Vanguard crew…Mark, Elena, Jax, Liam…you were more than colleagues, you were family. We faced the unknown together, laughed together, grieved together…and in the end, we fell together. Remember our courage, remember our bond. Let it be a testament to the indomitable spirit of humanity, even in the face of annihilation.”

Silence descended again, heavier now, pregnant with the finality of her words. Eva stared at the dwindling power indicator, its red bar creeping inexorably towards zero. She had poured her soul into that transmission, a lifeline cast into the cosmic ocean, a desperate plea for connection in the face of utter solitude.

But even as she typed the closing salutation, a chilling realization washed over her. The universe rarely answered pleas; it simply existed, indifferent to human sorrow and triumph. This message, like a firefly’s blink against starlight, would likely vanish into the vastness, unheard, unseen.

And yet, a flicker of defiance remained within her. Even if no one received her words, sending them was an act of rebellion against the encroaching darkness. It was a testament to her humanity, her refusal to surrender to despair completely. In those final moments, surrounded by the ghosts of her crew and the dying hum of her ship, Eva Rostova found solace not in the possibility of a response, but in the act of connection itself.

She had spoken, she had remembered, she had loved. And in that, there was a kind of victory, a quiet triumph over oblivion. As the lights flickered and died, plunging the cockpit into absolute blackness, Eva Rostova closed her eyes, a single tear tracing a path down her cheek, a silent echo of a heart that refused to be silenced. The Vanguard fell silent, a forgotten monument to human courage and loss, adrift in the infinite expanse, carrying with it a final, unheard message of love and sacrifice.

---

<h3>Example #2, Q8, rep pen 1.05, Temp:1 </h3>

---

<B> In the style of Ernest Hemingway: Write a scene in a larger story about a soldier returning home from war. The scene is set in a bar. It should carry the jadedness of a seasoned combatant but with a wry humour. Write in Heminway's voice, naturally: Use short, declarative sentences and simple, straightforward language. Include dialogue that is terse and revealing of character. Write with a keen sense of time and place. Capture realness and authenticity. 800-1000 words, past tense. </B>

The air in the Rusty Tap hung thick with stale beer and sawdust. Rain hammered against the windowpanes, a rhythm familiar from foxholes and trenches. Frank sat hunched over a chipped glass, whiskey neat. It burned going down, a familiar warmth that was more akin to a searing brand than comfort. He’d been home a week. A week in this town that felt both alien and desperately familiar.

A kid, barely out of his teens, sidled up beside him. Clean-cut, shiny boots, too eager smile. The boy wore a uniform – a fresh one, pressed – but it hung on him like a costume.

“You served?” the kid asked, eyes bright with misplaced admiration.

Frank grunted. “Somewhere.”

“Over there,” the kid pressed, nodding vaguely eastward. “Heard stories.” He took a gulp of his beer, all nervous energy and bravado. “They say it’s hell.”

Frank finished his whiskey, the burn lingering. “Hell is a place you carry inside, son. Not somewhere on a map.”

The kid blinked, the smile faltering. Frank watched him, saw the flicker of something akin to understanding in those too-bright eyes. It was a fleeting thing, quickly replaced by a forced cheerfulness.

“So, you back for good?” the kid asked, shifting uncomfortably.

“Maybe,” Frank said, his voice rough. “Depends on how long the ghosts stick around.”

The kid chuckled, a nervous, hollow sound. Frank didn’t offer him another word. He raised his empty glass, caught the eye of Maggie, the bartender, a woman with weathered hands and a face etched with stories of her own. She refilled his glass without a word.

The Rusty Tap was a sanctuary of sorts. Familiar faces, the dim yellow light filtering through grime-coated windows, the clinking of glasses – a symphony of normalcy. But the normalcy was a thin veneer over something raw and exposed. Frank saw it in the way the regulars avoided eye contact, in their stiff nods and mumbled greetings. They’d seen the world too, felt the undertow of loss and witnessed horrors that left scars deeper than any shrapnel wound.

A group of men in worn work clothes entered, their laughter loud and boisterous, a jarring counterpoint to the low hum of quiet resignation that permeated the bar. They slammed onto stools, their voices vying for dominance, drowning out the rain’s drumming. One of them, burly with a handlebar moustache, caught Frank’s eye. He grinned, a flash of white teeth in a sun-baked face.

“Frank! Back from the killing fields, eh? Figured you’d be sipping on something stronger than sorrow.” He slapped Frank’s shoulder hard enough to jolt him. “Come on, join us. We got stories to swap, tall tales to weave.”

Frank shook his head, a weariness settling deep in his bones. “Not tonight, Jimmy.”

The burly man, Jimmy, frowned, but didn’t press. He knew. They all did. Some wounds couldn’t be soothed with cheap beer and boisterous tales. Not anymore.

He raised his glass in a silent toast, then turned back to his companions, their laughter resuming, a shield against the unspoken truths that lingered in the air. Frank watched them, a detached observer. He understood their need for bravado, for communal storytelling as a balm against the silence that gnawed at him. But for him, the silence was a companion, a constant reminder of what had been lost, of the things left unsaid.

The door creaked open again, letting in a gust of wind and a young woman, her face hidden beneath a wide-brimmed hat. She moved with a grace that seemed out of place in the rough-hewn bar, her steps light despite the downpour. She scanned the room, her eyes finally settling on Frank. He recognized her – Mary, his neighbor’s daughter. A bright spark back then, full of laughter and youthful dreams. Now, a shadow of that girl lingered, her face etched with a maturity that belied her years.

She walked towards him, her gaze unwavering. Frank felt the familiar constriction in his chest, a tightness that had become his default setting. He didn’t meet her eyes. He knew what she saw, what she couldn’t articulate – the hollowness behind his façade of indifference.

“You look tired, Frank,” she said, her voice soft but carrying the weight of unspoken words. She placed a worn photograph on the bar in front of him. A faded picture of two laughing children, their faces smeared with ice cream.

Frank looked at the picture, at the innocent joy captured in frozen time. His own daughter and son. Gone. He didn’t need to speak. The silence between them was a shared language, forged in grief and unspoken understanding. He reached for the photo, his fingers brushing hers for a fleeting moment. A spark of connection, fragile yet real, flickered in that shared touch. Then, she was gone, swallowed by the rain-drenched night, leaving him alone with the ghosts and the memory of laughter long silenced.

The Rusty Tap’s familiar symphony continued, a muted backdrop to the internal storm raging within him. Frank raised his glass again, the whiskey burning a familiar path down his throat. It wasn’t solace, not really. It was a temporary anesthetic, a way to dull the edges of the truth he carried. He was home, yes, but home wasn't a place. It was a feeling, a phantom limb aching for what could never be reclaimed. And in that aching, in that profound emptiness, he found a twisted sort of peace. The peace of acceptance, of knowing he was forever marked, forever changed by the war he’d survived, and the battles he still waged within himself.

---

<h3>Example #3 at q4km, Temp:1</h3>

---

<B> Start a 1000 word scene (vivid horror, 1st person, present tense, include thoughts) with: The sky scraper swayed, as she watched the window in front of her on the 21 floor explode... </b>

The skyscraper sways, and I watch the window in front of me on the 21st floor explode outward in a shower of glass and splintered wood. A scream catches in my throat, a choked bird desperate for escape. My legs are rooted to the plush carpet, adrenaline a hot coil tightening in my gut. Dust motes dance in the sudden, frigid blast of air that whips through the office, carrying with it the metallic tang of blood and something else, something acrid and alien, like burnt ozone after a lightning strike. Panic claws its way up my throat, a feral thing hungry to be unleashed.

It wasn't supposed to be like this. Not today. Not ever. My carefully curated life, built on spreadsheets and board meetings, on predictable commutes and takeout dinners for one, shattered by an impossible, grotesque breach in reality. The world outside the gaping hole is no longer the familiar cityscape I knew. It writhes, a kaleidoscope of impossible angles and colors that assaults my vision. Jagged, obsidian structures pierce the distorted sky, their surfaces slick with a viscous, luminescent ooze that pulses with an eerie inner light. Tendrils of inky blackness snake from these monstrosities, reaching towards our floor like grasping claws, seeking purchase on the familiar solidity of our world.

My mind screams for logic, for explanation, but finds only a terrifying void. This…this can’t be happening. It's a nightmare, a hallucination brought on by stress, by the relentless pressure cooker of this corporate life. I squeeze my eyes shut, willing the scene away, but the stench of decay and the guttural growls emanating from the breach are undeniable. They seep into my very being, chilling me to the bone.

When I dare to open my eyes again, the tendrils have advanced, slithering across the shattered window frame like liquid shadows. Their touch leaves scorch marks on the pristine marble floor, hissing and smoking as if consuming the very essence of our reality. My breath hitches in my chest. This isn't a hallucination. This is real, terrifyingly real, and it's coming for us.

A strangled gasp escapes my neighbour, Mr. Henderson, his face pale as parchment, eyes wide with primal fear. He scrambles back, tripping over his own feet, the clatter of his expensive briefcase adding a jarring counterpoint to the symphony of horror unfolding before us. His terrified whimper is cut short by a sickening crunch as one of the tendrils lashes out, its barbed tip burying itself deep into his chest. A spray of crimson blossoms across his tailored suit, painting a macabre masterpiece on the pristine fabric. His lifeless eyes stare vacantly at the encroaching darkness, reflecting the abyss that has consumed our world.

Terror, raw and primal, floods my system, drowning out all thought save for one desperate imperative: survive. Adrenaline surges, propelling me into action. I scramble towards the back of the office, past the overturned filing cabinets and shattered glass, towards the emergency exit. My hand shakes as I fumble with the latch, the metal cold and slick under my clammy grip. It finally gives way with a screech that echoes through the office like a death knell.

The corridor beyond is a nightmare tableau. Bodies litter the floor, contorted in postures of unimaginable agony, their faces frozen in masks of terror. The tendrils writhe, pulsing with an unnatural light, consuming everything in their path. They slither over walls, staining the once-pristine surfaces with their viscous ichor. A low, guttural moan reverberates from the darkness, a sound that burrows into my soul, promising unimaginable torment.

I press myself against the cold metal wall, heart hammering against my ribs like a frantic bird trapped in a cage. My eyes dart frantically, searching for an escape, a glimmer of hope in this unfolding apocalypse. There—a fire escape, its rusted ladder leading precariously into the abyss. It’s my only chance.

Taking a shuddering breath, I clamber onto the narrow platform, the groaning metal protesting under my weight. Below, the cityscape has dissolved into a swirling vortex of impossible geometry and monstrous forms. The tendrils reach skyward, grasping at me like skeletal fingers, their touch promising unimaginable horrors. The growls intensify, morphing into a chorus of hungry voices, drawing closer with each agonizing second.

Fear threatens to paralyze me, but the primal instinct to survive surges stronger. I scramble upward, rung by rung, the wind whipping at my face, carrying with it the stench of decay and the whispers of nameless entities that haunt this newly birthed nightmare. Above, through the distorted veil of the alien sky, I glimpse a sliver of the old world – a lone, defiant star burning amidst the encroaching darkness. A flicker of hope, fragile yet tenacious, anchors me to this desperate climb.

I won't give in. I won't become another victim in this grotesque cosmic play. I will reach that star, cling to its light, and somehow, against all odds, survive.

---

<h3> Example #4, Rep pen 1.1, Temp 1, Q8 </h3> <B>

Epistolary Apocalyptic Survival: Write a story set in a post-apocalyptic world, told through a series of letters or messages left behind by survivors. The story should chronicle the challenges and triumphs of two small groups of survivors as they navigate the dangers of this new world and seek to rebuild their lives. The groups communicate through letters left in a drop box. The story proceeds as a new settlement is discovered. They are approached, and initially given entry on the ostensible promise of a trade deal, but things go sideways. Create distinct characters and write in their voice. The story will be set in New Zealand 30 years after a civilisational collapse. 800-1000 words.

</B>

Drop Box Dispatch #47: Aotearoa, Year 30

From: Elara (Kaitiaki Collective) - Piha Beach

To: Tane (Waka Hauora) - Waitomo Caves

Tane,

Another harsh winter bites at the coast. Supplies are dwindling again. The storm last week took our salt-fish drying racks; the ocean seems angrier these days, lashing out like a wounded beast. We salvaged what we could, but it won't be enough. We’re clinging to hope that spring brings plump shellfish and bountiful seaweed harvests.

Speaking of hope, young Mara caught something intriguing in our last scavenging run up north – a weathered metal box tucked beneath the rubble of an old supermarket. Inside were several worn journals and… letters! Addressed to someone named "Tane" from a place called "Waitomo Caves." Could this be you? We deciphered enough to know they spoke of trade routes, medicinal plants, and...a settlement thriving in the depths of the Waikato.

Mara believes it's our chance for real connection, a lifeline beyond bartering with scattered settlements like ours. We yearn for knowledge sharing, perhaps even a mutual defense pact. We’re cautiously optimistic, but wary. Remember what happened with the "New Eden" folk three years back? Promises whispered in the wind turned to ashes.

We'll leave a reply in the same box, marked with our symbol – the intertwined fern fronds. Pray for safe passage, Tane. May the winds guide your wings.

Elara

---

Drop Box Dispatch #48: Aotearoa, Year 30 (Waitomo Caves)

From: Tane (Waka Hauora)

To: Elara (Kaitiaki Collective)

Elara,

Your message found me tending to a patient suffering from sunstroke, a harsh reminder of the unforgiving world we inhabit. The journals you discovered belonged to my grandfather, a cartographer who charted this fractured land before… before everything changed. His hope for connection echoes in your words, a melody long silenced.

Waitomo is indeed a haven, carved into the heart of the caves where light filters through ancient formations like celestial tears. We've built a sustainable society here, reliant on geothermal energy, hydroponics, and traditional Maori knowledge. We’re not "New Eden," Elara. Our doors are open, but guarded by pragmatism, not naive idealism.

Your offer of trade interests us deeply. We possess medicinal herbs and fungi unseen outside our caverns, while your coastal access to shellfish and seaweed is invaluable. A formal pact, as you suggest, could be mutually beneficial. However, before full integration, we require a delegation from your collective to visit Waitomo. This ensures transparency and builds trust – a foundation shattered too often in this broken world.

Prepare yourselves for the journey south. Our people await your arrival at the "Glowworm Grotto," marked by a single white feather placed atop a limestone spire. May the ancestors guide your path.

Tane

---

Drop Box Dispatch #51: Aotearoa, Year 30 (Waitomo Caves)

From: Hina (Waka Hauora) - Healer

To: Elara (Kaitiaki Collective)

Elara,

A shadow has fallen over our glowworm haven. Your delegation arrived safely, but their visit... it did not go as expected. I write this with a heavy heart, for deceit festers within the "prosperity" promised by Tane. His words were honeyed, his smiles warm, yet beneath lay a cold calculation.

They speak of trade, but demand total subservience. Our knowledge, our skills – they want it all, packaged as “collaboration.” They offer scraps in return, barely enough to sustain your people. Their true goal? To exploit the resources of Piha and the surrounding coast, leaving your collective hollowed out like an ancient shell.

We, the healers and elders within Waka Hauora, saw through their facade. But Tane, blinded by ambition, dismissed our warnings. He sealed his fate, and ours, with a decree: “Integration or annihilation.” They've turned on us, Elara. Our sanctuary is besieged, our people hunted.

This message is a beacon, a plea for survival. Flee, Elara, warn your collective. Do not trust their promises. Remember the white feather atop the limestone spire – it now marks not welcome, but a trap. May the winds carry your escape, and guide you to safety.

Hina

---

Drop Box Dispatch #53: Aotearoa, Year 30 (Piha Beach)

From: Elara (Kaitiaki Collective) - Sole Survivor

To: ... Anyone who reads this

Hina's message reached us just as their scouts appeared on the horizon. We fled, abandoning our cherished home under a sky choked with smoke from burning pyres. Tane’s betrayal burned brighter than any fire.

We ran south, driven by desperation and Hina's desperate words. Only I remain, separated from my people in the chaotic scramble. The sea wind whispers their names, a chorus of loss echoing through my shattered heart.

I carry Hina's final warning like a talisman: Beware the white feather, a symbol twisted into a harbinger of death. Let this message be a testament to Waka Hauora’s fall, and a stark reminder that even in this fractured world, darkness can bloom where trust was sown.

May this find survivors, may it forge a path towards a future where whispers of hope are not drowned out by the screams of the betrayed.

Elara.

---

The metal box creaked shut, left on a wind-battered outcrop overlooking a desolate beach. Elara stared out at the churning sea, clutching a handful of seeds – remnants of a life lost, and a fragile promise for one that might yet be.

---

<h2>Special Thanks:</h2>

---

Special thanks to all the following, and many more...

All the model makers, fine tuners, mergers, and tweakers:

• Provides the raw "DNA" for almost all my models.
• Sources of model(s) can be found on the repo pages, especially the "source" repos with link(s) to the model creator(s).

Huggingface [ https://huggingface.co ] :

• The place to store, merge, and tune models endlessly.
• THE reason we have an open source community.

LlamaCPP [ https://github.com/ggml-org/llama.cpp ] :

• The ability to compress and run models on GPU(s), CPU(s) and almost all devices.
• Imatrix, Quantization, and other tools to tune the quants and the models.
• Llama-Server : A cli based direct interface to run GGUF models.
• The only tool I use to quant models.

Quant-Masters: Team Mradermacher, Bartowski, and many others:

• Quant models day and night for us all to use.
• They are the lifeblood of open source access.

MergeKit [ https://github.com/arcee-ai/mergekit ] :

• The universal online/offline tool to merge models together and forge something new.
• Over 20 methods to almost instantly merge model, pull them apart and put them together again.
• The tool I have used to create over 1500 models.

Lmstudio [ https://lmstudio.ai/ ] :

• The go to tool to test and run models in GGUF format.
• The Tool I use to test/refine and evaluate new models.
• LMStudio forum on discord; endless info and community for open source.

Text Generation Webui // KolboldCPP // SillyTavern:

• Excellent tools to run GGUF models with - [ https://github.com/oobabooga/text-generation-webui ] [ https://github.com/LostRuins/koboldcpp ] .
• Sillytavern [ https://github.com/SillyTavern/SillyTavern ] can be used with LMSTudio [ https://lmstudio.ai/ ] , TextGen [ https://github.com/oobabooga/text-generation-webui ], Kolboldcpp [ https://github.com/LostRuins/koboldcpp ], Llama-Server [part of LLAMAcpp] as a off the scale front end control system and interface to work with models.