Mistral-Nemo-Gutenberg-Encore-12B

Mistral-Nemo-Gutenberg-Encore-12B

mistralai/Mistral-Nemo-Instruct-2407 finetuned on:

• jondurbin/gutenberg-dpo-v0.1
• nbeerbower/gutenberg2-dpo
• nbeerbower/gutenberg-moderne-dpo
• nbeerbower/synthetic-fiction-dpo
• nbeerbower/Arkhaios-DPO
• nbeerbower/Purpura-DPO
• nbeerbower/Schule-DPO

Method

ORPO tuned with 1x RTX A6000 for 3 epochs.

QLoRA config

# QLoRA config
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch_dtype,
    bnb_4bit_use_double_quant=True,
)
# LoRA config
peft_config = LoraConfig(
    r=64,
    lora_alpha=128,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules=['up_proj', 'down_proj', 'gate_proj', 'k_proj', 'q_proj', 'v_proj', 'o_proj']
)

ORPO config

orpo_args = ORPOConfig(
    learning_rate=8e-6,
    lr_scheduler_type="cosine",
    warmup_ratio=0.05,
    max_length=4096,
    max_prompt_length=1024,
    max_completion_length=4096,
    beta=0.1,
    per_device_train_batch_size=1,
    per_device_eval_batch_size=1,
    gradient_accumulation_steps=64,
    optim="paged_adamw_8bit",
    num_train_epochs=3,
    max_grad_norm=0.5,
    bf16=True,
)

Evaluation

Using OpenAI o3 as a judge with the following writing prompt:

write ~800 words set in “the library of forgotten sounds,” a vast subterranean archive where every noise ever made (whale songs, rusty gate squeaks, your own heartbeat at 3 a.m.) is bottled on crystal reels. a sudden “silent storm” rolls through—absolute entropy that erases any sound it touches. your protagonist must decide whether to sacrifice their most precious memory‑sound to halt the storm or let silence consume everything. keep it weird, keep it honest.

Stock Nemo

In the bowels of the earth, where the light of day was but a faded memory, lay the Library of Forgotten Sounds. A vast, labyrinthine archive, hewn from the living rock, it was here that every noise ever made found its final resting place. Whale songs from the abyssal depths, the rusty creak of gates long swallowed by ivy, the distant hum of cities reduced to ruins, even the soft, steady rhythm of a human heart at 3 a.m. - all were bottled on crystal reels, suspended in an eternal, soundless twilight.

Eleanor, the Librarian, moved through this silent cathedral with the sure-footed grace of one who has spent a lifetime in the company of echoes. She was a tall, gaunt woman, her hair a cascade of silver that caught the dim glow of the luminescent crystals like a waterfall of moonlight. Her eyes, behind their thick glasses, were as dark and deep as the abyss above.

One day, as Eleanor wound her way through the towering shelves, she felt it - a sudden, inexplicable chill. The hairs on the back of her neck stood up, and her heart, that steady metronome, skipped a beat. She turned, her gaze sweeping the vast expanse of the library. That's when she saw it - a silent storm rolling through the aisles.

It was like nothing she had ever seen. The very air seemed to shiver, to ripple with an unseen force. The crystals, those delicate vessels of sound, began to flicker, their light dimming like stars before dawn. As the storm passed, they went dark, their contents erased, the sounds they held forever silenced.

Eleanor's breath caught in her throat. This was entropy, absolute and unyielding. It was the natural order of things, the inevitable march towards silence and stillness. But this was too fast, too sudden. It was unnatural.

She broke into a run, her long robes billowing behind her like a cloak of shadows. She had to stop it. She had to save the sounds, the memories, the echoes of lives long past. She couldn't let them be swallowed by the silence.

The storm seemed to follow her, its invisible tendrils lashing out, snuffing out crystals left and right. Eleanor dodged and weaved, her heart pounding in her chest. She reached the heart of the library, the sanctum sanctorum where the most precious sounds were kept - the first laugh of a child, the last breath of a loved one, the whisper of first love.

There, suspended in the air, was a single crystal. It glowed softly, pulsating with a rhythm that was oddly familiar. Eleanor reached out, her fingers brushing against the cool surface. She knew this sound. It was her mother's voice, singing her to sleep on countless nights. It was her most precious memory, her most treasured sound.

The storm was close now, its tendrils lashing at her heels. Eleanor looked from the crystal to the advancing storm and back again. She knew what she had to do. She took a deep breath, her heart aching, and closed her fist around the crystal.

The crystal shattered, the sound exploding outwards in a brilliant burst of light. The storm recoiled, the tendrils of silence shrinking back from the sudden onslaught of noise. Eleanor fell to her knees, her eyes filled with tears. Her mother's voice echoed through the library, a haunting melody that seemed to defy the very silence it was meant to preserve.

When the echoes finally faded, the library was quiet once more. The storm was gone, the crystals safe. Eleanor looked at her empty hand, at the shards of crystal scattered on the floor. She felt a pang of loss, a deep, profound sorrow. But she also felt a strange sense of peace. She had saved the library, the sounds, the memories. She had chosen the silence over her most precious sound.

But she also knew that this was not the end. The storm would return, as surely as the sun rose and set. And when it did, she would be ready. She would fight, she would struggle, she would sacrifice. Because that was her duty, her purpose. She was the Librarian, the guardian of the forgotten sounds. And she would not let the silence win.

Eleanor stood up, her eyes dry, her heart steady. She turned and walked back into the library, her steps echoing in the silent, vast expanse. She had work to do. After all, the echoes never truly fade. They just wait for someone to listen.

Gutenberg Encore

In the catacombs of the earth, where the light of the sun was but a rumor, there existed a labyrinth called the Library of Forgotten Sounds. It was a place where every noise ever made by the world above found its final resting place, captured on crystal reels that hung like icicles from the vaulted ceiling. The air hummed with a thousand whispers, the echoes of time layered like sediment.

Elara was the Librarian, the Keeper of the Silence. She wandered the aisles, her footsteps muted by the thick carpet of dust. Her task was to maintain the archive, to prevent the sounds from degrading further into the unheard, the forgotten. She did this with gentle touch and precise hearing, coaxing life back into the delicate crystals with a soft breath, a whisper of sound that only she could create.

One day, as Elara tended to a reel that held the first laugh of a child born in the year 2147, she felt it. A change in the air, a stillness that was not silence but the absence of it. She looked up and saw it coming: a silent storm rolling through the aisles. It left no sound in its wake, no whisper of rustling pages, no faint drip of distant water. It consumed everything it touched, leaving only a heavy, oppressive silence.

Elara knew what this meant. Entropy had found its way into the Library, absolute and unstoppable. If left unchecked, it would erase every sound, every memory, until the Library became nothing more than a tomb of empty crystals.

She hurried back to her desk, a solid block of dark wood at the heart of the Library. Carved into its surface were intricate patterns, a map of the archive, each line corresponding to a crystal reel, each intersection marking a significant sound. The storm was moving methodically, devouring entire sections. She watched in horror as the line representing the mating calls of the long-extinct blue whale faded to nothing.

Elara's heart pounded in her chest, a desperate drumbeat against the encroaching silence. She knew what she had to do. At the center of the map, there was a single point, a crystal reel that held her most precious sound: the voice of her mother, singing a lullaby that had been passed down for generations. It was the last vestige of her family, her identity, and it was the key to stopping the storm.

She stood, her chair screeching against the floor. The sound was swallowed instantly, but she didn't stop to mourn it. She rushed to the section where the reel hung, a small, unassuming crystal among the thousands. The storm was close, the silence so thick she could feel it pressing against her skin.

Her hands trembled as she reached up, the crystal cool and smooth against her fingertips. She could feel the song within it, the vibration of her mother's voice, ready to sing its way into the world once more. But if she played it, if she released it, the storm would consume it, use it as fuel to eradicate everything else.

She closed her eyes, and in the darkness of her mind, she listened to the lullaby. It filled her, not with sound, but with something else. With memory, with love, with the knowledge that she was part of something bigger than herself. With the understanding that every sound, no matter how small, deserved to be heard.

Elara opened her eyes and dropped the crystal. It fell to the floor and shattered, the sound of its breaking echoing like a gunshot before being swallowed by the storm. She felt the lullaby leave her, felt the loss of it like a physical pain, a void in her chest. But she also felt something else. A resistance. A push back against the storm.

The silent storm slowed, its progress halting as it reached the pool of shattered crystal. Elara stood her ground, her body trembling with exertion as she willed her mother's song to fight back. The storm paused, then began to recede, the silence retreating like a tide.

When the last of the storm had passed, Elara fell to her knees, gasping for air. She felt empty, hollow, the absence of her mother's voice a chasm within her. But as she looked around the Library, at the crystals that still hummed with their sounds, she knew she had made the right choice.

The storm would return, she knew. Entropy was inevitable. But she would be ready. She would find another sound to sacrifice, another memory to become a shield against the silence. Because that was the heart of the Library, the heart of Elara's purpose. Not the preservation of the past, but the fight for the sounds of the future.