StoryGen-12B-It

Model Card for StoryGen-12B-It

StoryGen-12B-It is a instruction-tuned narrative model created by integrating a specialized SFT LoRA adapter into the StoryGen-12B CPT checkpoint. It combines deep stylistic adaptation from continued pre-training with precise ChatML instruction following from supervised fine-tuning, resulting in a standalone model that generates direct, hook-driven, first-person stories without requiring external adapters or complex prompting scaffolds.

Model Details

Model Description

This model is the final production-ready artifact of the StoryGen storytelling pipeline. The CPT base internalized narrative structure, tone, and pacing at the token-distribution level; the SFT adapter then taught it to reliably activate that style within ChatML conversations. After merging, the LoRA weights are permanently folded into the base parameters, eliminating inference overhead and ensuring consistent behavior across all deployment environments. The model responds to story prompts with immediate shocking hooks, rhythmic prose, embedded unquoted dialogue, and zero AI-style preamble — matching the viral YouTube Shorts storytelling format it was trained on.

• Developed by: marekoth
• Model type: Merged Causal Language Model (CPT + SFT)
• Language(s): English
• License: Gemma Terms of Use
• Merged from: turtle0001/StoryGen-12B + SFT LoRA Adapter

Model Sources

• Repository: https://huggingface.co/turtle0001/StoryGen-12B-It
• Base Model (CPT): turtle0001/StoryGen-12B

Uses

Direct Use

Designed for creative storytelling generation via standard chat interfaces. Accepts user prompts and produces complete, formatted narratives in the target viral storytelling style. No adapter loading or special inference configuration required.

Recommended System Prompt:

You are a world-class YouTube Shorts scriptwriter. Write in direct, punchy, conversational language. No purple prose, no flowery adjectives, no melodramatic narration. Write like a real person telling a story to a friend. Start the story IMMEDIATELY without preamble or confirmation.

Prompting Guide

This model generates YouTube Shorts scripts and expects prompts structured like video topic briefs, not open-ended conversation. The optimal prompt format is:

write a story about [character/person] + [specific event/situation] + [optional detail/twist]

✅ Working Prompts

These follow the concrete character + event + detail structure and reliably produce high-quality outputs:

• write a story about comatose twin brother morse code
• write a story about undercover cop saves hijacked plane
• write a story about a father who finds his son's old voicemails after he passes away
• write a story about an employee who gets fired for stealing but the real thief turns out to be the CEO
• write a story about finding a hidden room in my new apartment
• write a story about my neighbor kept destroying our snowmen so I made the next one fight back

Common characteristics of effective prompts:

• 5–15 words
• Concrete character or narrator (not abstract concepts)
• Specific event or conflict (not vague actions)
• Twist or unexpected element included when relevant
• Structured like a video title or logline

❌ Non-Working Prompts

These fail because they lack the concrete narrative scaffolding the model was trained on:

• write a story about tuesday — abstract, no character, no event
• write a story about coworker helping — no context, no conflict, no outcome
• can you write me a story — zero narrative information provided
• tell me something interesting — completely open-ended, no structure
• write a story about love — abstract theme, no specific scenario

The model was fine-tuned exclusively on structured story transcripts paired with topic-brief-style prompts. It does not generalize well to conversational requests, thematic abstractions, or underspecified scenarios. Treat every prompt as a video topic assignment rather than a casual request.

Downstream Use

Suitable for integration into content creation platforms, AI-assisted scriptwriting tools, interactive fiction applications, and any system requiring consistent non-AI-sounding narrative generation. Can be further quantized (GGUF, AWQ, GPTQ) for edge deployment without significant quality loss.

Out-of-Scope Use

• ❌ General-purpose chatbot or assistant tasks (no broad conversational training)
• ❌ Factual Q&A, reasoning, coding, or analytical tasks
• ❌ Formal, academic, or professional business writing
• ❌ Safety-critical deployments (no RLHF or safety alignment applied)
• ❌ Multi-turn conversation outside storytelling context
• ❌ Content generation for children (training data contains mature themes)
• ❌ Vague, abstract, or underspecified prompts lacking narrative scaffolding

Bias, Risks, and Limitations

This model inherits all biases and limitations of Gemma 4 12B. Training data consists exclusively of fictional narrative transcripts; no factual grounding, safety alignment, or demographic balancing was performed. Generated content may reflect themes, tones, or perspectives present in the source corpus without moderation. The model has not been evaluated for harmful content generation, factual accuracy, or bias across populations. Conversational mode collapse may occur on non-story inputs due to narrow SFT data coverage — the model may produce incoherent, overly literal, or identity-confused responses when asked questions outside the storytelling domain. Users deploying this model are responsible for implementing appropriate safety measures, content filtering, and human review protocols before public or commercial use.

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "turtle0001/StoryGen-12B-It"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

messages = [
    {"role": "system", "content": "You are a world-class YouTube Shorts scriptwriter. Write in direct, punchy, conversational language. No purple prose, no flowery adjectives, no melodramatic narration. Write like a real person telling a story to a friend. Start the story IMMEDIATELY without preamble or confirmation."},
    {"role": "user", "content": "write a story about a father who finds his son's old voicemails after he passes away"}
]

input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
output = model.generate(input_ids, max_new_tokens=1024, temperature=0.8, do_sample=True)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training & Merge Details

Pipeline Overview

CPT Phase Summary

Adapted Gemma 4 12B on ~40K curated long-form English narrative transcripts (>600 words each). Trained for 500 steps at LR 5e-5 (embeddings 1e-5) with AdamW 8-bit, packing enabled, max_seq_length 4096. Modules saved: embed_tokens, lm_head (full weights). Final loss stabilized at 1.7–2.2. Hardware: NVIDIA RTX PRO 6000, 4 hours.

SFT Phase Summary

Fine-tuned LoRA adapter (r=32, alpha=32) on 536 ChatML-formatted story transcripts for 5 epochs (170 steps) at LR 5e-5 with full sequence loss. Target modules: q/k/v/o_proj, gate/up/down_proj. Trainable parameters: 67.8M (0.56%). Final loss: 1.571. Critical design decision: response-only masking initially produced a word-completion engine; switching to full sequence loss enabled proper ChatML format learning while preserving CPT style.

Merge Procedure

The SFT LoRA adapter was merged into the CPT base model using Unsloth's save_pretrained_merged with merged_16bit method. All LoRA delta weights were permanently added to base parameters in bfloat16 precision. No quantization was applied during merge. The resulting checkpoint is functionally identical to loading the base + adapter at inference time but requires no PEFT dependency.

Environmental Impact

Carbon emissions estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Actual emissions depend on local energy grid carbon intensity. Estimates assume average global carbon efficiency. No carbon offset information is available for these training runs.

Technical Specifications

Model Architecture and Objective

• Architecture: Gemma 4 Unified (12B parameters)
• Objective: Causal Language Modeling (merged CPT + SFT)
• Context Length: 4096 tokens (training), up to 262K supported by architecture
• Vocabulary: 262,144 tokens
• Precision: bfloat16
• Adapter Status: Fully merged, no external dependencies

Compute Infrastructure

Hardware

• CPT: 1× NVIDIA RTX PRO 6000
• SFT: 1× NVIDIA A100-SXM4-40GB
• Training accelerated with Unsloth 2× faster finetuning

Software

• Transformers ≥4.46.0
• PEFT 0.19.1
• Unsloth 2026.7.2
• TRL ≥0.12.0
• PyTorch 2.x (bf16)
• AdamW 8-bit optimizer

Citation

If you use this model, please cite the base model and the carbon emissions methodology:

BibTeX:

@misc{storygen-12b-it,
  title={StoryGen-12B-It: A Fully Integrated Narrative Storytelling Model via CPT and SFT Merge},
  author={turtle0001},
  year={2026},
  url={https://huggingface.co/turtle0001/StoryGen-12B-It}
}

@article{lacoste2019quantifying,
  title={Quantifying the Carbon Emissions of Machine Learning},
  author={Lacoste, Alexandre and Luccioni, Alexandra and Schmidt, Victor and Dandres, Thomas},
  journal={arXiv preprint arXiv:1910.09700},
  year={2019}
}

Framework Versions

• Unsloth: 2026.7.2
• PEFT: 0.19.1
• Transformers: ≥4.46.0
• TRL: ≥0.12.0

Model Card Authors

marekoth