StoryGen-12B-It-GGUF

Model Card for StoryGen-12B-It-GGUF

StoryGen-12B-It-GGUF is a quantized version of turtle0001/StoryGen-12B-It, converted to GGUF format for efficient local inference via llama.cpp, LM Studio, Ollama, and other compatible backends. It retains the direct, hook-driven, first-person narrative style of the original merged model while enabling deployment on consumer hardware with reduced VRAM/RAM requirements.

Model Details

Model Description

This GGUF checkpoint is a post-training quantization of the StoryGen-12B-It merged model (CPT + SFT). The source model combines continued pre-training on long-form narrative transcripts with supervised fine-tuning on ChatML-formatted story prompts, producing a standalone instruction-tuned storyteller. Quantization preserves the core narrative capabilities — immediate shocking hooks, rhythmic prose, embedded unquoted dialogue, and zero AI-style preamble — while making the model accessible on devices with limited compute resources.

• Developed by: marekoth
• Quantized by: turtle0001
• Model type: Quantized Causal Language Model (GGUF)
• Language(s): English
• License: Gemma Terms of Use
• Base model: turtle0001/StoryGen-12B-It
• Original base: google/gemma-4-12B

Model Sources

• Repository: https://huggingface.co/turtle0001/StoryGen-12B-It-GGUF
• Source Model (bf16): turtle0001/StoryGen-12B-It
• Quantization Tool: llama.cpp (quantize)

Uses

Direct Use

Designed for local creative storytelling generation via GGUF-compatible inference engines. Accepts user prompts and produces complete, formatted narratives in the marekoth style. No adapter loading or external dependencies required.

Recommended System Prompt:

You are marekoth, a world-class YouTube Shorts scriptwriter. Write in direct, punchy, conversational language. No purple prose, no flowery adjectives, no melodramatic narration. Write like a real person telling a story to a friend. Start the story IMMEDIATELY without preamble or confirmation.

Prompting Guide

This model generates YouTube Shorts scripts and expects prompts structured like video topic briefs, not open-ended conversation. The optimal prompt format is:

write a story about [character/person] + [specific event/situation] + [optional detail/twist]

✅ Working Prompts

• write a story about comatose twin brother morse code
• write a story about undercover cop saves hijacked plane
• write a story about a father who finds his son's old voicemails after he passes away
• write a story about an employee who gets fired for stealing but the real thief turns out to be the CEO
• write a story about finding a hidden room in my new apartment

Effective prompt characteristics: 5–15 words, concrete character/narrator, specific event/conflict, twist or unexpected element when relevant, structured like a video title or logline.

❌ Non-Working Prompts

• write a story about tuesday — abstract, no character, no event
• write a story about coworker helping — no context, no conflict, no outcome
• can you write me a story — zero narrative information provided
• tell me something interesting — completely open-ended, no structure

Treat every prompt as a video topic assignment rather than a casual request. The model does not generalize well to conversational requests, thematic abstractions, or underspecified scenarios.

Downstream Use

Suitable for integration into local content creation workflows, offline scriptwriting tools, interactive fiction applications, and edge deployment scenarios where cloud inference is unavailable or undesirable. Multiple quantization levels are provided to balance quality and resource constraints.

Out-of-Scope Use

• ❌ General-purpose chatbot or assistant tasks (no broad conversational training)
• ❌ Factual Q&A, reasoning, coding, or analytical tasks
• ❌ Formal, academic, or professional business writing
• ❌ Safety-critical deployments (no RLHF or safety alignment applied)
• ❌ Multi-turn conversation outside storytelling context
• ❌ Content generation for children (training data contains mature themes)
• ❌ Vague, abstract, or underspecified prompts lacking narrative scaffolding

Bias, Risks, and Limitations

This model inherits all biases and limitations of Gemma 4 12B and the source StoryGen-12B-It model. Training data consists exclusively of fictional narrative transcripts; no factual grounding, safety alignment, or demographic balancing was performed. Generated content may reflect themes, tones, or perspectives present in the source corpus without moderation. Conversational mode collapse may occur on non-story inputs due to narrow SFT data coverage. Quantization introduces additional quality degradation, particularly at lower bit levels (Q4_K_S, Q3_K_M), which may manifest as reduced narrative coherence, weaker hook generation, or increased repetition. Users deploying this model are responsible for implementing appropriate safety measures, content filtering, and human review protocols before public or commercial use.

How to Get Started with the Model

llama.cpp

./llama-cli -m StoryGen-12B-It-Q5_K_M.gguf \
  --prompt "<|turn>system\nYou are marekoth, a world-class YouTube Shorts scriptwriter. Write in direct, punchy, conversational language. No purple prose, no flowery adjectives, no melodramatic narration. Write like a real person telling a story to a friend. Start the story IMMEDIATELY without preamble or confirmation.<turn|>\n<|turn>user\nwrite a story about finding a hidden room in my new apartment<turn|>\n<|turn>model\n" \
  -n 1024 --temp 0.8 -ngl 99

LM Studio / Ollama

Load the GGUF file directly. Set the system prompt to the recommended marekoth prompt above. Use temperature 0.8–1.2 for creative variation. Ensure the chat template is set to Gemma 4 / ChatML format.

Quantization Procedure

Converted from bf16 safetensors using llama.cpp convert_hf_to_gguf.py followed by quantize with default importance-based mixed quantization (K-quants). No imatrix calibration was applied. Context length preserved at 4096 tokens.

Environmental Impact

Carbon emissions estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Actual emissions depend on local energy grid carbon intensity. Estimates assume average global carbon efficiency. No carbon offset information is available for these runs.

Technical Specifications

Model Architecture and Objective

• Architecture: Gemma 4 Unified (12B parameters)
• Objective: Causal Language Modeling (merged CPT + SFT, quantized)
• Context Length: 4096 tokens
• Vocabulary: 262,144 tokens
• Format: GGUF (llama.cpp compatible)
• Source Precision: bfloat16

Compute Infrastructure

Hardware

• CPT: 1× NVIDIA RTX PRO 6000
• SFT: 1× NVIDIA A100-SXM4-40GB
• Quantization: CPU (negligible emissions)

Software

• llama.cpp (latest master)
• Transformers ≥4.46.0
• Unsloth 2026.7.2
• PEFT 0.19.1
• TRL ≥0.12.0

Citation

If you use this model, please cite the source model and the carbon emissions methodology:

BibTeX:

@misc{storygen-12b-it-gguf,
  title={StoryGen-12B-It-GGUF: Quantized Narrative Storytelling Model for Local Inference},
  author={marekoth},
  year={2026},
  url={https://huggingface.co/turtle0001/StoryGen-12B-It-GGUF}
}

@article{lacoste2019quantifying,
  title={Quantifying the Carbon Emissions of Machine Learning},
  author={Lacoste, Alexandre and Luccioni, Alexandra and Schmidt, Victor and Dandres, Thomas},
  journal={arXiv preprint arXiv:1910.09700},
  year={2019}
}

Framework Versions

• llama.cpp: latest master
• Unsloth: 2026.7.2
• PEFT: 0.19.1
• Transformers: ≥4.46.0
• TRL: ≥0.12.0

Model Card Authors

marekoth