mechanicus-prayer-gpt2

Mechanicus Prayer Generator GPT-2

A 124M parameter transformer model trained on Mechanicus-style prayers using the GPT-2 architecture.

Model Description

This model generates thematic prayers, blessings, and activation rites for various machines and equipment in the Mechanicus style. It produces natural prose with appropriate Mechanicus flavor (references to the Omnissiah, Machine Spirits, sacred oils, litanies, etc.).

Architecture: GPT-2 (124M parameters, trained from scratch)
Training Data: 3,972 Grok-generated examples (dataset)
Tokenization: Trained without bias terms, uses tiktoken encoding

Performance Metrics

• Object Consistency: 86% (correctly references the machine/component from prompt)
• Format Quality: 100% (natural prose, no step labels)
• Overall Score: 93%

Quick Start

Installation

pip install transformers torch tiktoken

Basic Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained(
    "merileijona/mechanicus-prayer-gpt2",
    trust_remote_code=True  # Required for custom tiktoken tokenizer
)
model = AutoModelForCausalLM.from_pretrained("merileijona/mechanicus-prayer-gpt2")
model.eval()

# Format prompt
prompt = "<|user|>Prayer for activating a plasma reactor.<|end|>\n<|assistant|>"

# Generate
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(
        inputs["input_ids"],
        max_new_tokens=256,
        temperature=0.65,  # Lower = more consistent
        top_k=30,
        do_sample=True,
    )

# Decode
text = tokenizer.decode(outputs[0], skip_special_tokens=False)
prayer = text.split("<|assistant|>")[1].split("<|end|>")[0].strip()
print(prayer)

Example Output

Input: "Prayer for activating a plasma reactor."

Generated:

Encircle the plasma reactor, intoning the Litany of Containment.
Beseech the plasma reactor's volatile Machine Spirit to accept charging.
Infuse plasma conduits with sanctified isotopes into the plasma reactor's core.
Chant the Incantation of Contained Fury during the charging ritual.
Seal the plasma reactor's charged plasma with a final benediction.

Prompt Format

<|user|>[Your request]<|end|>
<|assistant|>

Example prompts:

• "Prayer for activating a [device]"
• "Blessing for a [weapon/tool]"
• "Emergency prayer for a [system]"
• "Litany for [action/sequence]"
• "Rite of [operation] for a [component]"

Recommended Parameters

• temperature: 0.65 (lower than default 1.0 for better consistency)
• top_k: 30 (lower than default 50)
• max_new_tokens: 256

Higher temperatures may produce less coherent results.

Technical Details

Custom Tokenizer

This model uses a custom tiktoken-based tokenizer wrapped in HuggingFace's interface. This ensures compatibility with the original training process while maintaining standard HuggingFace APIs.

Why? The model was trained using tiktoken (standard for nanoGPT). HuggingFace's default GPT2Tokenizer has subtle encoding differences that break generation quality.

The custom tokenizer loads automatically with trust_remote_code=True.

Architecture

• Base: GPT-2 (124M parameters)
• Modifications: No bias terms (common optimization)
• Vocabulary: 50,304 tokens (GPT-2 base + special tokens)
• Special tokens: <|user|> (50257), <|assistant|> (50258), <|end|> (50259)
• Context length: 512 tokens

Training

• Dataset: 3,972 synthetic examples generated by xAI Grok (Grok-4-1-reasoning)
• Quality: 100% format consistency, 0% duplicates, 100% natural prose
• Validation: Rigorous object consistency and format checking
• Framework: nanoGPT with tiktoken tokenization

Limitations

• Specialized for Mechanicus-style creative writing only
• Requires specific prompt format for best results (i.e., action + w40k object)
• Performance degrades with components not in training vocabulary
• May occasionally generate minor repetitions or inconsistencies

Ethical Considerations

✅ Appropriate uses:

• Creative writing and worldbuilding
• Educational exploration of language models
• Blessing of the Sacred Machine
• Appeasing the Machine Spirits

License & Copyright

Code & Model: Licensed under CC BY-NC-SA 4.0

Warhammer 40,000 IP: All Warhammer 40,000 content, including the Adeptus Mechanicus, Cult Mechanicus, and associated lore, names, characters, and imagery are ©, ®, and/or ™ Games Workshop Limited.

This is an unofficial, non-commercial fan project for educational and entertainment purposes. Not affiliated with, endorsed by, or sponsored by Games Workshop Limited.

Citation

@misc{mechanicus-prayer-gpt2-2026,
  author = {merileijona},
  title = {Mechanicus Prayer Generator GPT-2},
  year = {2026},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/merileijona/mechanicus-prayer-gpt2}}
}

Acknowledgments

• Base Model: OpenAI GPT-2
• Training Data: Generated by xAI Grok
• Framework: HuggingFace Transformers
• Tokenization: tiktoken
• Inspiration: Warhammer 40,000 by Games Workshop

---

From the weakness of the mind, Omnissiah save us.
From the lies of the Antipath, circuit preserve us.
From the rage of the Beast, iron protect us.
From the temptations of the Flesh, silica cleanse us.
From the ravages of the Destroyer, anima shield us.
From this rotting cage of biomatter, Machine God, set us free. ⚙️