smol_poet_35M

SmolPoet — A 35M Parameter Poetry Language Model

SmolPoet is a lightweight (~35M parameters) decoder-only transformer model designed for generating poetic text in a Shakespearean and classical style. It is trained on a large-scale poetry corpus and further refined using reinforcement learning to enhance stylistic structure and expressiveness.

---

Overview

SmolPoet demonstrates that relatively small language models can produce rich, stylistically aligned text when trained with a combination of large-scale pretraining and reinforcement learning.

The model specializes in:

• Poetic text generation
• Shakespearean-style language
• Structured, multi-line outputs
• Figurative and abstract expression

---

Architecture

SmolPoet is inspired by modern LLaMA-style architectures and incorporates several efficiency and performance improvements.

• Model Type: Decoder-only Transformer
• Parameters: ~35 million
• Embedding Dimension: 512
• Layers: 12
• Attention: Grouped Query Attention (GQA)
• Positional Encoding: Rotary Positional Embeddings (RoPE)
• Activation Function: SwiGLU
• Normalization: RMSNorm
• Weight Sharing: Input/output embedding tied

---

Training Data

The model is trained on a large corpus of poetry and literary text:

• ~1.35 billion characters
• ~225 million words
• ~300 million tokens

The dataset includes a mixture of classical poetry and Shakespearean-style language, enabling the model to learn:

• Rhythmic sentence structure
• Archaic vocabulary
• Figurative language patterns

---

Tokenization

A custom Byte Pair Encoding (BPE) tokenizer is used:

• Trained on a representative subset of the dataset
• Efficient for large-scale corpora
• Captures subword structure for poetic language

---

Training Methodology

Pretraining

The model is first trained using standard next-token prediction to learn general language patterns and poetic structure.

---

Reinforcement Learning (PPO)

After pretraining, the model is refined using Proximal Policy Optimization (PPO) to improve stylistic quality.

The reinforcement learning phase encourages:

• Multi-line poetic structure
• Richer stylistic expression
• More consistent tone and phrasing

A KL-divergence constraint is applied to ensure the model remains close to the pretrained distribution while improving style.

---

Example Outputs

Prompt:

"O gentle night"

Output:

O gentle night, And then the day melted to a night. I heard the performers reply, Who are on foot, I would say, But I my tongue supplied thy

---

Prompt:

"The starry night"

Output:

The starry night-hawk, stirring the light plunges, And at the wind the storm perchance the morrow. Therefore off, I will suffer more from my tongue

---

Observations

• Reinforcement learning improves poetic structure and stylistic consistency
• Outputs become more aligned with Shakespearean tone
• The model produces multi-line compositions more reliably after RL fine-tuning

---

Limitations

• Does not enforce strict poetic meter (e.g., iambic pentameter)
• Occasional repetition or incoherent phrasing
• Limited world knowledge due to model size
• May produce archaic or stylistically exaggerated language
• Dataset contains special characters and information from Project Gutenberg files, that werent cleaned

---

Intended Use

SmolPoet is suitable for:

• Creative writing assistance
• Poetry generation experiments
• Educational purposes (LLM + RL pipelines)
• Research on small-scale language models

---

Out-of-Scope Use

• Factual question answering
• High-accuracy reasoning tasks
• Production-grade language understanding

---

Key Takeaways

• Small models can achieve stylistic richness with proper training
• Reinforcement learning is effective for style alignment
• Qualitative improvements may not always correlate with reward metrics

---

Acknowledgements

This model is inspired by advancements in transformer architectures and reinforcement learning methods for language models.

---