Sienna-v1-0.5B

<p align="center"> <img src="Sienna.png" alt="Sienna Logo" width="1200"/> </p>

Sienna v1

Sienna is a children's story generation model that creates age-appropriate stories for babies and toddlers across four genres:

• Bedtime & Lullaby — Gentle, calming stories to help little ones fall asleep
• Animal Adventures — Fun stories featuring animals toddlers love
• Friendship & Sharing — Stories that teach kindness, sharing, and social skills
• Fantasy & Magic — Magical fairy-tale adventures with simple concepts

Model Details

Quick Start

Requirements

pip install torch transformers peft

Inference

import torch
import json
from model import SiennaConfig, SiennaModel, load_sienna_weights
from generate import sample
from transformers import AutoTokenizer

# Load model
config = SiennaConfig(**json.load(open("config.json")))
model = SiennaModel(config)
load_sienna_weights(model, hf_model_name=config.hf_model_name)
model.apply_lora()

state = torch.load("model.pt", map_location="cpu")
model.load_state_dict(state, strict=False)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
model.eval()

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("tokenizer")

# Generate a story
instruction = "Write a gentle bedtime story about a sleepy bunny"
prompt = f"<instruction>{instruction}</instruction><story>"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device)

output_ids = sample(
    model,
    input_ids,
    max_new_tokens=512,
    temperature=0.8,
    top_k=50,
    top_p=0.9,
    repetition_penalty=1.15,
    eos_token_id=tokenizer.eos_token_id,
)

story = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)
print(story)

Command-Line Generation

# Generate with a specific genre
python generate.py --model . --genre bedtime --max_new_tokens 512

# Generate with a custom instruction
python generate.py --model . --instruction_text "Write a story about a cat who makes new friends" --max_new_tokens 512

# Adjust generation parameters
python generate.py --model . --genre fantasy --temperature 0.9 --top_k 40 --top_p 0.92 --max_new_tokens 1024

Available Genres

Generation Parameters

File Structure

├── README.md           # This file
├── config.json         # Model configuration
├── model.pt            # Merged model weights
├── model.py            # Model architecture definition
├── generate.py         # Inference/generation script
├── genres.py           # Genre definitions
└── tokenizer/          # Tokenizer files
    ├── config.json
    ├── tokenizer.json
    ├── tokenizer_config.json
    └── chat_template.jinja

License

Apache 2.0