Maithili_Poem_Generator

MaithiliNano — Maithili Poem Generator (Variant A)

A decoder-only Small Language Model (SLM) that generates Maithili poetry. It was pre-trained from scratch on a 45.86M-token Maithili corpus, then fine-tuned on curated Maithili poems. Part of the MaiGen project.

Model

• Architecture: Decoder-only Transformer (RoPE, RMSNorm, SwiGLU, tied embeddings)
• Parameters: ~29.3M
• Layers / Heads / Dim: 8 / 8 / 512
• Vocab / Context: 8,000 Byte-Level BPE tokens / 512
• Checkpoint: epoch 10, step 1200, val loss ~0.70

Files

• model.py — model class + from_pretrained()
• model.safetensors + config.json — weights and config (auto-downloaded)
• vocab.json + merges.txt — ByteLevelBPE tokenizer (auto-downloaded)
• poem_finetune.pt — original PyTorch checkpoint (model_state, epoch, step, val_loss)

Quickstart

pip install torch tokenizers safetensors huggingface_hub

Put model.py in your working folder, then run:

import torch
from model import MaithiliNano

# Downloads weights, config, vocab + merges straight from HF
model = MaithiliNano.from_pretrained("kamal-018/Maithili_Poem_Generator")
model.eval()

# Tokenizer is auto-downloaded into the same cache folder
tok = model.tokenizer()


def generate(prompt, max_len=150, temperature=0.6, top_k=20):
    ids = [tok.token_to_id("<s>")] + tok.encode(prompt).ids
    idx = torch.tensor([ids])
    out = model.generate(idx, max_new_tokens=max_len,
                         temperature=temperature, top_k=top_k)
    return tok.decode(out[0].tolist())


print(generate("प्रेम"))

-This outputs a Maithili poem about love (प्रेम).
प्रेम आ भाव प्रेम भाव ऊँचे,
मानव के सजय छी हम संग।
जंगली घास के मैदावाला दिस बढ़लै
विचार हमर त्राण परिचालिनी,
लावारपाटी में हम मतिमे सहायता
अयली माता काली माय सेहो श्रेष्ठ
हम लोकक सासुर फुजल छै रहब

Notes

• The prompt is auto-prepended with <s> (BOS). Generation uses temperature 0.6, top-k 20.
• Uses a custom from_pretrained (this repo's model.py) — it is not a transformers.AutoModel. You do not need transformers installed.
• Full model definition and training code are in the project repo (Maithili_Poem_Generator).