Downloads · 30 days
0
mahanthisaikiran/srisrilm-bhajan-v1
srisrilm-bhajan-v1 is a machine learning model from mahanthisaikiran. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
A minimal educational GPT-style language model trained on Art of Living Bhajan content, satsangs, guru stotrams, devotional songs, and spiritual lyrics related to Gurudev Sri Sri Ravi Shankar.
Downloads · 30 days
0
Access
Public
Updated Jun 3, 2026
Repo size
—
Likes
0
Public
Click a slice to open those files.
.pdf4.1 MB · 26%
From the Hugging Face model README
A minimal educational GPT-style language model trained on Art of Living Bhajan content, satsangs, guru stotrams, devotional songs, and spiritual lyrics related to Gurudev Sri Sri Ravi Shankar.
This project builds a small decoder-only GPT architecture from scratch in PyTorch. It uses a Byte Pair Encoding (BPE) tokenizer and trains on text extracted from a Bhajan PDF.
data/bhajan_lyrics.pdf - source PDF containing bhajan lyricsdata/train.txt - extracted plain text used for tokenizer training and model trainingtokenizer/tokenizer.json - BPE tokenizer artifactcheckpoints/srisrilm.pt - saved PyTorch model weightsextract_text.py - PDF text extraction pipelinetrain_tokenizer.py - BPE tokenizer training scriptmodel.py - GPT decoder architecture implementationtrain.py - dataset, training loop, optimizer, scheduler, and checkpoint savinggenerate.py - inference script for prompt-based generationThe model is a small GPT-style decoder-only transformer with:
vocab_size = 5000embedding_dim = 128num_heads = 4num_layers = 4context_length = 128dropout = 0.1Implemented features:
pip install -r requirements.txt
Place your Bhajan PDF in data/bhajan_lyrics.pdf.
Extract the PDF text into data/train.txt:
python extract_text.py
python train_tokenizer.py
python train.py
The training script saves the model checkpoint to checkpoints/srisrilm.pt.
Run generation using the saved checkpoint and tokenizer:
python generate.py
The script will generate continuations for the prompts:
This code is intended for educational purposes, demonstrating how to build a small decoder-only transformer from scratch and train it on devotional bhajan text. The model is named SriSriLM-Bhajan-v1 and can be adapted for further fine-tuning or deployment.