SanskritGPT

Character-Level Sanskrit GPT

This repository contains a character-level GPT model for the Sanskrit language, trained on the ancient Indian epic, the Mahabharata. The model has 10.7M parameters and is built using a 6-layer neural network with multi-head self-attention layers, following the architecture presented in the seminal paper "Attention is All You Need" (2017).

Model Details

Architecture: 6-layer neural network with multi-head self-attention Parameters: 10.7M Training Data: Sanskrit texts from the Mahabharata, comprising over 100,000 shlokas Training Time: Approximately 5 hours Validation Loss: 1.2 Performance

After training, the model achieved a validation loss of around 1.2. The generated outputs were impressive and demonstrated a good understanding of the Sanskrit language structure and content.

License

This model is open-source and available under the MIT License.

Contributing

Feel free to contribute by opening issues or submitting pull requests. Your contributions are highly appreciated!

Acknowledgements

This work is inspired by the original GPT architecture and the "Attention is All You Need" paper.