Downloads · 30 days
37
7% of all-time downloads
Misal-7B-base-v0.1
smallstepai/Misal-7B-base-v0.1
Misal-7B-base-v0.1 is a text generation model from smallstepai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama2.
It is a language model based on Meta's Llama2 architecture, pretrained using Marathi Text Data.
All-time downloads
555
Public
Parameters
6.9B
68.6 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors13.7 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama2
- Model type
- llama
- Access
- Public
- Created
- Feb 19, 2024
- Updated
- Apr 22, 2024
- SHA
- 55b37d28
Model card
From the Hugging Face model README
Misal-7B-base-v0.1
It is a language model based on Meta's Llama2 architecture, pretrained using Marathi Text Data.
Built by - smallstep.ai
Making of Misal?
Detailed blog here.
Pretraining :
During the pretraining phase of our large language model, the model was exposed to a vast corpus of text data comprising approximately 2 billion Marathi tokens. This corpus primarily consisted of newspaper data spanning the years 2016 to 2022, sourced primarily from the CulturaX dataset. In addition to this, we supplemented our training data with additional sources such as l3cube, ai4bharat, and other internet-based datasets.
We chose bfloat16 as training precision due to stability issues with float16 precision.
License
The model inherits the license from meta-llama/Llama-2-7b.
Team
Sagar Sarkale, Prasad Mane, Shravani Chavan
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama2
- Languages
- mr
- Created
- Feb 19, 2024
- Updated
- Apr 22, 2024