Downloads · 30 days
7
9% of all-time downloads
Anurag9817/nepali-llama-590m-1.05b-tokens
nepali-llama-590m-1.05b-tokens is a machine learning model from Anurag9817. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
A 590M-parameter Nepali-oriented causal language model.
Downloads · 30 days
7
9% of all-time downloads
All-time downloads
78
Public
Parameters
590M
2.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors2.4 GB · 100%
From the Hugging Face model README
A 590M-parameter Nepali-oriented causal language model.
590,077,44040,0001536409618126The project uses the original raw SentencePiece tokenizer.
tokenizer/nepali_llama_40k.model
Special token IDs:
0123Load it using sentencepiece.SentencePieceProcessor.
Base checkpoint:
Anurag9817/nepali-llama-590m-800m-tokens
Dataset:
HuggingFaceFW/fineweb-edu
Config: sample-10BT
The first 800,000,000 tokenizer tokens were skipped using exact
SentencePiece token counting.
A fresh pool of 300,000,000 tokens was extracted.
Training used 250,019,840 tokens.
2 × Tesla T428102432,7687,630AdamW 8-bitFP16OFFON3e-41500.1cosine4 / GPU~800M tokens250,019,840 tokens~1.05B tokens3.0894~10.62 hoursThis checkpoint can be loaded as the base model for additional continued pretraining or supervised fine-tuning.
The repository includes training metadata and the training script.
Intermediate optimizer and scheduler states were not saved in this run, so future training starts from the model weights rather than an exact optimizer-state resume.