Downloads · 30 days
0
AhmetSemih/llama-50m-pretrained-news-merged_bpe
llama-50m-pretrained-news-merged_bpe is a machine learning model from AhmetSemih. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Downloads · 30 days
0
Access
Public
Updated Nov 5, 2025
Repo size
2.7 GB
Likes
0
Public
Click a slice to open those files.
.safetensors417 MB · 100%
From the Hugging Face model README
| Property | Value |
|---|---|
| Architecture | LLaMA (decoder-only transformer) |
| Parameters | ~50M |
| Vocab size | 32,768 |
| Embedding dim | 256 |
| Hidden dim | 2048 |
| Layers | 20 |
| Attention heads | 128 |
| KV groups | 64 |
| Context length | 256 |
| Tokenizer | AhmetSemih/merged_dataset-32k-bpe-tokenizer |
| Dataset | habanoz/news-tr-1.8M |
| Tokens seen | 217572073 |
| Epochs | 2 |
| Batch size | 64 |
| Language | Turkish 🇹🇷 |
This model is a 50 million parameter Turkish language model trained from scratch on the habanoz/news-tr-1.8M dataset.
It was developed as a lightweight experimental model to explore Turkish-specific pretraining using a custom BPE tokenizer trained on a merged Turkish dataset.(AhmetSemih/merged_dataset-32k-bpe-tokenizer )
The model follows the LLaMA-style causal transformer architecture and was trained with a context length of 256 tokens over ~372M tokens in total.
Hardware: NVIDIA B200 GPU ~4.5 hours