Downloads · 30 days
0
llama-50m-pretrained-books-tr_tokenizer
AhmetSemih/llama-50m-pretrained-books-tr_tokenizer
llama-50m-pretrained-books-tr_tokenizer is a machine learning model from AhmetSemih. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Access
Public
Updated Oct 14, 2025
Repo size
3.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors209 MB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Sep 18, 2025
- Updated
- Oct 14, 2025
- SHA
- c9143a0c
Datasets
Model card
From the Hugging Face model README
🧠 LLaMA-50M Turkish Books
Model Summary
| Property | Value |
|---|---|
| Architecture | LLaMA (decoder-only transformer) |
| Parameters | ~50M |
| Vocab size | 32,768 |
| Embedding dim | 256 |
| Hidden dim | 2048 |
| Layers | 20 |
| Attention heads | 128 |
| KV groups | 64 |
| Context length | 256 |
| Tokenizer | turkish_tokenizer (alibayram) |
| Dataset | alibayram/tr_books |
| Tokens seen | 372,679,971 |
| Epochs | 2 |
| Batch size | 64 |
| Language | Turkish 🇹🇷 |
Model Description
llama-50m-turkish-books is a 50 million parameter Turkish language model trained from scratch on the alibayram/tr_books dataset.
It was developed as a lightweight experimental model to explore Turkish-specific tokenization and morphology-aware pretraining using the custom turkish_tokenizer.
The model follows the LLaMA-style causal transformer architecture and was trained with a context length of 256 tokens over ~372M tokens in total.
Training Environment
Hardware: NVIDIA B200 GPU ~7 hours
- License
- apache-2.0
- Languages
- tr
- Created
- Sep 18, 2025
- Updated
- Oct 14, 2025