Downloads · 30 days
33
11% of all-time downloads
Armenian-SmolLlama-116M-Instruct
Evanroubert/Armenian-SmolLlama-116M-Instruct
Armenian-SmolLlama-116M-Instruct is a text generation model from Evanroubert. Use it when you need the model to write or continue text. The card lists the license as mit.
Armenian-SmolLlama-116M is a state-of-the-art compact Armenian language model built on the modern Deep & Thin paradigm pioneered by Meta's MobileLLM (ICML 2024) and Hugging Face's SmolLM (135M).
All-time downloads
304
Public
Repo size
463 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin463 MB · 100%
At a glance
- Task
- Text Generation
- License
- mit
- Access
- Public
- Created
- Aug 29, 2026
- Updated
- Aug 29, 2026
- SHA
- d4d1e960
Model card
From the Hugging Face model README
🇦🇲 Armenian-SmolLlama-116M-Instruct (30 Layers Deep & Thin)
Armenian-SmolLlama-116M is a state-of-the-art compact Armenian language model built on the modern Deep & Thin paradigm pioneered by Meta's MobileLLM (ICML 2024) and Hugging Face's SmolLM (135M).
Key Architectural Specifications:
- Parameters: 115,640,640 (115.6M)
- Transformer Layers: 30 Layers (2.5x deeper than legacy 12-layer models)
- Hidden Dimension: 576
- Attention: 9 Query Heads, 3 Key-Value Heads (3:1 Grouped-Query Attention)
- Feed-Forward: SwiGLU with intermediate dimension 1536
- Embedding Sharing: Weight tying (
lm_head.weight == tok_embeddings.weight) - Pre-training: 2.1 Billion tokens on AMD Instinct MI300X accelerator.
- Alignment: Case-Augmented LIMA Gold alignment (100% case-invariant).
- Task
- Text Generation
- License
- mit
- Languages
- hy
- Created
- Aug 29, 2026
- Updated
- Aug 29, 2026