Downloads · 30 days
61
1% of all-time downloads
cali-0.1B
Sandroeth/cali-0.1B
cali-0.1B is a text generation model from Sandroeth. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
CALI (Computer Assistant Lightweight Intelligence) is an experimental lightweight language model trained from scratch on a limited-scale bilingual dataset consisting of Indonesian and English text.
All-time downloads
6.1K
Public
Parameters
124M
892 MB on disk
Likes
5
Public
Repo makeup
Click a slice to open those files.
.safetensors495 MB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- cali
- Access
- Public
- Created
- May 21, 2026
- Updated
- Jul 28, 2026
- SHA
- c9de368e
Try a prompt
Datasets
Model card
From the Hugging Face model README
CALI
CALI (Computer Assistant Lightweight Intelligence) is an experimental lightweight language model trained from scratch on a limited-scale bilingual dataset consisting of Indonesian and English text.
The model was developed to explore lightweight transformer architectures, small-model efficiency, and language model training under constrained computational resources and limited datasets.
CALI is NOT a large-scale foundation model and was NOT trained on massive internet-scale datasets like modern commercial language models.
Important Notes
Due to the relatively small pretraining corpus, the model may exhibit noticeable bias toward the most recent or dominant domains seen during training. Continued pretraining, fine-tuning, or alignment is strongly recommended depending on the intended use case.
Model Details
| Property | Value |
|---|---|
| Parameters | 121M |
| Layers | 11 |
| Hidden Size | 768 |
| Attention Heads | 4 |
| KV Heads | 1 |
| Head Dimension | 192 |
| FFN Dimension | 2304 |
| Context Length | 1024 |
| Vocabulary Size | 32000 |
Architecture
For a detailed explanation of the CALI architecture:
Pretraining
CALI was trained entirely from scratch using carefully selected and filtered datasets designed for research purposes rather than maximizing dataset size.
The training corpus includes:
- English text
- Indonesian text
- Wikipedia
- News articles
- General documents
- Source code
Benchmark Results
Below is the performance comparison of CALI-0.1B against several prominent Small Language Models (SLMs) in the 100M+ parameter range.
| Model Name | Piqa | MMLU Math | ARC-Challenge | HellaSwag |
|---|---|---|---|---|
| CALI-0.1B | 54.19% | 28.04% | 24.66% | 27.00% |
| SmolLM2-135M | 58.50% | 29.90% | 31.10% | 43.20% |
| GPT-X2-125M | 51.60% | 27.80% | 27.80% | 40.50% |
| SmolLM-135M | 56.30% | 28.80% | 28.80% | 42.70% |
| MobileLLM-R1-140M-base | 49.90% | 24.70% | 24.70% | 33.90% |
| GPT-X-125M | 50.80% | 26.70% | 26.70% | 36.50% |
| GPT-2 (124M) | 39.50% | 22.60% | 22.60% | 31.50% |
| GPT-Neo-125M | 39.40% | 22.90% | 22.90% | 30.40% |
| OPT-125M | 40.20% | 22.90% | 22.90% | 31.40% |
Note: The CALI-0.1B scores are strict raw accuracies (acc / acc_norm) obtained directly from the evaluation tracker logs.
Training Progress
| Tokens | Step | Final Loss |
|---|---|---|
| 250M | 13,564 | 3.53 |
| 350M | 18,989 | 3.53 |
| 450M | 24,415 | 4.69 |
| 614M | 33,356 | 2.71 |
Notes
- Experimental transformer architecture
- Requires a custom inference implementation
- Uses Grouped-Query Attention (GQA)
- Intended for research and experimentation
Citation
If you use or reference this model in your research or projects, please cite:
@article{cali2026,
title = {CALI 0.1B},
author = {Sandroeth},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/Sandroeth/cali-0.1B}
}
Author
Sandroeth
- Task
- Text Generation
- Library
- transformers
- Type
- cali
- License
- apache-2.0
- Languages
- en
- Created
- May 21, 2026
- Updated
- Jul 28, 2026