Downloads · 30 days
0
tinychat-271M-base
kali-v/tinychat-271M-base
tinychat-271M-base is a text generation model from kali-v. Use it when you need the model to write or continue text. It is set up for jax. The card lists the license as mit.
An autoregressive Transformer language model:
Access
Public
Updated Sep 6, 2025
Repo size
426 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
Other426 MB · 100%
At a glance
- Task
- Text Generation
- Library
- jax
- License
- mit
- Access
- Public
- Created
- Sep 4, 2025
- Updated
- Sep 6, 2025
- SHA
- 18f8dcdb
Try a prompt
Model card
From the Hugging Face model README
tinychat 271M base
An autoregressive Transformer language model:
- Embedding size: 1440
- Layers: 12
- Attention heads: 16 (8 kv heads)
- Parameters: 271M
- Context length: 512
- Vocabulary size: 32,768
Trained for ~50 hours on a single RTX-3090, processing ~5B tokens from:
- 5M FineWeb articles
- 5M FineWeb-Edu articles
- Simple English Wikipedia
More details here
Performance (0-shot)
| tinychat | gpt2 | gpt2-medium | |
|---|---|---|---|
| swag | 47.3% | 48.9% | 56.3% |
| hellaswag | 30.7% | 29.0% | 37.0% |
| openbookqa | 32.8% | 28.0% | 30.6% |
- Task
- Text Generation
- Library
- jax
- License
- mit
- Languages
- en
- Created
- Sep 4, 2025
- Updated
- Sep 6, 2025