Downloads · 30 days
30
13% of all-time downloads
Llama-3.1-5.4B
XiaodongChen/Llama-3.1-5.4B
Llama-3.1-5.4B is a machine learning model from XiaodongChen. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
The model is derived from Llama-3.1-8B through pruning using LLM-Streamline (Streamlining Redundant Layers to Compress Large Language Models, ICLR 2025 Spotlight). The entire training process required only 1.3B tokens.
All-time downloads
233
Public
Parameters
5.4B
10.8 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors10.8 GB · 100%
At a glance
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Feb 26, 2025
- Updated
- Feb 26, 2025
- SHA
- 932931ac
Base models
Model card
From the Hugging Face model README
The model is derived from Llama-3.1-8B through pruning using LLM-Streamline (Streamlining Redundant Layers to Compress Large Language Models, ICLR 2025 Spotlight). The entire training process required only 1.3B tokens.
Below are the results of the evaluation using lm-eval:
| arc_c | arc_e | boolq | hellaswag | openbookqa | rte | winogrande | Avg | |
|---|---|---|---|---|---|---|---|---|
| Llama-3.1-8B | 50.4 | 80.3 | 81.2 | 60.2 | 34.8 | 67.9 | 73.0 | 64.0 |
| Llama-3.1-5.4B | 42.1 | 72.2 | 78.0 | 54.3 | 27.2 | 62.8 | 71.0 | 58.2 |
- Type
- llama
- License
- mit
- Created
- Feb 26, 2025
- Updated
- Feb 26, 2025