Downloads · 30 days
2
6% of all-time downloads
llama3-7b-en2-en1-134k
The-CoLab/llama3-7b-en2-en1-134k
llama3-7b-en2-en1-134k is a machine learning model from The-CoLab. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama3.
LLaMA-3 7B pretrained on English bilingual data (en1 + en2), 134k steps. Uses the en1 embedding matrix (--extractvocab 0) from the 2×vocabulary checkpoint. For the en2 embedding matrix version see The-CoLab/llama3-7b-…
All-time downloads
33
Public
Parameters
6.3B
12.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors12.6 GB · 100%
At a glance
- License
- llama3
- Model type
- llama
- Access
- Public
- Created
- Aug 17, 2026
- Updated
- Aug 17, 2026
- SHA
- 5fbb421c
Model card
From the Hugging Face model README
llama3-7b-en2-en1-134k
LLaMA-3 7B pretrained on English bilingual data (en1 + en2), 134k steps.
Uses the en1 embedding matrix (--extract_vocab 0) from the 2×vocabulary checkpoint.
For the en2 embedding matrix version see The-CoLab/llama3-7b-en1-en2-134k.
Validation Results
Final checkpoint: step 133,600
| Validation set | Cross-entropy loss (nats) | Perplexity |
|---|---|---|
| English 1 (en1) | 2.4598 | 11.70 |
| English 2 (en2) | 2.5351 | 12.62 |
Training Curves
See The-CoLab/llama3-7b-en1-en2-134k for training curves (same checkpoint, different embedding extraction).
Citation
Part of the The-CoLab multilingual-transfer collection.
- Type
- llama
- License
- llama3
- Languages
- en
- Created
- Aug 17, 2026
- Updated
- Aug 17, 2026