Downloads · 30 days
325
39% of all-time downloads
rwkv-7-pile
BlinkDL/rwkv-7-pile
rwkv-7-pile is a machine learning model from BlinkDL. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for rwkv. The card lists the license as apache-2.0.
RWKV-7 trained on the Pile w/ "20b tokenizer" (332115325534 tokens)
All-time downloads
839
Public
Repo size
6.8 GB
Likes
15
Public
Repo makeup
Click a slice to open those files.
.pth4.8 GB · 100%
At a glance
- Library
- rwkv
- License
- apache-2.0
- Access
- Public
- Created
- Nov 28, 2024
- Updated
- Jul 5, 2026
- SHA
- dac041b0
Datasets
Model card
From the Hugging Face model README
RWKV-7 trained on the Pile w/ "20b tokenizer" (332115325534 tokens)
0.1B = L12-D768, lr 8e-4 to 3e-5 cosine decay, wd 0.1, bsz 8x30x4096
0.4B = L24-D1024, lr 6e-4 to 2e-5 cosine decay, wd 0.1, bsz 8x30x4096
1.5B = L24-D2048, lr 5e-4 to 1.5e-5 cosine decay, wd 0.1, bsz 8x45x4096
Check https://github.com/BlinkDL/RWKV-LM for details.
How to run it:
https://pypi.org/project/rwkv/
or
- Library
- rwkv
- License
- apache-2.0
- Created
- Nov 28, 2024
- Updated
- Jul 5, 2026