Downloads · 30 days
1.1K
33% of all-time downloads
rwkv-4-pileplus
BlinkDL/rwkv-4-pileplus
rwkv-4-pileplus is a text generation model from BlinkDL. Use it when you need the model to write or continue text. It is set up for rwkv. The card lists the license as apache-2.0.
RWKV-4-pile models finetuning on [RedPajama + some of Pile v2 = 1.7T tokens]. Updated with 2020+2021+2022 data, and better at all European languages.
All-time downloads
3.4K
Public
Repo size
31.5 GB
Likes
40
Public
Repo makeup
Click a slice to open those files.
.pth10.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- rwkv
- License
- apache-2.0
- Access
- Public
- Created
- May 4, 2023
- Updated
- May 21, 2023
- SHA
- e637e53d
Try a prompt
Model card
From the Hugging Face model README
RWKV-4 PilePlus
Model Description
RWKV-4-pile models finetuning on [RedPajama + some of Pile v2 = 1.7T tokens]. Updated with 2020+2021+2022 data, and better at all European languages.
Although some of these are intermedia checkpoints (XXXGtokens means finetuned for XXXG tokens), you can already use them because I am finetuning from Pile models (instead of retraining).
Note: not instruct tuned yet, and recommended to replace vanilla Pile models.
7B and 14B coming soon.
See https://github.com/BlinkDL/RWKV-LM for details.
Use https://github.com/BlinkDL/ChatRWKV to run it.
- Task
- Text Generation
- Library
- rwkv
- License
- apache-2.0
- Languages
- en
- Created
- May 4, 2023
- Updated
- May 21, 2023