Downloads · 30 days
0
rwkv-minipile
leliuga/rwkv-minipile
rwkv-minipile is a machine learning model from leliuga. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mpl-2.0.
- Architecture: RWKV - Vocabulary Size: 65,536 - Embedding Size: 768 - Number of Layers: 12 - Context Length: 512 - Data Type: bfloat16 - Dataset: Minipile - Tokens: 20,643,840 (20 Million)
Access
Public
Updated Jun 21, 2024
Repo size
3.2 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pth3.2 GB · 100%
At a glance
- License
- mpl-2.0
- Access
- Public
- Created
- Jun 21, 2024
- Updated
- Jun 21, 2024
- SHA
- 852f1f9a
Datasets
Model card
From the Hugging Face model README
RWKV Minipile
Model Specifications
- Architecture: RWKV
- Vocabulary Size: 65,536
- Embedding Size: 768
- Number of Layers: 12
- Context Length: 512
- Data Type: bfloat16
- Dataset: Minipile
- Tokens: 20,643,840 (20 Million)
The model underwent a rigorous training regimen, completing 32 epochs to optimize performance.
Wandb
Verifying checksums
sha512sum -c sha512sums.txt
Inference
pip install torch numpy
python inference.py
- License
- mpl-2.0
- Languages
- en
- Created
- Jun 21, 2024
- Updated
- Jun 21, 2024