Downloads · 30 days
109
0% of all-time downloads
grok-1-hf
keyfan/grok-1-hf
grok-1-hf is a text generation model from keyfan. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Unofficial dequantized weight of grok-1 in HF Transformers format.
All-time downloads
24.4K
Public
Repo size
2 TB
Likes
41
Public
Repo makeup
Click a slice to open those files.
.bin631 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- grok
- Access
- Public
- Created
- Mar 19, 2024
- Updated
- Mar 25, 2024
- SHA
- 64e73730
Try a prompt
Model card
From the Hugging Face model README
Unofficial dequantized weight of grok-1 in HF Transformers format.
Note: If you haven't download the weight yet, please use the fp32 revision instead which uses float32 precision for RMSNorm and Router layer for better consistency.
The (fp32) weights are converted using the script here ran inside the grok-1 repo. Since downloading the dequantized weight needs twice as much time, it's recommended to download the original weight and convert on your own.
Benchmarks
(I ran with load_in_8bit using lm-evaluation-harness due to limited hardware, so the result will be slightly worse)
- MMLU 5-shot: 0.7166
- BBH 3-shot: 0.5204
- Task
- Text Generation
- Library
- transformers
- Type
- grok
- License
- apache-2.0
- Created
- Mar 19, 2024
- Updated
- Mar 25, 2024