Downloads · 30 days
27
1% of all-time downloads
Llama-3-5B-Sheard
raincandy-u/Llama-3-5B-Sheard
Llama-3-5B-Sheard is a text generation model from raincandy-u. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
All-time downloads
4.3K
Public
Parameters
5.8B
11.7 GB on disk
Likes
9
Public
Repo makeup
Click a slice to open those files.
.safetensors11.7 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- other
- Model type
- llama
- Access
- Public
- Created
- Apr 23, 2024
- Updated
- Apr 23, 2024
- SHA
- 3d586899
Try a prompt
Model card
From the Hugging Face model README
Llama-3-5B-Sheard
Pruned version of Llama-3-8b.
Tool used: PrunMe, Mergekit.
Meta Llama 3 is licensed under the Meta Llama 3 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.
Training
After sliced by mergekit, the model is continue-pretrained on minipile for 1 epoch and ~100k samples. Then we trained it using ORPO on Llama-3-70b generated DPO pairs.
Disclaimer
This model is for testing purposes only, and when the system prompt is not empty, the output may repeat and not stop!
<h1><a href="https://discord.gg/Dw8SkfBm">Join our discord</a></h1>- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- other
- Languages
- en
- Created
- Apr 23, 2024
- Updated
- Apr 23, 2024