Downloads · 30 days
7
4% of all-time downloads
Orpo-GutenLlama-3-8B-v2
macadeliccc/Orpo-GutenLlama-3-8B-v2
Orpo-GutenLlama-3-8B-v2 is a text generation model from macadeliccc. Use it when you need the model to write or continue text. It is set up for transformers.
+ Learning Rate: 8e-6 + Batch Size: 1 + Eval Batch size: 1 + Gradient accumulation steps: 4 + Epochs: 3 + Training Loss: 0.88
All-time downloads
180
Public
Parameters
8B
16.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors16.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Apr 21, 2024
- Updated
- Apr 21, 2024
- SHA
- f8a13a79
Try a prompt
Model card
From the Hugging Face model README
Orpo-GutenLlama-3-8B-v2
Training Params
- Learning Rate: 8e-6
- Batch Size: 1
- Eval Batch size: 1
- Gradient accumulation steps: 4
- Epochs: 3
- Training Loss: 0.88
Training time: 4 hours on 1x4090. This is a small 1800 sample fine tune to get comfortable with ORPO fine tuning before scaling up.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Apr 21, 2024
- Updated
- Apr 21, 2024