Downloads · 30 days
7
2% of all-time downloads
Llama-3-8b-iter-dpo-179k
OpenRLHF/Llama-3-8b-iter-dpo-179k
Llama-3-8b-iter-dpo-179k is a text generation model from OpenRLHF. Use it when you need the model to write or continue text. It is set up for transformers.
This model is trained with Iterative DPO in OpenRLHF
All-time downloads
374
Public
Parameters
8B
16.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors16.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Jul 13, 2024
- Updated
- Jul 28, 2024
- SHA
- a9a5d181
Try a prompt
Model card
From the Hugging Face model README
This model is trained with Iterative DPO in OpenRLHF
Datasets and Hyperparameters
- Reward Model:https://huggingface.co/OpenLLMAI/Llama-3-8b-rm-700k
- SFT Model: https://huggingface.co/OpenLLMAI/Llama-3-8b-sft-mixture
- Prompt Dataset: https://huggingface.co/datasets/OpenLLMAI/prompt-collection-v0.1
Max Prompt Length: 2048
Max Response Length: 2048
best_of_n: 2 (2 samples for each prompt)
Learning Rate: 5e-7
Beta: 0.1
Scheduler: Cosine with Warmup (0.03) and MinLR (0.1 * init_lr)
Rollout Batch Size: 20000
Training Batch Size: 256
Number of Iterations: 9
Evaluation
########## First turn ##########
score
model turn
Llama3-iter-dpo 1 8.55
########## Second turn ##########
score
model turn
Llama3-iter-dpo 2 7.95625
########## Average ##########
score
model
Llama3-iter-dpo 8.253125
Llama3-sft-baseline 7.69
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Jul 13, 2024
- Updated
- Jul 28, 2024