Downloads · 30 days
14
1% of all-time downloads
online-dpo-qwen2-4
qgallouedec/online-dpo-qwen2-4
online-dpo-qwen2-4 is a text generation model from qgallouedec. Use it when you need the model to write or continue text. It is set up for transformers.
This model is a fine-tuned version of Qwen/Qwen2-0.5B-Instruct on the trl-lib/ultrafeedback-prompt dataset. It has been trained using TRL.
All-time downloads
1.2K
Public
Parameters
494M
13.8 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors2 GB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- qwen2
- Access
- Public
- Created
- Sep 25, 2024
- Updated
- Sep 25, 2024
- SHA
- c97b66bc
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
Model Card for online-dpo-qwen2-4
This model is a fine-tuned version of Qwen/Qwen2-0.5B-Instruct on the trl-lib/ultrafeedback-prompt dataset. It has been trained using TRL.
Quick start
from transformers import pipeline
question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
generator = pipeline("text-generation", model="qgallouedec/online-dpo-qwen2-4", device="cuda")
output = generator([{"role": "user", "content": question}], max_new_tokens=500)[0]
print(output["generated_text"][1]["content"])
Training procedure
This model was trained with Online DPO, a method introduced in Direct Language Model Alignment from Online AI Feedback.
Framework versions
- TRL: 0.12.0.dev0
- Transformers: 4.45.0.dev0
- Pytorch: 2.4.1
- Datasets: 3.0.0
- Tokenizers: 0.19.1
- Task
- Text Generation
- Library
- transformers
- Type
- qwen2
- Created
- Sep 25, 2024
- Updated
- Sep 25, 2024