Downloads · 30 days
7
11% of all-time downloads
burtenshaw/Qwen1.5-0.5B-dpo-mix-7k-3000
Qwen1.5-0.5B-dpo-mix-7k-3000 is a text generation model from burtenshaw. Use it when you need the model to write or continue text. It is set up for transformers.
- [X] Sample script for ORPOTrainer in 🤗<a class="link" href="https://github.com/huggingface/trl"TRL</a is added to trl/testorpotrainerdemo.py - [X] New model, 🤗<a class="link" href="https://huggingface.co/kaist-ai/…
Downloads · 30 days
7
11% of all-time downloads
All-time downloads
61
Public
Parameters
620M
12.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.bin9.9 GB · 73%
From the Hugging Face model README
Updates (24.03.25)trl/test_orpo_trainer_demo.py
This is the official repository for <a class="link" href="https://arxiv.org/abs/2403.07691">ORPO: Monolithic Preference Optimization without Reference Model</a>. The detailed results in the paper can be found in:
Model CheckpointsOur models trained with ORPO can be found in:
And the corresponding logs for the average log probabilities of chosen/rejected responses during training are reported in:
AlpacaEval
MT-Bench
IFEvalIFEval scores are measured with <a class="link" href="https://github.com/EleutherAI/lm-evaluation-harness">EleutherAI/lm-evaluation-harness</a> by applying the chat template. The scores for Llama-2-Chat (70B), Zephyr-β (7B), and Mixtral-8X7B-Instruct-v0.1 are originally reported in <a class="link" href="https://twitter.com/wiskojo/status/1739767758462877823">this tweet</a>.
| Model Type | Prompt-Strict | Prompt-Loose | Inst-Strict | Inst-Loose |
|---|---|---|---|---|
| Llama-2-Chat (70B) | 0.4436 | 0.5342 | 0.5468 | 0.6319 |
| Zephyr-β (7B) | 0.4233 | 0.4547 | 0.5492 | 0.5767 |
| Mixtral-8X7B-Instruct-v0.1 | 0.5213 | 0.5712 | 0.6343 | 0.6823 |
| Mistral-ORPO-⍺ (7B) | 0.5009 | 0.5083 | 0.5995 | 0.6163 |
| Mistral-ORPO-β (7B) | 0.5287 | 0.5564 | 0.6355 | 0.6619 |