Downloads · 30 days
23
11% of all-time downloads
neox_mistral_7b_dpo_ultrafeedback
EleutherAI/neox_mistral_7b_dpo_ultrafeedback
neox_mistral_7b_dpo_ultrafeedback is a machine learning model from EleutherAI. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
GPT-NeoX now supports preference learning (SFT, DPO, KTO)! For more information on this joint effort between EleutherAI and SynthLabs, view our associated blog posts:
All-time downloads
217
Public
Parameters
7.2B
14.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors14.5 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Sep 10, 2024
- Updated
- Sep 17, 2024
- SHA
- 9384d6be
Model card
From the Hugging Face model README
GPT-NeoX now supports preference learning (SFT, DPO, KTO)! For more information on this joint effort between EleutherAI and SynthLabs, view our associated blog posts:
SynthLabs: https://www.synthlabs.ai/blog/rlhf-and-rlaif-in-gpt-neox
EleutherAI: https://www.eleuther.ai/rlhf-and-rlaif-in-gpt-neox
This is a direct preference optimization (DPO) model produced by:
- Taking the ultrachat SFT checkpoint from https://huggingface.co/HuggingFaceH4/mistral-7b-sft-beta
- Loading the model into the GPT-NeoX library and running DPO on the Zephyr 7B recipe (ultrafeedback). Example usage for running post-training in GPT-NeoX is at: https://github.com/EleutherAI/gpt-neox/tree/main/post-training
| Model | gsm8k 5-shot flexible-extract | MMLU 5-shot acc | ARC Challenge 25-shot acc_norm | HellaSwag 10-shot acc_norm | Winogrande 5-shot acc | TruthfulQA mc2 0-shot acc |
|---|---|---|---|---|---|---|
| NeoX DPO from Zephyr-SFT | 64.1 | 41.8 | 60 | 63.2 | 85.2 | 79.2 |
| Zephyr-7b-Beta | 62.5 | 34.3 | 59.8 | 63.6 | 84.4 | 77.6 |
- Type
- llama
- License
- apache-2.0
- Created
- Sep 10, 2024
- Updated
- Sep 17, 2024