Downloads · 30 days
15
1% of all-time downloads
Wizard-Zephyr-Orpo-8x22B
tlphams/Wizard-Zephyr-Orpo-8x22B
Wizard-Zephyr-Orpo-8x22B is a text generation model from tlphams. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-sa-4.0.
This is a merge of pre-trained language models created using mergekit.
All-time downloads
2K
Public
Parameters
141B
281 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors281 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- cc-by-nc-sa-4.0
- Model type
- mixtral
- Access
- Public
- Created
- Apr 24, 2024
- Updated
- May 6, 2024
- SHA
- 5dceb6cf
Try a prompt
Base models
Model card
From the Hugging Face model README
merge
This is a merge of pre-trained language models created using mergekit.
Merge Details
Models Merged
The following models were included in the merge:
Benchmark results
1. MT-Bench from lmsys
We adapted the code from FastChat to benchmark our model with GPT-4 as a judge. Here is the result
| | Model | Turn | Score |
|--------|-----------------------------------------|------|----------|
| First | tlphams/Wizard-Zephyr-Orpo-8x22B | 1 | 9.1625 |
| | mistralai/Mixtral-8x22B-Instruct-v0.1 | 1 | 9.1500 |
| Second | tlphams/Wizard-Zephyr-Orpo-8x22B | 2 | 8.873418 |
| | mistralai/Mixtral-8x22B-Instruct-v0.1 | 2 | 8.250000 |
| Average| tlphams/Wizard-Zephyr-Orpo-8x22B | | 9.018868 |
| | mistralai/Mixtral-8x22B-Instruct-v0.1 | | 8.700000 |
The score is slightly lower than alpindale/WizardLM-2-8x22B, but still higher than GPT-4-0314. Then the research and experimental work still need to continue ^^
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- cc-by-nc-sa-4.0
- Created
- Apr 24, 2024
- Updated
- May 6, 2024