Downloads · 30 days
65
0% of all-time downloads
brucethemoose/CaPlatTessDolXaBoros-Yi-34B-200K-DARE-Ties-ExtremeDensity
CaPlatTessDolXaBoros-Yi-34B-200K-DARE-Ties-ExtremeDensity is a text generation model from brucethemoose. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
Just a test of a very high density DARE ties merge, for benchmarking on the open llm leaderboard.
Downloads · 30 days
65
0% of all-time downloads
All-time downloads
36.6K
Public
Parameters
34.4B
68.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors68.8 GB · 100%
From the Hugging Face model README
Just a test of a very high density DARE ties merge, for benchmarking on the open llm leaderboard.
You probably shouldn't use this model, use this one instead: https://huggingface.co/brucethemoose/CaPlatTessDolXaBoros-Yi-34B-200K-DARE-Ties-HighDensity
mergekit config:
models:
- model: /home/alpha/Storage/Models/Raw/chargoddard_Yi-34B-200K-Llama
# no parameters necessary for base model
- model: /home/alpha/Storage/Models/Raw/migtissera_Tess-34B-v1.4
parameters:
weight: 0.19
density: 0.83
- model: /home/alpha//Storage/Models/Raw/bhenrym14_airoboros-3_1-yi-34b-200k
parameters:
weight: 0.14
density: 0.6
- model: /home/alpha/Storage/Models/Raw/Nous-Capybara-34B
parameters:
weight: 0.19
density: 0.83
- model: /home/alpha/Storage/Models/Raw/kyujinpy_PlatYi-34B-200K-Q
parameters:
weight: 0.14
density: 0.6
- model: /home/alpha/FastModels/ehartford_dolphin-2.2-yi-34b-200k
parameters:
weight: 0.19
density: 0.83
- model: /home/alpha/FastModels/fblgit_una-xaberius-34b-v1beta
parameters:
weight: 0.15
density: 0.08
merge_method: dare_ties
base_model: /home/alpha/Storage/Models/Raw/chargoddard_Yi-34B-200K-Llama
parameters:
int8_mask: true
dtype: bfloat16
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 71.57 |
| AI2 Reasoning Challenge (25-Shot) | 66.89 |
| HellaSwag (10-Shot) | 85.69 |
| MMLU (5-Shot) | 77.35 |
| TruthfulQA (0-shot) | 57.63 |
| Winogrande (5-shot) | 82.00 |
| GSM8k (5-shot) | 59.82 |