Downloads · 30 days
10
8% of all-time downloads
sophosympatheia/Wizard-Tulu-Dolphin-70B-v1.0
Wizard-Tulu-Dolphin-70B-v1.0 is a text generation model from sophosympatheia. Use it when you need the model to write or continue text. It is set up for transformers.
I used this blend as part of Midnight Rose v2.0.3. I haven't tested it much on its own, but I created this blend to be good at instruction following. It's nothing fancy. I just figured that WizardLM, Tulu, and Dolphin…
Downloads · 30 days
10
8% of all-time downloads
All-time downloads
124
Public
Parameters
69B
138 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors138 GB · 100%
From the Hugging Face model README
I used this blend as part of Midnight Rose v2.0.3. I haven't tested it much on its own, but I created this blend to be good at instruction following. It's nothing fancy. I just figured that WizardLM, Tulu, and Dolphin 2.2 together would be a heavy hitter for smarts.
This model was merged using the SLERP merge method.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
models:
- model: /home/llm/mergequant/models/wizard-tulu-70b-v1.0
- model: /home/llm/mergequant/models/BASE/ehartford_dolphin-2.2-70b-32000vocab
merge_method: slerp
base_model: /home/llm/mergequant/models/wizard-tulu-70b-v1.0
parameters:
t:
- value: 0.5
dtype: float16
models:
- model: /home/llm/mergequant/models/BASE/NousResearch_Llama-2-70b-hf
# no parameters necessary for base model
- model: /home/llm/mergequant/models/BASE/allenai_tulu-2-dpo-70b
parameters:
density: 0.35
weight: 0.75
- model: /home/llm/mergequant/models/BASE/WizardLM_WizardLM-70B-V1.0
parameters:
density: 0.35
weight: 0.5
merge_method: dare_ties
base_model: /home/llm/mergequant/models/BASE/NousResearch_Llama-2-70b-hf
parameters:
normalize: true
int8_mask: true
dtype: float16
tokenzer_source: union