Downloads · 30 days
23
0% of all-time downloads
zephyr-beta-wizardLM-2-merge-7B
grimjim/zephyr-beta-wizardLM-2-merge-7B
zephyr-beta-wizardLM-2-merge-7B is a text generation model from grimjim. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This is a merge of two pre-trained language models created using mergekit. The goal was to explore the impact of merging on reasoning and narrative generation. Given that both fine-tuned models are BF16 precision (des…
All-time downloads
5.4K
Public
Parameters
7.2B
14.5 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors14.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- mistral
- Access
- Public
- Created
- Apr 17, 2024
- Updated
- Apr 17, 2024
- SHA
- a59ffc24
Try a prompt
Base models
Model card
From the Hugging Face model README
zephyr-beta-wizardLM-2-merge-7B
This is a merge of two pre-trained language models created using mergekit. The goal was to explore the impact of merging on reasoning and narrative generation. Given that both fine-tuned models are BF16 precision (despite the precision of the base model Mistral 7B v0.1 being FP16), this avoids issues with mixed precision during merging.
Merge Details
Merge Method
This model was merged using the SLERP merge method.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
slices:
- sources:
- model: HuggingFaceH4/zephyr-7b-beta
layer_range: [0,32]
- model: lucyknada/microsoft_WizardLM-2-7B
layer_range: [0,32]
merge_method: slerp
base_model: HuggingFaceH4/zephyr-7b-beta
parameters:
t:
- value: 0.5
dtype: bfloat16
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- License
- apache-2.0
- Created
- Apr 17, 2024
- Updated
- Apr 17, 2024