Downloads · 30 days
8.8K
3% of all-time downloads
DARE_TIES_13B
yunconglong/DARE_TIES_13B
DARE_TIES_13B is a text generation model from yunconglong. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
This is a merge of pre-trained language models created using mergekit.
All-time downloads
280K
Public
Parameters
12.9B
25.8 GB on disk
Likes
10
Public
Repo makeup
Click a slice to open those files.
.safetensors25.8 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- other
- Model type
- mixtral
- Access
- Public
- Created
- Jan 30, 2024
- Updated
- Jan 30, 2024
- SHA
- 74c6e4fb
Try a prompt
Model card
From the Hugging Face model README
DARE_TIES_13B
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the DARE TIES merge method using yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B as a base.
Models Merged
The following models were included in the merge:
- ./13B_DPO
- ./13B_MATH_DPO
Configuration
The following YAML configuration was used to produce this model:
models:
- model: yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B
parameters:
density: 1.0
weight: 1.0
- model: ./13B_MATH_DPO
parameters:
density: 0.5
weight: [0.33, 0.4, 0.33]
- model: ./13B_DPO
parameters:
density: [0.33, 0.45, 0.66]
weight: 0.66
merge_method: dare_ties
base_model: yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B
parameters:
normalize: true
int8_mask: true
dtype: bfloat16
tokenizer_source : union
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- other
- Created
- Jan 30, 2024
- Updated
- Jan 30, 2024