shadow-clown-7B-slerp

CorticalStack/shadow-clown-7B-slerp

shadow-clown-7B-slerp is a text generation model from CorticalStack. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.

<img src="shadowclown.png" alt="Shadow clown logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/

Downloads · 30 days

1% of all-time downloads

All-time downloads

10K

Public

Parameters

7.2B

14.5 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors14.5 GB · 100%

At a glance

Task: Text Generation
Library: transformers
License: apache-2.0
Model type: mistral
Access: Public
Created: Mar 13, 2024
Updated: Mar 13, 2024
SHA: 432ca893

Try a prompt

Model card

From the Hugging Face model README

shadow-clown-7B-slerp

shadow-clown-7B-slerp is a DARE merge of the following models using mergekit:

See the paper Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch for more on the method.

🧩 Configuration

slices:
  - sources:
      - model: CorticalStack/pastiche-crown-clown-7b-dare-dpo
        layer_range: [0, 32]
      - model: MSL7/INEX12-7b
        layer_range: [0, 32]
merge_method: slerp
base_model: CorticalStack/pastiche-crown-clown-7b-dare-dpo
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16

slices:
  - sources:
      - model: liminerity/M7-7b
        layer_range: [0, 32]
      - model: CorticalStack/pastiche-crown-clown-7b-dare-dpo
        layer_range: [0, 32]
merge_method: slerp
base_model: liminerity/M7-7b
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16


slices:
  - sources:
      - model: ammarali32/multi_verse_model
        layer_range: [0, 32]
      - model: liminerity/merge
        layer_range: [0, 32]
merge_method: slerp
base_model: ammarali32/multi_verse_model
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16


slices:
  - sources:
      - model: Gille/StrangeMerges_32-7B-slerp
        layer_range: [0, 32]
      - model: yam-peleg/Experiment26-7B
        layer_range: [0, 32]
merge_method: slerp
base_model: Gille/StrangeMerges_32-7B-slerp
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16

Task: Text Generation
Library: transformers
Type: mistral
License: apache-2.0
Created: Mar 13, 2024
Updated: Mar 13, 2024