Liph.42-slerp

MSL7/Liph.42-slerp

Liph.42-slerp is a text generation model from MSL7. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.

Liph.43 is a merge of the following models using mergekit: liminerity/merge5 liminerity/Phigments12

Downloads · 30 days

1% of all-time downloads

All-time downloads

9.9K

Public

Parameters

2.8B

5.6 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors5.6 GB · 100%

At a glance

Task: Text Generation
Library: transformers
License: apache-2.0
Model type: phi
Access: Public
Created: Mar 11, 2024
Updated: Mar 11, 2024
SHA: 12444de7

Try a prompt

Model card

From the Hugging Face model README

Liph.43

Liph.43 is a merge of the following models using mergekit:

🧩 Configuration


slices:
  - sources:
      - model: liminerity/phigment6-slerp
        layer_range: [0, 32]
      - model: liminerity/phigment6-slerp
        layer_range: [0, 32]
merge_method: slerp
base_model: liminerity/phigment6-slerp
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16


slices:
  - sources:
      - model: rhysjones/phi-2-orange-v2
        layer_range: [0, 32]
      - model: liminerity/merge
        layer_range: [0, 32]
merge_method: slerp
base_model: rhysjones/phi-2-orange-v2
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16


slices:
  - sources:
      - model: liminerity/merge1
        layer_range: [0, 32]
      - model: liminerity/phigment6-slerp
        layer_range: [0, 32]
merge_method: slerp
base_model: liminerity/merge1
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16


slices:
  - sources:
      - model: liminerity/Liph.42
        layer_range: [0, 32]
      - model: liminerity/merge2
        layer_range: [0, 32]
merge_method: slerp
base_model: liminerity/Liph.42
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16


slices:
  - sources:
      - model: liminerity/merge3
        layer_range: [0, 32]
      - model: rhysjones/phi-2-orange-v2
        layer_range: [0, 32]
merge_method: slerp
base_model: liminerity/merge3
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16


slices:
  - sources:
      - model: liminerity/Phigments12
        layer_range: [0, 32]
      - model: liminerity/merge4
        layer_range: [0, 32]
merge_method: slerp
base_model: liminerity/Phigments12
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16


slices:
  - sources:
      - model: liminerity/merge5
        layer_range: [0, 32]
      - model: liminerity/Phigments12
        layer_range: [0, 32]
merge_method: slerp
base_model: liminerity/merge5
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16

Task: Text Generation
Library: transformers
Type: phi
License: apache-2.0
Created: Mar 11, 2024
Updated: Mar 11, 2024