Neo_7b-merge6

Neo_7b-merge6

Neo_7b-merge6 is a merge of the following models using LazyMergekit:

• DewEfresh/neo_7b

🧩 Configuration

models:
  - model: m-a-p/neo_7b
  - model: DewEfresh/neo_7b
    parameters:
      weight: 0.5
merge_method: slerp
base_model: m-a-p/neo_7b
parameters:
  t: 0.5
dtype: bfloat16

custom_layers:
  - sources:
      - model: m-a-p/neo_7b
        layers: [0, 3]
      - model: DewEfresh/neo_7b
        layers: [0, 3]
    method: weighted_average
    target_layers: [0]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [1, 3]
      - model: DewEfresh/neo_7b
        layers: [1, 3]
    method: weighted_average
    target_layers: [1]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [2, 3]
      - model: DewEfresh/neo_7b
        layers: [2, 3]
    method: weighted_average
    target_layers: [2]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [4, 7]
      - model: DewEfresh/neo_7b
        layers: [4, 7]
    method: weighted_average
    target_layers: [3]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [5, 7]
      - model: DewEfresh/neo_7b
        layers: [5, 7]
    method: weighted_average
    target_layers: [4]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [6, 7]
      - model: DewEfresh/neo_7b
        layers: [6, 7]
    method: weighted_average
    target_layers: [5]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [8, 11]
      - model: DewEfresh/neo_7b
        layers: [8, 11]
    method: weighted_average
    target_layers: [6]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [9, 11]
      - model: DewEfresh/neo_7b
        layers: [9, 11]
    method: weighted_average
    target_layers: [7]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [10, 11]
      - model: DewEfresh/neo_7b
        layers: [10, 11]
    method: weighted_average
    target_layers: [8]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [12, 15]
      - model: DewEfresh/neo_7b
        layers: [12, 15]
    method: weighted_average
    target_layers: [9]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [13, 15]
      - model: DewEfresh/neo_7b
        layers: [13, 15]
    method: weighted_average
    target_layers: [10]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [14, 15]
      - model: DewEfresh/neo_7b
        layers: [14, 15]
    method: weighted_average
    target_layers: [11]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [16, 19]
      - model: DewEfresh/neo_7b
        layers: [16, 19]
    method: weighted_average
    target_layers: [12]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [17, 19]
      - model: DewEfresh/neo_7b
        layers: [17, 19]
    method: weighted_average
    target_layers: [13]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [18, 19]
      - model: DewEfresh/neo_7b
        layers: [18, 19]
    method: weighted_average
    target_layers: [14]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [20, 23]
      - model: DewEfresh/neo_7b
        layers: [20, 23]
    method: weighted_average
    target_layers: [15]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [21, 23]
      - model: DewEfresh/neo_7b
        layers: [21, 23]
    method: weighted_average
    target_layers: [16]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [22, 23]
      - model: DewEfresh/neo_7b
        layers: [22, 23]
    method: weighted_average
    target_layers: [17]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [24, 27]
      - model: DewEfresh/neo_7b
        layers: [24, 27]
    method: weighted_average
    target_layers: [18]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [25, 27]
      - model: DewEfresh/neo_7b
        layers: [25, 27]
    method: weighted_average
    target_layers: [19]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [26, 27]
      - model: DewEfresh/neo_7b
        layers: [26, 27]
    method: weighted_average
    target_layers: [20]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [28, 31]
      - model: DewEfresh/neo_7b
        layers: [28, 31]
    method: weighted_average
    target_layers: [21]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [29, 31]
      - model: DewEfresh/neo_7b
        layers: [29, 31]
    method: weighted_average
    target_layers: [22]
    layer_weights: [0.75, 0.25]
  - sources:
      - model: m-a-p/neo_7b
        layers: [30, 31]
      - model: DewEfresh/neo_7b
        layers: [30, 31]
    method: weighted_average
    target_layers: [23]
    layer_weights: [0.75, 0.25]

💻 Usage

!pip install -qU transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "DewEfresh/Neo_7b-merge6"
messages = [{"role": "user", "content": "What is a large language model?"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])