Ethos-Alpha-LLaMa-70B

Part of a multi merge experiment. The idea behind it is to create 3 individual models:

• Pathos: For ERP and uncensored NSFW content
• Ethos: For prose and storytelling
• Logos: For intelligence and awareness

The three models above will then be combined into:

• Kairos: The best of all three hopefully.

I will be using differnet merge methods for these merges in an attempt to find the best combinations hence the Alpha, Beta and Delta tags you will see on each which represent different merge methods.

merge

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Linear DELLA merge method using meta-llama/Llama-3.3-70B-Instruct as a base.

Models Merged

The following models were included in the merge:

• Sao10K/L3.3-70B-Euryale-v2.3
• Sao10K/70B-L3.3-mhnnn-x1
• EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1
• Doctor-Shotgun/L3.3-70B-Magnum-v4-SE
• Sao10K/70B-L3.3-Cirrus-x1

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: Doctor-Shotgun/L3.3-70B-Magnum-v4-SE
    parameters:
      weight: 0.20
      density: 0.7
  - model: Sao10K/70B-L3.3-mhnnn-x1
    parameters:
      weight: 0.20
      density: 0.7
  - model: Sao10K/70B-L3.3-Cirrus-x1
    parameters:
      weight: 0.20
      density: 0.7
  - model: EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1
    parameters:
      weight: 0.20
      density: 0.7
  - model: Sao10K/L3.3-70B-Euryale-v2.3
    parameters:
      weight: 0.20
      density: 0.7
merge_method: della_linear
base_model: meta-llama/Llama-3.3-70B-Instruct
parameters:
  epsilon: 0.2
  lambda: 1.1
out_dtype: bfloat16
tokenizer:
 source: Sao10K/L3.3-70B-Euryale-v2.3