Mira-Demeter

<div align="center"> <h1 style="font-size: 1.2em; font-weight: bold;"> All credits go to:</h1> <h1 style="font-size: 1.2em; font-weight: bold;"> <a href="https://huggingface.co/Lambent">Lambent</a> for the wonderful model series. They did all the work, I just ran a few commands.</h1> </div> <div align="center"> <img src="https://upload.wikimedia.org/wikipedia/commons/thumb/a/a2/Demeter_Altemps_Inv8546.jpg/500px-Demeter_Altemps_Inv8546.jpg" width="200" alt="Aphrodite"> </div> <div align="center"> <h1 style="font-size: 1.23em; font-weight: bold;"> <a href="https://web.archive.org/web/20251008221112if_/https://uh.edu/~cldue/texts/demeter.html">Homeric Hymn to Demeter</a></h1> </h1> </div> ### 📖 Overview [Files will be uploaded in 2 1/2 hours, give or take.]

This is a merge of pre-trained language models created using [mergekit](https://github.com/arcee-ai/mergekit. It utilizes custom merge methods.

The Recipe [Is it any good?]

ϻira Dεmeter| Methods: [Karcher, MagStock, MagVar, CABS]

<details> <summary><b>View Configuration</b></summary>

# https://huggingface.co/Lambent
name: Mira-Aphrodite-v19-freckles

merge_method: magstock

models:
    - model: ./Ingredients/Lambent_Mira-v1.8.1a-27B
      parameters:
          weight: 0.5
          density: 0.5
          epsilon: 0.4
    - model: ./Ingredients/Mira-v14 # karcher
      parameters:
          weight: 0.5
          density: 0.5

base_model: ./Ingredients/Lambent_Mira-v1.8.1a-27B

dtype: bfloat16

parameters:
  rescale: true
  t_factor: 1.0

tokenizer:
  source: union

---
name: Mira-Aphrodite-v31

merge_method: magvar

base_model: ./Ingredients/Lambent_Mira-v1.21-27B-rlvr
models:
  - model: ./Ingredients/Lambent_Mira-v1.21-27B-rlvr
    parameters:
      weight: 0.40
  - model: ./Ingredients/Lambent_Mira-v1.20-27B-dpo
    parameters:
      weight: 0.20
  - model: ./Ingredients/Lambent_Mira-v1.19.3c-27B-dpo
    parameters:
      weight: 0.20
  - model: ./Ingredients/Lambent_Mira-v1.19-27B-dpo
    parameters:
      weight: 0.20

parameters:
  top_k: 0.45
  eps: 1e-6
  bond: 0.2

tokenizer:
  source: base

dtype: float32
out_dtype: bfloat16

---
name: Mira-Demeter

merge_method: cabs

base_model: ./Ingredients/Lambent_Mira-v1.21-27B-rlvr/

models:
  - model: ./Ingredients/Lambent_Mira-v1.21-27B-rlvr/
  - model: ./Ingredients/Mira-Aphrodite-v31/
    parameters:
      weight: 0.6
      n_val: 16
      m_val: 32
  - model: ./Ingredients/Mira-Aphrodite-v19-freckles/
    parameters:
      weight: 0.4
      n_val: 11
      m_val: 33


default_n_val: 8
default_m_val: 32

pruning_order:
  - ./Ingredients/Mira-Aphrodite-v31/
  - ./Ingredients/Mira-Aphrodite-v19-freckles/

# base_model: base

dtype: float32
out_dtype: bfloat16

tokenizer:
  source: union

</details>

A conversation snippet

<details> <summary><b>View Conversation</b></summary>

#

</details>
</div>