Qwen2.5-142B-Doubled72B-Math-Instruct-Mergekit-Merge

Solshine/Qwen2.5-142B-Doubled72B-Math-Instruct-Mergekit-Merge

Qwen2.5-142B-Doubled72B-Math-Instruct-Mergekit-Merge is a text generation model from Solshine. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.

Downloads · 30 days

6% of all-time downloads

All-time downloads

214

Public

Parameters

142B

284 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors284 GB · 100%

At a glance

Task: Text Generation
Library: transformers
License: other
Model type: qwen2
Access: Public
Created: Sep 24, 2024
Updated: Sep 24, 2024
SHA: 18db99f6

Try a prompt

Base models

Model card

From the Hugging Face model README

Qwen2.5-142B-Doubled72B-Math-Instruct (Mergekit-Merge) by Solshine (Caleb DeLeeuw)

image/png

merge

This is a merge of pre-trained language models created using mergekit.

License

Hippocratic License 3.0 + Ecocide module, + Extractive Industries module, + Copyleft https://firstdonoharm.dev/version/3/0/cl-eco-extr.txt

Merge Details

Merge Method

This model was merged using the passthrough merge method. Every layer is doubled in order, from Qwen/Qwen2.5-72B-Instruct and Qwen/Qwen2.5-Math-72B, alternating which model is adding a layer and the MLP layers + 2 output layers only taken from the instruct model, creating 142B parameters. No additional fine-tune has been done in this merged model.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

slices:
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [0, 1]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [0, 1]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [1, 2]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [1, 2]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [2, 3]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [2, 3]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [3, 4]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [3, 4]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [4, 5]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [4, 5]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [5, 6]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [5, 6]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [6, 7]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [6, 7]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [7, 8]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [7, 8]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [8, 9]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [8, 9]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [9, 10]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [9, 10]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [10, 11]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [10, 11]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [11, 12]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [11, 12]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [12, 13]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [12, 13]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [13, 14]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [13, 14]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [14, 15]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [14, 15]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [15, 16]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [15, 16]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [16, 17]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [16, 17]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [17, 18]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [17, 18]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [18, 19]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [18, 19]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [19, 20]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [19, 20]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [20, 21]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [20, 21]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [21, 22]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [21, 22]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [22, 23]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [22, 23]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [23, 24]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [23, 24]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [24, 25]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [24, 25]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [25, 26]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [25, 26]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [26, 27]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [26, 27]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [27, 28]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [27, 28]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [28, 29]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [28, 29]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [29, 30]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [29, 30]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [30, 31]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [30, 31]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [31, 32]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [31, 32]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B 
        layer_range: [32, 33]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [32, 33]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [33, 34]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [33, 34]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [34, 35]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [34, 35]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [35, 36]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [35, 36]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [36, 37]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [36, 37]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [37, 38]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [37, 38]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [38, 39]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [38, 39]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [39, 40]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [39, 40]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [40, 41]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [40, 41]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [41, 42]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [41, 42]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [42, 43]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [42, 43]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [43, 44]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [43, 44]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [44, 45]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [44, 45]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [45, 46]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [45, 46]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [46, 47]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [46, 47]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [47, 48]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [47, 48]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [48, 49]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [48, 49]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [49, 50]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [49, 50]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [50, 51]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [50, 51]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [51, 52]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [51, 52]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [52, 53]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [52, 53]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [53, 54]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [53, 54]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [54, 55]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [54, 55]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [55, 56]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [55, 56]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [56, 57]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [56, 57]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [57, 58]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [57, 58]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [58, 59]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [58, 59]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [59, 60]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [59, 60]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [60, 61]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [60, 61]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [61, 62]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [61, 62]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [62, 63]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [62, 63]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [63, 64]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [63, 64]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [64, 65]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [64, 65]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [65, 66]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [65, 66]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [66, 67]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [66, 67]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [67, 68]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [67, 68]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [68, 69]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [68, 69]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [69, 70]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [69, 70]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [70, 71]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [70, 71]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [71, 72]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [71, 72]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [72, 73]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [72, 73]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [73, 74]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [73, 74]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [74, 75]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [74, 75]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [75, 76]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [75, 76]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [76, 77]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [76, 77]
  - sources:
      - model: Qwen/Qwen2.5-Math-72B
        layer_range: [77, 78]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [77, 78]
  - sources:
      - model: Qwen/Qwen2.5-72B-Instruct
        layer_range: [77, 80]
merge_method: passthrough
dtype: float16

Task: Text Generation
Library: transformers
Type: qwen2
License: other
Created: Sep 24, 2024
Updated: Sep 24, 2024