MSL7_-_INEX4-7b-gguf

RichardErkhov/MSL7_-_INEX4-7b-gguf

MSL7_-_INEX4-7b-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.

Downloads · 30 days

1% of all-time downloads

All-time downloads

570

Public

Repo size

82.2 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.gguf95.2 GB · 100%

At a glance

Access: Public
Created: Sep 10, 2024
Updated: Sep 10, 2024
SHA: 5c57e118

Model card

From the Hugging Face model README

Quantization made by Richard Erkhov.

Github

Discord

Request more models

INEX4-7b - GGUF

Model creator: https://huggingface.co/MSL7/
Original model: https://huggingface.co/MSL7/INEX4-7b/

Name	Quant method	Size
INEX4-7b.Q2_K.gguf	Q2_K	2.53GB
INEX4-7b.IQ3_XS.gguf	IQ3_XS	2.81GB
INEX4-7b.IQ3_S.gguf	IQ3_S	2.96GB
INEX4-7b.Q3_K_S.gguf	Q3_K_S	2.95GB
INEX4-7b.IQ3_M.gguf	IQ3_M	3.06GB
INEX4-7b.Q3_K.gguf	Q3_K	3.28GB
INEX4-7b.Q3_K_M.gguf	Q3_K_M	3.28GB
INEX4-7b.Q3_K_L.gguf	Q3_K_L	3.56GB
INEX4-7b.IQ4_XS.gguf	IQ4_XS	3.67GB
INEX4-7b.Q4_0.gguf	Q4_0	3.83GB
INEX4-7b.IQ4_NL.gguf	IQ4_NL	3.87GB
INEX4-7b.Q4_K_S.gguf	Q4_K_S	3.86GB
INEX4-7b.Q4_K.gguf	Q4_K	4.07GB
INEX4-7b.Q4_K_M.gguf	Q4_K_M	4.07GB
INEX4-7b.Q4_1.gguf	Q4_1	4.24GB
INEX4-7b.Q5_0.gguf	Q5_0	4.65GB
INEX4-7b.Q5_K_S.gguf	Q5_K_S	4.65GB
INEX4-7b.Q5_K.gguf	Q5_K	4.78GB
INEX4-7b.Q5_K_M.gguf	Q5_K_M	4.78GB
INEX4-7b.Q5_1.gguf	Q5_1	5.07GB
INEX4-7b.Q6_K.gguf	Q6_K	5.53GB
INEX4-7b.Q8_0.gguf	Q8_0	7.17GB

Original model description:

license: apache-2.0 tags:

merge
mergekit
lazymergekit
liminerity/merge2
liminerity/merge1 model-index:
name: INEX4-7b results:
- task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics:
  - type: acc_norm value: 72.95 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MSL7/INEX4-7b name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics:
  - type: acc_norm value: 88.79 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MSL7/INEX4-7b name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics:
  - type: acc value: 64.7 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MSL7/INEX4-7b name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics:
  - type: mc2 value: 74.42 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MSL7/INEX4-7b name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics:
  - type: acc value: 83.9 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MSL7/INEX4-7b name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics:
  - type: acc value: 70.28 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MSL7/INEX4-7b name: Open LLM Leaderboard

Made by Liminerity <#

INEX4-7b

INEX4-7b is a merge of the following models using mergekit:

🧩 Configuration

slices:
  - sources:
      - model: liminerity/Ingot-7b-slerp-7-forged
        layer_range: [0, 32]
      - model: yam-peleg/Experiment26-7B
        layer_range: [0, 32]
merge_method: slerp
base_model: liminerity/Ingot-7b-slerp-7-forged
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16
model: liminerity/merge

slices:
  - sources:
      - model: liminerity/Ingot-7b-slerp-7-forged
        layer_range: [0, 32]
      - model: liminerity/merge
        layer_range: [0, 32]
merge_method: slerp
base_model: liminerity/Ingot-7b-slerp-7-forged
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16
model: liminerity/merge1

slices:
  - sources:
      - model: yam-peleg/Experiment26-7B
        layer_range: [0, 32]
      - model: liminerity/merge1
        layer_range: [0, 32]
merge_method: slerp
base_model: yam-peleg/Experiment26-7B
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: float16
model: liminerity/merge2

slices:
  - sources:
      - model: liminerity/merge2
        layer_range: [0, 32]
      - model: liminerity/merge1
        layer_range: [0, 32]
merge_method: slerp
base_model: liminerity/merge2
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16
model: INEX-7b

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	75.84
AI2 Reasoning Challenge (25-Shot)	72.95
HellaSwag (10-Shot)	88.79
MMLU (5-Shot)	64.70
TruthfulQA (0-shot)	74.42
Winogrande (5-shot)	83.90
GSM8k (5-shot)	70.28

Created: Sep 10, 2024
Updated: Sep 10, 2024