Rombos-2.5-T.E-8.1

suayptalha/Rombos-2.5-T.E-8.1

Rombos-2.5-T.E-8.1 is a text generation model from suayptalha. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-sa-4.0.

This is a merge of pre-trained language models created using mergekit.

Downloads · 30 days

26% of all-time downloads

All-time downloads

234

Public

Parameters

7.6B

15.2 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors15.2 GB · 100%

At a glance

Task: Text Generation
Library: transformers
License: cc-by-nc-sa-4.0
Model type: qwen2
Access: Public
Created: Nov 16, 2024
Updated: Dec 22, 2024
SHA: 1550a096

Try a prompt

Base models

Model card

From the Hugging Face model README

merged

This is a merge of pre-trained language models created using mergekit.

You can use ChatML format.

Merge Method

This model was merged using the SLERP merge method.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

base_model: rombodawg/Rombos-LLM-V2.5-Qwen-7b
dtype: bfloat16
merge_method: slerp
parameters:
  t:
  - filter: self_attn
    value: [0.0, 0.5, 0.3, 0.7, 1.0]
  - filter: mlp
    value: [1.0, 0.5, 0.7, 0.3, 0.0]
  - value: 0.5
slices:
- sources:
  - layer_range: [0, 28]
    model: rombodawg/Rombos-LLM-V2.5-Qwen-7b
  - layer_range: [0, 28]
    model: Cran-May/T.E-8.1

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	27.34
IFEval (0-Shot)	69.25
BBH (3-Shot)	36.50
MATH Lvl 5 (4-Shot)	0.83
GPQA (0-shot)	8.17
MuSR (0-shot)	10.98
MMLU-PRO (5-shot)	38.28

Task: Text Generation
Library: transformers
Type: qwen2
License: cc-by-nc-sa-4.0
Created: Nov 16, 2024
Updated: Dec 22, 2024