Downloads · 30 days
19
9% of all-time downloads
HerculeanSea-upd-7b-128k
Nitral-Archive/HerculeanSea-upd-7b-128k
HerculeanSea-upd-7b-128k is a text generation model from Nitral-Archive. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
The following models were included in the merge: Locutusque/Hercules-2.5-Mistral-7B Test157t/Pasta-Sea-7b-128k
All-time downloads
203
Public
Parameters
7.2B
14.5 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors14.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- other
- Model type
- mistral
- Access
- Public
- Created
- Feb 13, 2024
- Updated
- Mar 4, 2024
- SHA
- e646ec76
Try a prompt
Base models
Model card
From the Hugging Face model README
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
slices:
- sources:
- model: Test157t/Pasta-Sea-7b-128k
layer_range: [0, 32]
- model: Locutusque/Hercules-2.5-Mistral-7B
layer_range: [0, 32]
merge_method: slerp
base_model: Test157t/Pasta-Sea-7b-128k
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5
dtype: float16
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 69.03 |
| AI2 Reasoning Challenge (25-Shot) | 66.13 |
| HellaSwag (10-Shot) | 85.89 |
| MMLU (5-Shot) | 64.48 |
| TruthfulQA (0-shot) | 55.54 |
| Winogrande (5-shot) | 81.22 |
| GSM8k (5-shot) | 60.96 |
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- License
- other
- Created
- Feb 13, 2024
- Updated
- Mar 4, 2024