Downloads · 30 days
35
0% of all-time downloads
ABX-AI/Silver-Sun-11B
Silver-Sun-11B is a text generation model from ABX-AI. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
Downloads · 30 days
35
0% of all-time downloads
All-time downloads
7.1K
Public
Parameters
10.7B
21.5 GB on disk
Likes
5
Public
Click a slice to open those files.
.safetensors21.5 GB · 100%
From the Hugging Face model README

I'd like to experiment more with merging 11B, hopefully adding more options of this weight class. This model is good at writing and reasoning, with a preference for more profane NSFW language when the appropriate cards are used. I've been having fun with it so far, although at times it can be a bit blunt, although some may prefer that. It's also highly uncensored.
Works best with Alpaca instruction presets.
This is a merge of pre-trained language models created using mergekit.
This model was merged using the SLERP merge method.
The following models were included in the merge:
[!NOTE] A mixture of Sao10K/Solstice-11B-v1 and saishf/Fimbulvetr-Kuro-Lotus-10.7B
Detailed Results + Failed GSM8K
[!NOTE] I had to remove GSM8K from the results and manually re-average the rest. GSM8K failed due to some issue with formatting, which is not experienced during practical usage. By removing the GSM8K score, the average is VERY close to upstage/SOLAR-10.7B-v1.0 (74.20), which would make sense. Feel free to ignore the actual average and use the other scores individually for reference.
| Metric | Value |
|---|---|
| Avg. | 74.13 |
| AI2 Reasoning Challenge (25-Shot) | 69.80 |
| HellaSwag (10-Shot) | 87.91 |
| MMLU (5-Shot) | 66.90 |
| TruthfulQA (0-shot) | 61.89 |
| Winogrande (5-shot) | 84.14 |
The following YAML configuration was used to produce this model:
slices:
- sources:
- model: ABX-AI/Solstice-FKL-11B
layer_range: [0, 48]
- model: Himitsui/Kaiju-11B
layer_range: [0, 48]
merge_method: slerp
base_model: ABX-AI/Solstice-FKL-11B
parameters:
t:
- filter: self_attn
value: [0.6, 0.7, 0.8, 0.9, 1]
- filter: mlp
value: [0.4, 0.3, 0.2, 0.1, 0]
- value: 0.5
dtype: bfloat16