Downloads · 30 days
97
14% of all-time downloads
Epiculous/Fett-uccine-Long-Noodle-7B-120k-Context
Fett-uccine-Long-Noodle-7B-120k-Context is a text generation model from Epiculous. Use it when you need the model to write or continue text. It is set up for transformers.
This is a merge of pre-trained language models created using mergekit.
Downloads · 30 days
97
14% of all-time downloads
All-time downloads
698
Public
Parameters
7.2B
14.5 GB on disk
Likes
22
Public
Click a slice to open those files.
.safetensors14.5 GB · 100%
From the Hugging Face model README
This is a merge of pre-trained language models created using mergekit.
A merge with Fett-uccine and Mistral Yarn 120k ctx.
Credit to Nitral for the merge script and idea.
This model was merged using the SLERP merge method.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
slices:
- sources:
- model: Z:\ModelColdStorage\Fett-uccine-7B
layer_range: [0, 32]
- model: Z:\ModelColdStorage\Yarn-Mistral-7b-128k
layer_range: [0, 32]
merge_method: slerp
base_model: Z:\ModelColdStorage\Fett-uccine-7B
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5
dtype: bfloat16