Downloads · 30 days
13
4% of all-time downloads
kim512/Llama-3-70b-Arimas-story-RP-V1.6-4.0bpw-h6-exl2
Llama-3-70b-Arimas-story-RP-V1.6-4.0bpw-h6-exl2 is a text generation model from kim512. Use it when you need the model to write or continue text. It is set up for transformers.
3.00 bits per weight 3.50 bits per weight 4.00 bits per weight 4.50 bits per weight
Downloads · 30 days
13
4% of all-time downloads
All-time downloads
303
Public
Repo size
37.2 GB
Likes
1
Public
Click a slice to open those files.
.safetensors37.2 GB · 100%
From the Hugging Face model README
3.00 bits per weight
3.50 bits per weight
4.00 bits per weight
4.50 bits per weight
6.00 bits per weight
8.00 bits per weight
Created using the defaults from exllamav2 1.4.0 convert.py
3.0bpw to 6.0bpw head bits = 6
8.0bpw head bits = 8
length = 8192
dataset rows = 200
measurement rows = 32
measurement length = 8192
Llama-3-70b-Arimas-story-RP-V1.6
This is a merge of pre-trained language models created using mergekit.
I Greatly expanded the amount of models used in this merge, experimented a lot with different idea's. This version feels a lot more convincing than V1.5 Hopefully the long context window will also remain strong after Quants. Because of the many merges switched back from BFloat to Float. Tried breadcrums without the Ties, that went very poorly.
This model was merged using the breadcrumbs_ties merge method using I:\Llama-3-70B-Instruct-Gradient-262k as a base.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
models:
- model: \Llama-3-70B-Instruct-Gradient-262k
parameters:
weight: 0.25
density: 0.90
gamma: 0.01
- model: \Meta-LLama-3-Cat-Smaug-LLama-70b
parameters:
weight: 0.28
density: 0.90
gamma: 0.01
- model: \Llama-3-Lumimaid-70B-v0.1-alt
parameters:
weight: 0.15
density: 0.90
gamma: 0.01
- model: \Tess-2.0-Llama-3-70B-v0.2
parameters:
weight: 0.06
density: 0.90
gamma: 0.01
- model: \Smaug-Llama-3-70B-Instruct
parameters:
weight: 0.04
density: 0.90
gamma: 0.01
- model: \Llama-3-70B-Synthia-v3.5
parameters:
weight: 0.05
density: 0.90
gamma: 0.01
- model: \Llama-3-70B-Instruct-Gradient-524k
parameters:
weight: 0.03
density: 0.90
gamma: 0.01
- model: \Meta-LLama-3-Cat-A-LLama-70b
parameters:
weight: 0.14
density: 0.90
gamma: 0.01
merge_method: breadcrumbs_ties
base_model: I:\Llama-3-70B-Instruct-Gradient-262k
dtype: float16