Downloads · 30 days
4
7% of all-time downloads
IceLemonTeaRP-32k-7b-7.1bpw-exl2
Natkituwu/IceLemonTeaRP-32k-7b-7.1bpw-exl2
IceLemonTeaRP-32k-7b-7.1bpw-exl2 is a text generation model from Natkituwu. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-4.0.
7.1bpw-exl2 quant of icefog72/IceLemonTeaRP-32k-7b
All-time downloads
55
Public
Repo size
6.6 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors6.6 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- cc-by-nc-4.0
- Model type
- mistral
- Access
- Public
- Created
- Apr 9, 2024
- Updated
- Apr 9, 2024
- SHA
- aea08ba9
Try a prompt
Base models
Model card
From the Hugging Face model README
IceLemonTeaRP-32k-7b-8.0bpw-h6-exl2
7.1bpw-exl2 quant of icefog72/IceLemonTeaRP-32k-7b
Tested 16k context at 4bit cache max for windows and 32k context at 4bit cache max for linux (RTX 4060 8GB)
perfect for 8GB VRAM users that want as much quality as possible with alot of context.
Merge Details
Merge Method
This model was merged using the SLERP merge method.
Models Merged
The following models were included in the merge:
- Kunokukulemonchini-32k-7b
- Mixtral_AI_Cyber_3.m1-BigL
Configuration
The following YAML configuration was used to produce this model:
slices:
- sources:
- model: Mixtral_AI_Cyber_3.m1-BigL
layer_range: [0, 32]
- model: Kunokukulemonchini-32k-7b
layer_range: [0, 32]
merge_method: slerp
base_model: Kunokukulemonchini-32k-7b
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5
dtype: float16
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- License
- cc-by-nc-4.0
- Created
- Apr 9, 2024
- Updated
- Apr 9, 2024