SolarMaid-v0.1.1-exl2

Exl2 version of Undi95/SolarMaid-v0.1.1

branch

3bh8 : 3bpw h8
3.5bh8 : 3.5bpw h8
3.8bh8 : 3.8bpw h8
4bh8 : 4bpw h8
6bh8 : 6bpw h8
8bh8 : 8bpw h8

Using ThePile 0007.parquet as dataset

Quantization settings : python convert.py -i models/Undi95_SolarMaid-v0.1.1 -o SolarMaid-v0.1.1-temp -cf SolarMaid-v0.1.1-8bpw-h8-exl2 -c 0007.parquet -l 8192 -b 8 -hb 8 -ml 8192
python convert.py -i models/Undi95_SolarMaid-v0.1.1 -o SolarMaid-v0.1.1-temp2 -cf SolarMaid-v0.1.1-6bpw-h8-exl2 -c 0007.parquet -l 8192 -b 6 -hb 8 -m SolarMaid-v0.1.1-temp/measurement.json -ml 8192
python convert.py -i models/Undi95_SolarMaid-v0.1.1 -o SolarMaid-v0.1.1-temp3 -cf SolarMaid-v0.1.1-4bpw-h8-exl2 -c 0007.parquet -l 8192 -b 4 -hb 8 -m SolarMaid-v0.1.1-temp/measurement.json -ml 8192
python convert.py -i models/Undi95_SolarMaid-v0.1.1 -o SolarMaid-v0.1.1-temp4 -cf SolarMaid-v0.1.1-3bpw-h8-exl2 -c 0007.parquet -l 8192 -b 3.8 -hb 8 -m SolarMaid-v0.1.1-temp/measurement.json -ml 8192
python convert.py -i models/Undi95_SolarMaid-v0.1.1 -o SolarMaid-v0.1.1-temp5 -cf SolarMaid-v0.1.1-2.4bpw-h8-exl2 -c 0007.parquet -l 8192 -b 3.5 -hb 8 -m SolarMaid-v0.1.1-temp/measurement.json -ml 8192
python convert.py -i models/Undi95_SolarMaid-v0.1.1 -o SolarMaid-v0.1.1-temp5 -cf SolarMaid-v0.1.1-2.4bpw-h8-exl2 -c 0007.parquet -l 8192 -b 3 -hb 8 -m SolarMaid-v0.1.1-temp/measurement.json -ml 8192

below this line is original readme

SolarMaid is a model made primarely for ERP (Erotic Roleplay) without being too dumb, it use 40 out of 48 layers of "SOLAR Instruct 10.7B", and 32 out of 32 layers of "Noromaid 7B v0.1.1".

Despite being a little unstable, benchmark stay high, and can output some hot sh*t.

I'm working on making it more stable.

<!-- description start -->

Description

This repo contains fp16 files of SolarMaid v0.1.1.

<!-- description end --> <!-- description start -->

Models used

• upstage/SOLAR-10.7B-Instruct-v1.0
• NeverSleep/Noromaid-7b-v0.1.1

<!-- description end --> <!-- prompt-template start -->

Prompt template: Alpaca

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{prompt}

### Response:

Prompt template: Custom (SOLAR)

### User:
{prompt}

### Assistant:

If you want to support me, you can here.