Sakura-SOLAR-Instruct-EXL2

Sakura-SOLAR-Instruct

• Model creator: KyujinHan
• Original model: Merged AGI 7B
• Is a merge of:
  • VAGOsolutions/SauerkrautLM-SOLAR-Instruct
  • upstage/SOLAR-10.7B-Instruct-v1.0

Quantizations

Measured using ExLlamav2_HF and 4096 max_seq_len with Oobabooga's Text Generation WebUI.

I also provided zipped quantization because a lot of people find gguf single download convenient. Zipped quantization is relatively smaller in size to download. After extracted, you can use the model folder as usual.

Use TheBloke's 4bit-32g quants (7.4GB VRAM usage) if you have 8GB cards.

Calibration Dataset

• argilla/distilabel-math-preference-dpo
  • Training dataset of VAGOsolutions/SauerkrautLM-SOLAR-Instruct

Prompt template: Orca-Hashes

From TheBloke

### System:
{system_message}

### User:
{prompt}

### Assistant:

If you use Oobabooga's Chat tab

From my testing, the template "Orca-Mini" or any of the Orca templates produced the best result. Feel free to leave a suggestion if you know better.

Original Info

Sakura-SOLAR-Instruct

<img src='./sakura.png' width=512>

(주)미디어그룹사람과숲과 (주)마커의 LLM 연구 컨소시엄에서 개발된 모델입니다

Model Details

Model Developers Kyujin Han (kyujinpy)

Method
Using Mergekit.
I shared the information about my model. (training and code)
Please see: ⭐Sakura-SOLAR.

Blog

• Sakura-SOLAR 모델 제작 과정 및 후기.

Model Benchmark

Open leaderboard

• Follow up as link.

Rank1 2023.12.27 PM 11:50

Implementation Code

### KO-Platypus
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "kyujinpy/Sakura-SOLAR-Instruct"
OpenOrca = AutoModelForCausalLM.from_pretrained(
        repo,
        return_dict=True,
        torch_dtype=torch.float16,
        device_map='auto'
)
OpenOrca_tokenizer = AutoTokenizer.from_pretrained(repo)

---