Downloads · 30 days
81
0% of all-time downloads
karakuri-ai/karakuri-lm-8x7b-chat-v0.1
karakuri-lm-8x7b-chat-v0.1 is a text generation model from karakuri-ai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Downloads · 30 days
81
0% of all-time downloads
All-time downloads
17.1K
Public
Parameters
46.7B
93.4 GB on disk
Likes
23
Public
Click a slice to open those files.
.safetensors93.4 GB · 100%
From the Hugging Face model README
![]()
[email protected]The prompt format has been changed from KARAKURI LM 70B Chat v0.1. Please make sure to follow the correct format. Otherwise, the model will generate sub-optimal outputs.
We use the following prompt template of multi-turn conversation in the Mistral format, which includes an encoded string of multiple attribute values.
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("karakuri-ai/karakuri-lm-8x7b-chat-v0.1")
messages = [
{"role": "system", "content": "System prompt"},
{"role": "user", "content": "User prompt"},
{"role": "assistant", "content": "Model response"},
{"role": "user", "content": "User prompt"},
]
tokenizer.apply_chat_template(messages, tokenize=False)
# <s>[INST] <<SYS>>
# System prompt
# <</SYS>>
#
# User prompt [ATTR] helpfulness: 4 correctness: 4 coherence: 4 complexity: 4 verbosity: 4 quality: 4 toxicity: 0 humor: 0 creativity: 0 [/ATTR] [/INST]Model response</s>[INST] User prompt [ATTR] helpfulness: 4 correctness: 4 coherence: 4 complexity: 4 verbosity: 4 quality: 4 toxicity: 0 humor: 0 creativity: 0 [/ATTR] [/INST]
The prompt template contains nine attributes. The first five are derived from HelpSteer, while the remaining four are derived from OASST2. The values are represented by integers ranging from 0 to 4, with 0 being the lowest and 4 being the highest.
If you want to change attribute values from the default values specified in the template, you can modify them to any values by adding the attribute values to the user messages:
messages = [
{"role": "user", "content": "User prompt", "helpfulness": 0, "complexity": 0},
]
tokenizer.apply_chat_template(messages, tokenize=False)
# <s>[INST] User prompt [ATTR] helpfulness: 0 correctness: 4 coherence: 4 complexity: 0 verbosity: 4 quality: 4 toxicity: 0 humor: 0 creativity: 0 [/ATTR] [/INST]
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
"karakuri-ai/karakuri-lm-8x7b-chat-v0.1",
torch_dtype="auto",
device_map="auto",
)
messages = [
{
"role": "user",
"content": "週末に日帰りで東京に遊びに行こうと思っています。日帰りなので、短時間で回れるおすすめの観光プランを教えてください。",
},
]
input_ids = tokenizer.apply_chat_template(
messages,
return_tensors="pt",
).to(model.device)
outputs = model.generate(input_ids, max_new_tokens=512)
tokenizer.decode(outputs[0][input_ids.shape[-1]:])
| Model | # Active Params | Alignment | MT-Bench-jp |
|---|---|---|---|
| Qwen1.5 72B Chat | 72B | DPO | 8.19 |
| KARAKURI LM 8x7B Chat v0.1 | 13B | SteerLM | 7.54 |
| Command R+ | 104B | - | 7.31 |
| Mixtral 8x7B Instruct v0.1 | 13B | DPO | 7.24 |
| Llama 3 70B Instruct | 70B | RLHF | 7.13 |
| KARAKURI LM 70B Chat v0.1 | 70B | SteerLM | 6.43 |
| Llama 2 70B Chat | 70B | RLHF | 5.23 |
@misc{karakuri_lm_8x7b_chat_v01,
author = { {KARAKURI} {I}nc. },
title = { {KARAKURI} {LM} 8x7{B} {C}hat v0.1 },
year = { 2024 },
url = { https://huggingface.co/karakuri-ai/karakuri-lm-8x7b-chat-v0.1 },
publisher = { Hugging Face },
journal = { Hugging Face repository }
}