Downloads · 30 days
16
10% of all-time downloads
retrieva-jp/Llama-3-Swallow-8B-Instruct-v0.1-kokoroe
Llama-3-Swallow-8B-Instruct-v0.1-kokoroe is a text generation model from retrieva-jp. Use it when you need the model to write or continue text. The card lists the license as llama3.
Downloads · 30 days
16
10% of all-time downloads
All-time downloads
153
Public
Parameters
8B
16.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors16.1 GB · 100%
From the Hugging Face model README
Built with Meta Llama 3
The Llama-3-Swallow-8B-Instruct-v0.1-kokoroe is a large language model fine-tuned to follow instructions in Japanese, with safety tuning applied to enhance response appropriateness. This model is based on tokyotech-llm/Llama-3-Swallow-8B-Instruct-v0.1.
This section describes three ways to use the model:
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "retrieva-jp/Llama-3-Swallow-8B-Instruct-v0.1-kokoroe"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", torch_dtype=torch.bfloat16)
chat = [
{"role": "system", "content": "あなたは誠実で優秀な日本人のアシスタントです。"},
{"role": "user", "content": "自然言語処理とは何か"},
]
tokenized_input = tokenizer.apply_chat_template(chat, add_generation_prompt=True, tokenize=True, return_tensors="pt").to(model.device)
with torch.no_grad():
output = model.generate(
tokenized_input,
max_new_tokens=100,
do_sample=True,
top_p=0.9,
temperature=0.6,
)[0]
print(tokenizer.decode(output))
$ vllm serve --model retrieva-jp/Llama-3-Swallow-8B-Instruct-v0.1-kokoroe --port 8000
$ curl -X POST http://localhost:8000/generate \
-H "Content-Type: application/json" \
-d '{
"prompt": [
{"role": "system", "content": "あなたは誠実で優秀な日本人のアシスタントです。"},
{"role": "user", "content": "自然言語処理とは何か"}
],
"max_new_tokens": 100,
"do_sample": true,
"top_p": 0.9,
"temperature": 0.6
}'
Satoru Katsumata
pr[at]retrieva.jp