Downloads · 30 days
0
izumi-lab/llama-7b-japanese-lora-v0-5ep
llama-7b-japanese-lora-v0-5ep is a machine learning model from izumi-lab. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-sa-4.0.
This repo contains a low-rank adapter for LLaMA-7b fit on the llm-japanese-dataset dataset.
Downloads · 30 days
0
Access
Public
Updated Jun 23, 2023
Repo size
8.4 MB
Likes
3
Public
Click a slice to open those files.
.bin8.4 MB · 100%
From the Hugging Face model README
This repo contains a low-rank adapter for LLaMA-7b fit on the llm-japanese-dataset dataset.
This version of the weights was trained with the following hyperparameters:
import torch
from transformers import LlamaForCausalLM, LlamaTokenizer
from peft import PeftModel
base_model = "decapoda-research/llama-7b-hf"
# Please note that the special license of decapoda-research/llama-7b-hf is applied.
model = LlamaForCausalLM.from_pretrained(base_model, torch_dtype=torch.float16)
tokenizer = LlamaTokenizer.from_pretrained(base_model)
model = PeftModel.from_pretrained(
model,
"izumi-lab/llama-7b-japanese-lora-v0",
torch_dtype=torch.float16,
)
To see more latest information, please go to llm.msuzuki.me.
Citation:
@preprint{Suzuki2023-llmj,
title={{日本語インストラクションデータを用いた対話可能な日本語大規模言語モデルのLoRAチューニング}},
author={鈴木 雅弘 and 平野 正徳 and 坂地 泰紀},
doi={10.51094/jxiv.422},
archivePrefix={Jxiv},
year={2023}
}
If you have any inquiries, such as joint research, data provision, various types of support, please email to [email protected] .