Downloads · 30 days
0
Chinese-Vicuna/Chinese-Vicuna-lora-7b-belle-and-guanaco-4bit
Chinese-Vicuna-lora-7b-belle-and-guanaco-4bit is a machine learning model from Chinese-Vicuna. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as gpl-3.0.
This is a Chinese instruction-tuning lora checkpoint based on llama-7B from this repo's work. Specially, this is the 4bit version trained with qlora
Downloads · 30 days
0
Access
Public
Updated Jun 12, 2023
Repo size
25.5 MB
Likes
1
Public
Click a slice to open those files.
.bin16.8 MB · 66%
From the Hugging Face model README
This is a Chinese instruction-tuning lora checkpoint based on llama-7B from this repo's work. Specially, this is the 4bit version trained with qlora
You can use it like this:
from transformers import LlamaForCausalLM
from peft import PeftModel
model = LlamaForCausalLM.from_pretrained(
"decapoda-research/llama-7b-hf",
load_in_8bit=True,
torch_dtype=torch.float16,
device_map="auto",
)
model = PeftModel.from_pretrained(
model,
LORA_PATH, # specific checkpoint path from "Chinese-Vicuna/Chinese-Vicuna-lora-7b-belle-and-guanaco"
torch_dtype=torch.float16,
device_map={'': 0}
)