Downloads · 30 days
6
17% of all-time downloads
RichardErkhov/minlik_-_chinese-alpaca-plus-7b-merged-8bits
minlik_-_chinese-alpaca-plus-7b-merged-8bits is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
6
17% of all-time downloads
All-time downloads
35
Public
Parameters
6.9B
7.3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors7.3 GB · 100%
How the weights are stored.
I86.5B · 94%
From the Hugging Face model README
Quantization made by Richard Erkhov.
chinese-alpaca-plus-7b-merged - bnb 8bits
加入中文词表并继续预训练中文Embedding,并在此基础上继续使用指令数据集finetuning,得到的中文Alpaca-plus模型。
详情可参考:https://github.com/ymcui/Chinese-LLaMA-Alpaca/releases/tag/v3.0
pip install sentencepiece
pip install transformers>=4.28.0
import torch
import transformers
from transformers import LlamaTokenizer, LlamaForCausalLM
def generate_prompt(text):
return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{text}
### Response:"""
tokenizer = LlamaTokenizer.from_pretrained('minlik/chinese-alpaca-plus-7b-merged')
model = LlamaForCausalLM.from_pretrained('minlik/chinese-alpaca-plus-7b-merged').half().to('cuda')
model.eval()
text = '第一个登上月球的人是谁?'
prompt = generate_prompt(text)
input_ids = tokenizer.encode(prompt, return_tensors='pt').to('cuda')
with torch.no_grad():
output_ids = model.generate(
input_ids=input_ids,
max_new_tokens=128,
temperature=1,
top_k=40,
top_p=0.9,
repetition_penalty=1.15
).cuda()
output = tokenizer.decode(output_ids[0], skip_special_tokens=True)
print(output.replace(prompt, '').strip())