Downloads · 30 days
0
ymatsuda-97/llm-jp-3-13B-finetune-ver4-CB_lora
llm-jp-3-13B-finetune-ver4-CB_lora is a machine learning model from ymatsuda-97. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
- Developed by: ymatsuda-97 - License: apache-2.0 - Finetuned from model : llm-jp/llm-jp-3-13B
Downloads · 30 days
0
Access
Public
Updated Dec 16, 2024
Repo size
501 MB
Likes
0
Public
Click a slice to open those files.
.safetensors501 MB · 99%
From the Hugging Face model README
This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
Use the code below to get started with the model.
from unsloth import FastLanguageModel
from peft import PeftModel
import torch
import json
from tqdm import tqdm
model_id = "llm-jp/llm-jp-3-13b"
adapter_id = "" # input this model_id
HF_TOKEN = ""
dtype = None
load_in_4bit = True
model, tokenizer = FastLanguageModel.from_pretrained(
model_name=model_id,
dtype=dtype,
load_in_4bit=load_in_4bit,
trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, adapter_id, token=HF_TOKEN)
datasets = # please download datasets
FastLanguageModel.for_inference(model)
results = []
for dt in tqdm(datasets):
input = dt["input"]
prompt = f"""### 指示\n{input}\n### 回答\n"""
inputs = tokenizer([prompt], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, use_cache=True, do_sample=False, repetition_penalty=1.2)