Downloads · 30 days
0
shigedon/llm-jp-3-13b-finetune-16-Dec-num-02-2024
llm-jp-3-13b-finetune-16-Dec-num-02-2024 is a machine learning model from shigedon. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
Downloads · 30 days
0
Access
Public
Updated Dec 26, 2024
Repo size
250 MB
Likes
0
Public
Click a slice to open those files.
.safetensors250 MB · 97%
From the Hugging Face model README
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
Use the code below to get started with the model.
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
# Fine-Tuning (LoRa)
pip install torch==2.5.0 torchvision==0.20.0 torchaudio==2.5.0 --index-url https://download.pytorch.org/whl/cu121
pip install -U pip
#pip install -U transformers==4.43.0
pip install -U transformers
#pip install -U tokenizers==0.19.1
pip install -U tokenizers
pip install -U bitsandbytes
pip install -U accelerate
pip install -U datasets
pip install -U peft
pip install -U trl
from transformers import (
AutoModelForCausalLM,
AutoTokenizer,
BitsAndBytesConfig,
)
from peft import PeftModel
import torch
from tqdm import tqdm
import json
HF_TOKEN = ""
model_id = "llm-jp/llm-jp-3-13b"
adapter_id = "shigedon/llm-jp-3-13b-finetune-16-Dec-num-02-2024"
# QLoRA config
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
)
# Load model
model = AutoModelForCausalLM.from_pretrained(
model_id,
quantization_config=bnb_config,
device_map="auto",
token = HF_TOKEN
)
# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True, token = HF_TOKEN)
# 元のモデルにLoRAのアダプタを統合。
model = PeftModel.from_pretrained(model, adapter_id, token = HF_TOKEN)
# データセットの読み込み。
datasets = []
with open("./elyza-tasks-100-TV_0.jsonl", "r") as f:
item = ""
for line in f:
line = line.strip()
item += line
if item.endswith("}"):
datasets.append(json.loads(item))
item = ""
results = []
for data in tqdm(datasets):
input = data["input"]
prompt = f"""### 指示
{input}
### 回答
"""
input_ids = tokenizer(prompt, return_tensors="pt",return_token_type_ids=False,).to(model.device)
outputs = model.generate(**input_ids, max_new_tokens=512, do_sample=False, repetition_penalty=1.2,)
output = tokenizer.decode(outputs[0][input_ids.input_ids.size(1):], skip_special_tokens=True)
results.append({"task_id": data["task_id"], "input": input, "output": output})
# llmjp
results = []
for data in tqdm(datasets):
input = data["input"]
prompt = f"""### 指示
{input}
### 回答
"""
tokenized_input = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt").to(model.device)
attention_mask = torch.ones_like(tokenized_input)
with torch.no_grad():
outputs = model.generate(
tokenized_input,
attention_mask=attention_mask,
max_new_tokens=100,
do_sample=False,
repetition_penalty=1.2,
pad_token_id=tokenizer.eos_token_id
)[0]
output = tokenizer.decode(outputs[tokenized_input.size(1):], skip_special_tokens=True)
results.append({"task_id": data["task_id"], "input": input, "output": output})
# jsonl
import re
jsonl_id = re.sub(".*/", "", adapter_id)
with open(f"./{jsonl_id}-outputs.jsonl", 'w', encoding='utf-8') as f:
for result in results:
json.dump(result, f, ensure_ascii=False) # ensure_ascii=False for handling non-ASCII characters
f.write('\n')