llm-jp-3-13b-it_33

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: Ryuji Fujieda
• Model type: LoRAアダプター

Model Sources [optional]

<!-- Provide the basic links for the model. -->

• Repository: 　https://huggingface.co/llm-jp/llm-jp-3-13b

How to Get Started with the Model

Use the code below to get started with the model.

from unsloth import FastLanguageModel
from peft import PeftModel
from tqdm import tqdm
# ベースとなるモデルと学習したLoRAのアダプタ（Hugging FaceのIDを指定）。
model_id = "llm-jp/llm-jp-3-13b"
adapter_id = "ryukin777/llm-jp-3-13b-it_33"
dtype = None # Noneにしておけば自動で設定
load_in_4bit = True # 今回は13Bモデルを扱うためTrue

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name=model_id,
    dtype=dtype,
    load_in_4bit=load_in_4bit,
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, adapter_id)

FastLanguageModel.for_inference(model)

input = "LLMとはなんですか？"

prompt = f"""### 指示\n{input} 簡潔に回答してください。\n### 回答\n"""

inputs = tokenizer([prompt], return_tensors = "pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens = 512, use_cache = True, do_sample=False, repetition_penalty=1.2)
prediction = tokenizer.decode(outputs[0], skip_special_tokens=True).split('\n### 回答')[-1]
print('prediction')

Training Details

SFT

Training Data

Ichikara datasets ELyza tasks100 https://huggingface.co/datasets/DeL-TaiseiOzaki/Tengentoppa-sft-v1.0