llm-jp-3-13b-it_lora

Uploaded model

• Developed by: joijwoafeij
• License: apache-2.0
• Finetuned from model : llm-jp/llm-jp-3-13b

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>

Fine-Tuning Template with Unsloth

このテンプレートは、Google Colab 無料版や NVIDIA L4/T4 GPU などの環境で動作する Fine-Tuning 用コード。Unsloth を使用し、簡易に高速な学習が可能。

---

概要

• モデル: llm-jp/llm-jp-3-13b (4bit量子化/qLoRA設定)
• データ: 日本語インストラクションデータセット ichikara-instruction-003-001-1.json
• 学習: 指定のプロンプト形式で SFT (Supervised Fine-Tuning) を実施

---

環境設定

Conda (Omnicampus環境など)

wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3-$(uname)-$(uname -m).sh
export PATH=/root/miniforge3/bin:$PATH
conda init
conda create --name unsloth_env python=3.10 pytorch-cuda=12.1 pytorch cudatoolkit xformers -c pytorch -c nvidia -c xformers -y
conda activate unsloth_env
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"

from unsloth import FastLanguageModel
import torch

model_id = "llm-jp/llm-jp-3-13b"
new_model_id = "llm-jp-3-13b-it"
max_seq_length = 512

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name=model_id,
    dtype=None,
    load_in_4bit=True,
    trust_remote_code=True,
)

model = FastLanguageModel.get_peft_model(
    model,
    r=32,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    use_gradient_checkpointing="unsloth",
    max_seq_length=max_seq_length,
)

from datasets import load_dataset

dataset = load_dataset("json", data_files="./ichikara-instruction-003-001-1.json")
prompt = """### 指示
{}
### 回答
{}"""
EOS_TOKEN = tokenizer.eos_token

def formatting_prompts_func(examples):
    text = prompt.format(examples["text"], examples["output"]) + EOS_TOKEN
    return {"formatted_text": text}

dataset = dataset.map(formatting_prompts_func, num_proc=4)

from trl import SFTTrainer
from transformers import TrainingArguments
from unsloth import is_bfloat16_supported

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset["train"],
    max_seq_length=max_seq_length,
    dataset_text_field="formatted_text",
    packing=False,
    args=TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        num_train_epochs=1,
        logging_steps=10,
        save_steps=100,
        save_total_limit=2,
        learning_rate=2e-4,
        fp16=not is_bfloat16_supported(),
        bf16=is_bfloat16_supported(),
        group_by_length=True,
        seed=3407,
        output_dir="outputs",
    ),
)

trainer.train()

model.push_to_hub_merged(
    new_model_id + "_lora",
    tokenizer=tokenizer,
    save_method="lora",
    token="YOUR_HF_TOKEN",
    private=True,
)