llm-jp-3-13b-finetune

Model Card for llm-jp-3-13b-finetune

This model is a fine-tuned version of llm-jp/llm-jp-3-13b, using the Hugging Face Transformers library, created for the submission of the final exam in "大規模言語モデル Deep Learning 応用講座 2024 Fall".

Model Details

This is quantized and fine-tuned model from llm-jp/llm-jp-3-13b.

What have been done:

• 4b quantization
• LoRA fine-tuning with "Ichikara Instruction" dataset

Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: @sugiyama34
• Funded by [optional]: -
• Shared by [optional]: -
• Model type: AutoModelForCausalLM
• Language(s) (NLP): Japanese
• License: CC-BY-NC-SA, Apache License, Version 2.0
• Finetuned from model [optional]: llm-jp/llm-jp-3-13b

instruction dataset "Ichikara Instruction" is given by: 関根聡, 安藤まや, 後藤美知子, 鈴木久美, 河原大輔, 井之上直也, 乾健太郎. ichikara-instruction: LLMのための日本語インストラクションデータの構築. 言語処理学会第30回年次大会(2024)

Model Sources [optional]

<!-- Provide the basic links for the model. -->

• Repository: -
• Paper [optional]: -
• Demo [optional]: -

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Only for submission of final exam.

Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

Only for submission of final exam.

Downstream Use [optional]

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

Only for submission of final exam.

Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

Anything other than my submission of final exam.

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

I'm not sure right now, I need to study hard.

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

松尾研 LLM 講座 2024 Fall の最終課題の評価を動かすためのコードは以下です。

In the Google Colab. env. with GPU, use the code below to get started with the model.

When running the following code,

• replace "YOUR TOKEN" below with your hugging face token,
• replace PATH TO YOUR DATA below with the path to the evaluation data

After running, we can get jsonl file in the "files" in the Colab.

# install necessary libraries
!pip install -U pip
!pip install -U transformers
!pip install -U bitsandbytes

# import
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
)
import os, torch, gc, re
import bitsandbytes as bnb
import json
from tqdm import tqdm

# setting model id
base_model_id = "llm-jp/llm-jp-3-13b"
new_model_id = "llm-jp-3-13b-finetune" # evaluation に利用するモデルの名称

# YOUR HUGGING FACE TOKEN HERE
HF_TOKEN = "YOUR TOKEN"

# BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

# load model
model_name = 'sugiyama34/llm-jp-3-13b-finetune'
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=bnb_config,
    device_map='auto'
)

# load data
datasets = []

elyza_tasks_100_tv_0_path = "PATH TO YOUR DATA"

with open(elyza_tasks_100_tv_0_path, "r") as f:
    item = ""
    for line in f:
      line = line.strip()
      item += line
      if item.endswith("}"):
        datasets.append(json.loads(item))
        item = ""

# inference
results = []
for data in tqdm(datasets):

  input = data["input"]

  prompt = f"""### 指示
  {input}
  ### 回答
  """

  tokenized_input = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt").to(model.device)
  attention_mask = torch.ones_like(tokenized_input)

  with torch.no_grad():
      outputs = model.generate(
          tokenized_input,
          attention_mask=attention_mask,
          max_new_tokens=100,
          do_sample=False,
          repetition_penalty=1.2,
          pad_token_id=tokenizer.eos_token_id
      )[0]
  output = tokenizer.decode(outputs[tokenized_input.size(1):], skip_special_tokens=True)

  results.append({"task_id": data["task_id"], "input": input, "output": output})

# export output data
jsonl_id = re.sub(".*/", "", new_model_id)
with open(f"./{jsonl_id}-outputs.jsonl", 'w', encoding='utf-8') as f:
    for result in results:
        json.dump(result, f, ensure_ascii=False)  # ensure_ascii=False for handling non-ASCII characters
        f.write('\n')

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

ichikara-instruction: LLMのための日本語インストラクションデータ – LIAT-AIP homepage instruction dataset "Ichikara Instruction" is given by: 関根聡, 安藤まや, 後藤美知子, 鈴木久美, 河原大輔, 井之上直也, 乾健太郎. ichikara-instruction: LLMのための日本語インストラクションデータの構築. 言語処理学会第30回年次大会(2024)

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

I'm not sure right now, I need to study hard.

Preprocessing [optional]

The data of ichikara instruction dataset is transformed into the following format:

prompt = """### 指示
{input}
### 回答
{output}"""

And then feeded into LoRA training.

Training Hyperparameters

I'm not sure right now, I need to study hard.

• Training regime: [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

I'm not sure right now, I need to study hard.

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

I'm not sure right now, I need to study hard.

Testing Data, Factors & Metrics

Testing Data

<!-- This should link to a Dataset Card if possible. -->

I'm not sure right now, I need to study hard.

Factors

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

I'm not sure right now, I need to study hard.

Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

I'm not sure right now, I need to study hard.

Results

I'm not sure right now, I need to study hard.

Summary

I'm not sure right now, I need to study hard.

Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

I'm not sure right now, I need to study hard.

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

I'm not sure right now, I need to study hard.

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [More Information Needed]
• Hours used: [More Information Needed]
• Cloud Provider: [More Information Needed]
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

I'm not sure right now, I need to study hard.

Compute Infrastructure

I'm not sure right now, I need to study hard.

Hardware

I'm not sure right now, I need to study hard.

Software

I'm not sure right now, I need to study hard.

Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

No paper for this work.

Glossary [optional]

<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->

not yet written

More Information [optional]

not yet written

Model Card Authors [optional]

Satoshi Sugiyama

Model Card Contact

not given here