llm-jp-3-13b-finetune1

Model Card for Model ID

Model Details

Model Description

• Developed by: maktag
• Language(s) (NLP): Japanese
• Finetuned from model [optional]: llm-jp/llm-jp-3-13b

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForCausalLM

# Load the fine-tuned model and tokenizer
model_id = "your-username/llm-jp-3-13b-finetune1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Prepare your input
prompt = """### 指示
以下の文章を英語に翻訳してください:
猫はかわいいです
### 回答
"""

# Tokenize and generate
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    inputs["input_ids"],
    max_new_tokens=100,
    do_sample=False,
    repetition_penalty=1.2,
    pad_token_id=tokenizer.eos_token_id
)

# Decode and print the response
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

[More Information Needed]

Training Details

• Fine-Tuning Framework: LoRA-based PEFT (Parameter-Efficient Fine-Tuning).
• Dataset: Proprietary Japanese instruction-following dataset.
• Sequence Length: 512 tokens.
• Hyperparameters:
  • Batch size: 16
  • Learning rate: 2e-5
  • Epochs: 3

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

• elyza/ELYZA-tasks-100
• Ichikara Instruction