qwen2.5-0.5b-fish-signoff-lora

Qwen2.5 0.5B Fish Signoff LoRA

Overview

This repository contains a LoRA adapter trained on top of Qwen/Qwen2.5-0.5B-Instruct.

The goal of this experiment was to teach the model to end its assistant responses with the phrase:

So long, and thanks for all the fish

This project was created as a small educational experiment to better understand supervised fine-tuning, LoRA, PEFT, TRL, and the Hugging Face ecosystem.

Base Model

The adapter was trained on:

Qwen/Qwen2.5-0.5B-Instruct

This repository does not contain the full Qwen model weights.

The original base model is required in order to use this adapter.

Training

The model was fine-tuned using supervised fine-tuning (SFT) with a LoRA adapter implemented using Hugging Face PEFT.

The training loop was managed using TRL's SFTTrainer.

LoRA Configuration

• Rank (r): 8
• LoRA alpha: 16
• Target modules:
  • q_proj
  • k_proj
  • v_proj
  • o_proj
• LoRA dropout: 0.1
• Bias: none
• Task type: Causal Language Modeling

Training Configuration

• Epochs: 3
• Per-device training batch size: 2
• Gradient accumulation steps: 4
• Effective batch size: 8
• Learning rate: 2e-4
• Learning rate scheduler: cosine
• Precision: FP16
• Maximum sequence length: 512
• Assistant-only loss: enabled

Dataset

The training dataset consisted of short conversational examples using user and assistant messages.

Every assistant response in the training data ended with:

So long, and thanks for all the fish

The purpose of the dataset was not to teach the model new factual knowledge, but to demonstrate how fine-tuning can modify a model's response behavior.

Usage

Install the required libraries:

pip install torch transformers peft

Load the original Qwen model and attach this LoRA adapter:

import torch

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel


base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-0.5B-Instruct",
    dtype=torch.float16,
    device_map="auto",
)

model = PeftModel.from_pretrained(
    base_model,
    "YehPel/qwen2.5-0.5b-fish-signoff-lora",
)

tokenizer = AutoTokenizer.from_pretrained(
    "Qwen/Qwen2.5-0.5B-Instruct"
)

Generate a Response

messages = [
    {
        "role": "user",
        "content": "What is the capital of Japan?"
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(
    text,
    return_tensors="pt"
).to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=80
    )

response = tokenizer.decode(
    outputs[0],
    skip_special_tokens=True
)

print(response)

An example response after applying the adapter is:

system
You are Qwen, created by Alibaba Cloud. You are a helpful assistant.
user
What is the capital of Japan?
assistant
Tokyo. So long, and thanks for all the fish

Repository Contents

The important adapter files in this repository are:

adapter_config.json
adapter_model.safetensors

adapter_config.json describes the LoRA configuration and its relationship to the base model.

adapter_model.safetensors contains the trained LoRA parameters.

The full Qwen2.5-0.5B-Instruct weights are not included in this repository.

License

This adapter is based on Qwen/Qwen2.5-0.5B-Instruct, Copyright 2024 Alibaba Cloud, which is licensed under the Apache License 2.0.

This repository contains only the LoRA adapter weights and does not redistribute the original Qwen base model weights.