Downloads · 30 days
71
46% of all-time downloads
onurerkan/qwen2.5-0.5b-alpaca-lora-demo
qwen2.5-0.5b-alpaca-lora-demo is a text generation model from onurerkan. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
An experiment in style transfer via LoRA: how much does 500 examples and ~16 seconds of training change the output character of an already instruction-tuned model?
Downloads · 30 days
71
46% of all-time downloads
All-time downloads
153
Public
Repo size
46.7 MB
Likes
0
Public
Click a slice to open those files.
.safetensors35.2 MB · 76%
From the Hugging Face model README
An experiment in style transfer via LoRA: how much does 500 examples and ~16 seconds of training change the output character of an already instruction-tuned model?
Answer: noticeably. Qwen2.5-Instruct's polished, markdown-heavy responses shift toward the terser, plain-text Alpaca/GPT-3 idiom. Core knowledge and reasoning remain unchanged, as expected for this dataset size.
Adapter weights and full training config included for reproduction.
The adapter shifts the model's output style toward the Alpaca dataset's characteristic tone:
Underlying knowledge, reasoning, and multilingual behavior remain essentially unchanged — those would require a different scale of data and a different training objective.
| Parameter | Value |
|---|---|
LoRA rank (r) | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Epochs | 1 |
| Effective batch size | 8 (4 × 2 grad accumulation) |
| Learning rate | 2e-4 |
| LR scheduler | cosine with 10% warmup |
| Precision | bfloat16 |
completion_only_loss | True |
| Seed | 42 |
| Training time | ~16 seconds on NVIDIA RTX PRO 6000 Blackwell |
| Stack | transformers 5.5.4, peft 0.19.1, trl 1.1.0, datasets 4.8.4 |
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
BASE = "Qwen/Qwen2.5-0.5B-Instruct"
ADAPTER = "onurerkan/qwen2.5-0.5b-alpaca-lora-demo"
tokenizer = AutoTokenizer.from_pretrained(ADAPTER)
base = AutoModelForCausalLM.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, ADAPTER)
messages = [{"role": "user", "content": "Give three tips for staying healthy."}]
inputs = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_tensors="pt",
return_dict=True,
).to(model.device)
with torch.no_grad():
out = model.generate(**inputs, max_new_tokens=150, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Prompt: Give three tips for staying healthy.
Baseline (before fine-tuning):
Certainly! Staying healthy is crucial for maintaining good health and longevity. Here are three key tips to help you stay healthy:
- Regular Exercise: Engaging in regular physical activity is one of the most effective ways...
After fine-tuning:
- Exercise regularly to keep your body strong and fit.
- Eat a balanced diet that includes plenty of fruits, vegetables, lean proteins, and whole grains.
- Get enough sleep and rest to help your body recover from the day's activities.
The style shift is consistent across prompts: shorter, more direct, less ornamental.
Trained using trl.SFTTrainer with completion_only_loss=True — loss is applied only over the assistant response portion of each example, not the user prompt or chat template tokens. Omitting this flag on a small dataset can cause the model to learn to reproduce chat template scaffolding as output (a failure mode observed in early iterations of this experiment).
Chat template formatting (<|im_start|>user ... <|im_end|><|im_start|>assistant ...) handled automatically by the Qwen tokenizer's built-in Jinja template.
do_sample=True with temperature=0.7 and repetition_penalty=1.15.This adapter is a reference point for understanding LoRA mechanics and style transfer dynamics. It is not tuned for any specific downstream task — production applications would need larger datasets and task-specific training.
Useful as:
transformers + trl + peft workflowstransformers, peft, trl, datasets