qwen2.5-7b-alpaca-1pct-lora

LoRA Adapter for Qwen/Qwen2.5-7B-Instruct

This repository hosts a LoRA adapter (and tokenizer files) trained on top of Qwen/Qwen2.5-7B-Instruct.

✨ What’s inside

• PEFT type: LORA
• LoRA r: 16
• LoRA alpha: 16
• LoRA dropout: 0.0
• Target modules: q_proj, gate_proj, o_proj, down_proj, k_proj, v_proj, up_proj

📚 Datasets

• yahma/alpaca-cleaned

🌐 Languages

• en

📝 Usage

(A) Use adapter with the official base model

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = "Qwen/Qwen2.5-7B-Instruct"
adapter_id = "hwang2006/qwen2.5-7b-alpaca-1pct-lora"

tok = AutoTokenizer.from_pretrained(base)
base_model = AutoModelForCausalLM.from_pretrained(
    base,
    torch_dtype=torch.bfloat16 if torch.cuda.is_available() else torch.float32,
    device_map="auto",
)

model = PeftModel.from_pretrained(base_model, adapter_id)

messages = [
    {"role":"system","content":"You are a helpful assistant."},
    {"role":"user","content":"Quick test?"},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
    out = model.generate(**inputs, max_new_tokens=256, do_sample=True, temperature=0.7, top_p=0.9)

print(tok.decode(out[0], skip_special_tokens=True))

(B) 4-bit on the fly (if VRAM is tight)

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)
base = "Qwen/Qwen2.5-7B-Instruct"
adapter_id = "hwang2006/qwen2.5-7b-alpaca-1pct-lora"

tok = AutoTokenizer.from_pretrained(base)
base_model = AutoModelForCausalLM.from_pretrained(base, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base_model, adapter_id)

⚠️ Notes

• Use a compatible base (architecture & tokenizer) with this LoRA.
• This repo contains only adapters/tokenizer, not full model weights.
• License here reflects this adapter’s repository. Ensure the base model’s license fits your use.