Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-LORA

LoRA Fine-Tuned Qwen2.5-0.5B-Instruct

This repository contains a LoRA (Low-Rank Adaptation) fine-tuned version of Qwen2.5-0.5B-Instruct. The repository stores only the LoRA adapter weights, making it lightweight and easy to share. To use this model, first load the base model and then apply the LoRA adapter.

Model Details

• Base Model: Qwen/Qwen2.5-0.5B-Instruct
• Fine-Tuning Method: LoRA (PEFT)
• Framework: Hugging Face Transformers + PEFT
• Task: Conversational Text Generation

---

Installation

pip install transformers peft accelerate torch safetensors

---

Loading the Model

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen2.5-0.5B-Instruct"
LORA_MODEL = "vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-LORA"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto"
)

model = PeftModel.from_pretrained(
    base_model,
    LORA_MODEL
)

model.eval()

---

Example Inference

messages = [
    {
        "role": "system",
        "content": (
            "You are Vishnu's personal AI assistant. "
            "Answer questions about Vishnu."
        )
    },
    {
        "role": "user",
        "content": "Tell me about Vishnu."
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(
    text,
    return_tensors="pt"
).to(model.device)

with torch.no_grad():

    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        temperature=0.7,
        do_sample=True,
        top_p=0.9,
        repetition_penalty=1.1,
    )

response = tokenizer.decode(
    outputs[0][inputs.input_ids.shape[-1]:],
    skip_special_tokens=True,
)

print(response)

---

Gradio Demo

import torch
import gradio as gr

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen2.5-0.5B-Instruct"
LORA_MODEL = "vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-LORA"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto"
)

model = PeftModel.from_pretrained(
    base_model,
    LORA_MODEL
)

model.eval()

SYSTEM_PROMPT = (
                """ You are Vishnu's personal AI assistant.
Always answer using the information you have learned about Vishnu.
Do not invent facts.
If you do not know the answer, say you don't know.
Answer as Vishnu in first person."""
)

def generate(message, history):

    messages = [
        {
            "role": "system",
            "content": SYSTEM_PROMPT,
        }
    ]

    for user, assistant in history:
        messages.append({"role": "user", "content": user})
        messages.append({"role": "assistant", "content": assistant})

    messages.append({"role": "user", "content": message})

    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
    )

    inputs = tokenizer(
        text,
        return_tensors="pt",
    ).to(model.device)

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=256,
            temperature=0.7,
            do_sample=True,
            top_p=0.9,
            repetition_penalty=1.1,
        )

    answer = tokenizer.decode(
        outputs[0][inputs.input_ids.shape[-1]:],
        skip_special_tokens=True,
    )

    return answer

gr.ChatInterface(
    fn=generate,
    title="Vishnu Personal AI",
    description="LoRA Fine-Tuned Qwen2.5-0.5B-Instruct",
).launch()

---

Generation Parameters

---

Repository Structure

adapter_config.json
adapter_model.safetensors
chat_template.jinja
tokenizer.json
tokenizer_config.json
README.md

---

Training Overview

This model was fine-tuned using LoRA (Low-Rank Adaptation) through the PEFT library. Only a small set of trainable adapter weights were updated while the base model remained frozen, resulting in a lightweight adapter that can be merged with the original base model for inference.

---

Citation

If you use this model in your work, please cite this repository.

@misc{vishnu_qwen25_lora,
  author = {Vishnu Amarapu},
  title = {LoRA Fine-Tuned Qwen2.5-0.5B-Instruct},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-LORA}}
}

---

Acknowledgements

• Alibaba Cloud Qwen Team for the base model.
• Hugging Face Transformers.
• PEFT (Parameter-Efficient Fine-Tuning).
• PyTorch.