Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-QLora

QLoRA Fine-Tuned Qwen2.5-0.5B-Instruct

This repository contains a QLoRA (Quantized Low-Rank Adaptation) fine-tuned version of Qwen2.5-0.5B-Instruct.

The model was trained using QLoRA, where the base model was loaded in 4-bit precision during training to reduce GPU memory usage. Only the LoRA adapter weights were updated and are included in this repository.

Note: This repository contains only the QLoRA adapter. During inference, the adapter can be loaded with the original base model in FP32, FP16, BF16, 8-bit, or 4-bit depending on your hardware.

---

Model Details

• Base Model: Qwen/Qwen2.5-0.5B-Instruct
• Fine-Tuning Method: QLoRA (PEFT)
• Framework: Hugging Face Transformers + PEFT
• Task: Conversational Text Generation

---

Installation

pip install transformers peft accelerate torch safetensors

---

Loading the Model

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen2.5-0.5B-Instruct"
ADAPTER = "vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-QLora"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.float32
)

model = PeftModel.from_pretrained(
    base_model,
    ADAPTER
)

model.eval()

---

Example Inference

messages = [
    {
        "role": "system",
        "content": (
            "You are Vishnu's personal AI assistant. "
            "Answer questions about Vishnu."
        )
    },
    {
        "role": "user",
        "content": "Tell me about Vishnu."
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(
    text,
    return_tensors="pt"
)

with torch.no_grad():

    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        temperature=0.7,
        do_sample=True,
        top_p=0.9,
    )

response = tokenizer.decode(
    outputs[0][inputs.input_ids.shape[-1]:],
    skip_special_tokens=True,
)

print(response)

---

Gradio Demo

import torch
import gradio as gr

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen2.5-0.5B-Instruct"
ADAPTER = "vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-QLora"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.float32
)

model = PeftModel.from_pretrained(
    base_model,
    ADAPTER
)

model.eval()

SYSTEM_PROMPT = (
    "You are Vishnu's personal AI assistant. "
    "Answer questions about Vishnu."
)

def predict(message, history):

    messages = [
        {
            "role": "system",
            "content": SYSTEM_PROMPT,
        }
    ]

    for user, assistant in history:
        messages.append({"role": "user", "content": user})
        messages.append({"role": "assistant", "content": assistant})

    messages.append({"role": "user", "content": message})

    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
    )

    inputs = tokenizer(
        text,
        return_tensors="pt",
    )

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=256,
            temperature=0.7,
            do_sample=True,
            top_p=0.9,
        )

    response = tokenizer.decode(
        outputs[0][inputs.input_ids.shape[-1]:],
        skip_special_tokens=True,
    )

    return response

gr.ChatInterface(
    fn=predict,
    title="Vishnu Personal AI (QLoRA)",
    description="Qwen2.5-0.5B fine-tuned using QLoRA",
).launch()

---

Generation Parameters

---

Repository Structure

adapter_config.json
adapter_model.safetensors
chat_template.jinja
tokenizer.json
tokenizer_config.json
README.md

---

Training Overview

This model was fine-tuned using QLoRA (Quantized Low-Rank Adaptation).

During training:

• The base model was loaded in 4-bit NF4 quantization.
• Only the LoRA adapter weights were trained.
• The original model weights remained frozen.
• QLoRA significantly reduced GPU memory requirements while maintaining strong model performance.

The uploaded repository contains only the adapter weights. During inference, these adapters are applied to the original Qwen2.5-0.5B-Instruct base model.

---

Citation

If you use this model in your work, please cite this repository.

@misc{vishnu_qwen25_qlora,
  author = {Vishnu Amarapu},
  title = {QLoRA Fine-Tuned Qwen2.5-0.5B-Instruct},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-QLora}}
}

---

Acknowledgements

• Alibaba Cloud Qwen Team for the base model.
• Hugging Face Transformers.
• PEFT (Parameter-Efficient Fine-Tuning).
• BitsAndBytes for efficient 4-bit quantized training.
• PyTorch.