Qwen2.5-Coder-1.5B-DEEP-LoRA

Qwen2.5-Coder-1.5B-DEEP-LoRA

This model is a Fine-Tuned version of Qwen2.5-Coder-1.5B-Instruct using LoRA (Low-Rank Adaptation). It was trained as part of a Competitive Code Reasoning project.

🎯 Model Description

• Model Type: LoRA Adapter
• Base Model: Qwen/Qwen2.5-Coder-1.5B-Instruct
• Focus: This specific adapter was trained on the DEEP dataset (containing deeper reasoning traces) to enhance code reasoning capabilities.

📊 Dataset

The model was fine-tuned on the CodeGen-Deep-5K dataset.

• Training Field: Sadece solution (code-only) alanı kullanılarak eğitilmiştir.
• System Prompt: Training sırasında şu prompt zorunlu tutulmuştur:

  "You are an expert Python programmer. Please read the problem carefully before writing any Python code."

⚙️ Hyperparameters

The following hyperparameters were used during training (as per project specifications):

📉 Training Results

Training loss logs demonstrating the model's convergence:

💻 How to Use

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# 1. Load Base Model
base_model_id = "Qwen/Qwen2.5-Coder-1.5B-Instruct"
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# 2. Load This LoRA Adapter
adapter_id = "colak46/Qwen2.5-Coder-1.5B-DEEP-LoRA"
model = PeftModel.from_pretrained(model, adapter_id)
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

# 3. Inference
prompt = "Write a Python function to solve the Valid Parentheses problem."
messages = [
    {"role": "system", "content": "You are an expert Python programmer. Please read the problem carefully before writing any Python code."},
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))