Qwen2.5-7B-distilled-Claude-4.6-Opus

base_model: unsloth/Qwen2.5-Coder-7B-bnb-4bit library_name: peft pipeline_tag: text-generation tags:

• lora
• qlora
• sft
• unsloth
• reasoning
• chain-of-thought
• qwen2.5

---

Qwen2.5-Coder-7B-Reasoning-LoRA

This model is a fine-tuned version of Qwen2.5-Coder-7B-bnb-4bit designed to enhance logical reasoning and algorithmic planning in code generation tasks. It utilizes a "Chain-of-Thought" approach, encouraging the model to plan and verify its logic within <think> tags before outputting final code.

Model Details

Model Description

The primary goal of this fine-tune was to move the base model from pattern-matching to engineering-level reasoning. By training on distilled reasoning data, the model has learned to identify edge cases, plan complex algorithms (like Shunting Yard), and self-correct during the generation process.

• Developed by: Hamza Mughal
• Model type: Causal Language Model (LoRA Adapter)
• Language(s) (NLP): English, Python
• Finetuned from model: unsloth/Qwen2.5-Coder-7B-bnb-4bit (Qwen 2.5 architecture)
• Training Method: QLoRA (Quantized Low-Rank Adaptation)

Uses

Direct Use

This model is intended for coding assistance where complex logic is required. Users should provide a prompt and allow the model to generate its thinking process for better accuracy.

Out-of-Scope Use

The model is not designed for non-technical prose or general-purpose creative writing. It is highly optimized for technical reasoning and Pythonic logic.

Training Details

Training Data

The model was trained on a specialized dataset consisting of 700+ rows of high-quality reasoning traces. This data is a distillation of Claude 3.5/4.6 Opus outputs, specifically curated to teach multi-step algorithmic planning.

Training Procedure

Methodology

• Quantization: The base model was loaded in 4-bit NormalFloat (NF4) using bitsandbytes.
• PEFT Technique: QLoRA was used to target all linear modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj.
• Optimization: 8-bit AdamW was used to minimize VRAM footprint.
• Gradient Checkpointing: Enabled (Unsloth optimized) to facilitate training on consumer-grade hardware.

Hyperparameters

• Learning Rate: 5e-5
• Batch Size: 2 (with 4 Gradient Accumulation steps)
• Max Steps: 60
• LR Scheduler: Linear warmup
• Weight Decay: 0.01

Technical Specifications

Hardware and Inference

During development, it was noted that while the model can be trained in a multi-GPU environment, inference is most stable when the adapters are loaded onto a single GPU (e.g., device_map={"": 0}). This prevents shape mismatch errors in custom Unsloth kernels during the auto-regressive decoding phase.

How to Get Started with the Model

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "your_username/Qwen2.5-Coder-7B-Reasoning-LoRA",
    max_seq_length = 4096,
    load_in_4bit = True,
)
FastLanguageModel.for_inference(model)

inputs = tokenizer([
    "### Instruction:\nWrite a function to solve math strings without eval().\n\n### Response:\n<think>\n"
], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 1024)
print(tokenizer.batch_decode(outputs))

## Evaluation Results

### Observed Improvements
- **Constraint Adherence:** Successfully avoids using forbidden functions like `eval()` when instructed.
- **Algorithmic Depth:** Capable of implementing complex parsers and stack-based evaluators from scratch.
- **Edge Case Identification:** Proactively tests for negative numbers, empty strings, and null inputs in its internal monologue.

### Framework versions
- PEFT 0.18.1
- Unsloth 2024.12
- Transformers 4.46.3