qwen2.5-3b-reasoning-full-ft

Qwen2.5-3B Reasoning (Full Fine-Tuning)

This repository contains the fine-tuned weights for Qwen/Qwen2.5-3B-Instruct trained to produce explicit step-by-step <think>...</think> Chain-of-Thought (CoT) reasoning.

Model Summary

• Fine-Tuning Methodology: Method 3: Full Fine-Tuning (16-bit BF16 - All 3.09 Billion Parameters)
• Base Model: Qwen/Qwen2.5-3B-Instruct
• Trainable Parameters: 3.09 Billion (100% full weight updates)
• Dataset: FreedomIntelligence/medical-o1-reasoning-SFT (10,000 instructions)
• Hardware Acceleration: 2x NVIDIA H200 NVL GPUs (288 GB VRAM)

---

Empirical Benchmark Performance

---

Usage Instructions

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "DexterSptizu/qwen2.5-3b-reasoning-full-ft"

# Load fine-tuned reasoning model & tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Inference Example
prompt = "<|im_start|>system\nYou are a helpful reasoning assistant.<|im_end|>\n<|im_start|>user\nSolve step by step: What is 15 * 24?<|im_end|>\n<|im_start|>assistant\n<think>\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Citation & Acknowledgements

• Base model developed by Alibaba Cloud (Qwen Team).
• Fine-tuning executed on dual NVIDIA H200 NVL GPUs.