qwen-2-5-3b-instruct-using-openai-gsm8k-data-enhanced-with-deepseek-v3

Qwen 2.5-3B-Instruct Fine-Tuned on OpenAI GSM8K with DeepSeek Augmentation

Model Overview

This model is a fine-tuned version of Qwen/Qwen2.5-3B-Instruct, optimized for mathematical reasoning tasks using the OpenAI GSM8K dataset. The fine-tuning process enhances the model's ability to generate step-by-step explanations for grade school math problems, incorporating reasoning augmentation through DeepSeek. The model improves upon GSM8K’s standard answers by integrating additional contextual reasoning derived from DeepSeek’s small model.

Key Features:

• Base Model: Qwen 2.5-3B-Instruct
• Fine-Tuned On: OpenAI GSM8K dataset
• Enhancement: Answer augmentation with reasoning insights from DeepSeek-V3-Small
• Improved Reasoning: Model not only provides correct answers but also augments explanations with logical steps inspired by DeepSeek’s generative capabilities.

Dataset & Training Details

• Dataset: OpenAI’s GSM8K (Grade School Math 8K), a collection of high-quality math problems designed to test problem-solving skills.
• Data Curation: We have provided enhanced OpenAI's GSM8K dataset with reasoning skills generated using DeepSeek-R1, see here.
• Enhancement: After fine-tuning on GSM8K, additional reasoning layers were introduced using DeepSeek-V3-Small, leading to richer, more interpretable answers.
• Training Objective: Improve step-by-step mathematical reasoning and enhance logical deductions in model-generated responses.

I have adopted some code from Unsloth and here's an updated notebook on Colab. Please feel free to copy it and run it yourself.

You will need:

• Huggingface token
• Together.AI API Key
• Unsloth package

How to Use Model via Terminal (Mac)

Goal Run Qwen-2.5-3B Instruct on Your Mac Using llama.cpp

Yes! You can run Qwen-2.5-3B Instruct on your Mac using llama.cpp. Here’s a step-by-step guide assuming you are starting from a clean macOS installation with only pyenv installed.

Step 1: Install Homebrew (if not installed)

Homebrew is required to install llama.cpp.

1. Open Terminal (Cmd + Space, type Terminal, and press Enter).
2. Run:

   /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
   

3. After installation, add Homebrew to your PATH:

   echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> ~/.zprofile
   eval "$(/opt/homebrew/bin/brew shellenv)"
   

---

Step 2: Install llama.cpp via Homebrew

Run:

brew install llama.cpp

Once installed, you should be able to use llama-cli.

---

Step 3: Run Qwen-2.5-3B Instruct with llama-cli

To run the model, execute:

llama-cli -hf eagle0504/qwen-2-5-3b-instruct-using-openai-gsm8k-data-enhanced-with-deepseek-v3:Q8_0

---

Step 4: Additional Configurations (If Needed)

If you encounter issues or need finer control, you may want to:

A. Verify Installation

Check if llama-cli is installed:

llama-cli --version

If you see a version output, it’s installed correctly.

B. Run with Explicit Model Path

If the default Hugging Face loader doesn't work, you can manually download the model:

1. Create a models directory:

   mkdir -p ~/llama_models && cd ~/llama_models
   

2. Download the GGUF model file from Hugging Face:

   wget https://huggingface.co/eagle0504/qwen-2-5-3b-instruct-using-openai-gsm8k-data-enhanced-with-deepseek-v3/blob/main/unsloth.Q8_0.gguf
   

3. Run the model manually:

   llama-cli -m ~/llama_models/Q8_0.gguf
   

---

Step 5: Test the Model

Try prompting it:

llama-cli -m ~/llama_models/Q8_0.gguf -p "Explain quantum computing in simple terms."

or interactively:

llama-cli -m ~/llama_models/Q8_0.gguf --interactive

How to Use Model via Python

You can load this model with transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "eagle0504/qwen-2-5-3b-instruct-using-openai-gsm8k-data-enhanced-with-deepseek-v3"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Example prompt
prompt = "A farmer has 24 apples. He gives 6 to each of his 3 children. How many does he have left?"
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_length=200)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Expected Performance

Compared to the base Qwen2.5-3B-Instruct, this fine-tuned model:

• Provides more detailed explanations when answering GSM8K math problems.
• Improves logical reasoning by incorporating DeepSeek-style augmented reasoning.
• Generates clearer step-by-step solutions, making it useful for educational or tutoring applications.

Model Directory

The model is hosted on Hugging Face Hub: 👉 eagle0504/qwen-2-5-3b-instruct-using-openai-gsm8k-data-enhanced-with-deepseek-v3

License

This model is released under the MIT License, allowing open usage and modifications.

---

If you have any questions or suggestions for improvements, feel free to reach out!

🔗 LinkedIn