Qwen2.5-7B-Math-Distill-Sens

Qwen2.5-Math-DeepSeekR1-Sens-7B

A 7B merged model created by applying Sensitivity-aware Model Merging (Sens Merging) to:

• deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
• Qwen/Qwen2.5-Math-7B

The goal of this model is to preserve the strong mathematical reasoning ability of DeepSeek-R1-Distill while significantly reducing reasoning verbosity and output token length.

---

Highlights

• Average accuracy: 66.9%
• Average output tokens: 701
• Output tokens reduced by 75.2% compared to DeepSeek-R1-Distill-Qwen-7B
• Only 2.5 points lower average accuracy than DeepSeek-R1-Distill-Qwen-7B

This model provides an attractive trade-off between reasoning quality and inference cost.

---

Base Models

---

Benchmark Results

---

Motivation

Large reasoning models such as DeepSeek-R1-Distill often produce long chains of thought, which increases inference cost.

This model explores whether model merging can reduce reasoning verbosity without requiring additional training.

By merging a reasoning model (DeepSeek-R1-Distill-Qwen-7B) with a compact mathematical model (Qwen2.5-Math-7B) using Sensitivity-aware Model Merging, the merged model:

• Maintains competitive reasoning performance
• Produces significantly shorter outputs
• Requires no gradient-based fine-tuning
• Uses only a small calibration dataset

---

Comparison with DPO

We additionally compared Sens Merging with a DPO-trained model:

Sens Merging achieves a much larger reduction in output length while remaining competitive in accuracy.

---

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "quangdung/Qwen2.5-Math-DeepSeekR1-Sens-7B"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

prompt = "Solve: If x^2 + 5x + 6 = 0, find x."

messages = [{"role": "user", "content": prompt}]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))