Qwen-2.5-0.5B-gsm8k

Model Details

---

Training

The model was fine-tuned using:

• Unsloth Studio
• QLoRA adapters
• Supervised Fine-Tuning (SFT)
• GSM8K training split

The objective was to teach the model to generate correct step-by-step solutions for mathematical word problems.

---

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "Rzkoohi/Qwen-2.5-0.5B-gsm8k"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

messages = [
    {
        "role": "user",
        "content": "If John has 5 apples and buys 3 more, then gives away 2, how many apples does he have?"
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(text, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=256
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Intended Use

Recommended for:

• Mathematical reasoning
• Educational experiments
• Fine-tuning demonstrations
• Small-model research
• Learning QLoRA workflows

Not recommended for:

• Production deployments
• Complex reasoning
• Long-context tasks
• High-accuracy mathematical applications

---

Limitations

This model has only been fine-tuned on GSM8K.

Although mathematical reasoning is improved, performance on unrelated tasks may differ from the original instruction model. Like all 0.5B parameter models, it has limited world knowledge and reasoning capacity compared to larger LLMs.

---

Future Work

Planned improvements include:

• Evaluation against the original base model
• Additional reasoning datasets
• Multi-domain instruction tuning
• GGUF and merged model releases
• Benchmark reporting

---

Acknowledgements

This model is based on:

• Qwen2.5-0.5B-Instruct
• GSM8K
• Unsloth
• Hugging Face Transformers

---

Citation

If you use this model in your work, please cite this repository.