mita-v1.2-7b-2-24-2025

Formulae/MITA-V1.2-7B-2-24-2025

Overview

Formulae/MITA-V1.2-7B is designed primarily for mathematics, with some capability in coding. Built using the Linear DARE merge method, this model blends powerful mathematical reasoning with computational accuracy.

Merge Details

• Base Model: Qwen/Qwen2.5-Math-7B-Instruct
• Merged Models:
  • nvidia/AceMath-7B-Instruct
  • open-r1/OpenR1-Qwen-7B
• Merge Method: Linear DARE
• Data Type: bfloat16
• Merge Parameters:
  • Density & Weight: 0.5 for AceMath & OpenR1
  • Normalization: Disabled
  • Int8 Masking: Enabled

What is DARE?

DARE (Density-Aware Residual Estimation) is an advanced model merging technique designed to preserve task-specific knowledge. Unlike simple model averaging, DARE adjusts parameter density to ensure that merged models retain their specialized skills while improving general performance.

📖 Reference: DARE Paper

This merge is also inspired by task arithmetic, which shows that models can be linearly combined to enhance capabilities in specialized domains.

📖 Reference: Task Arithmetic Paper

Capabilities

✅ Advanced Mathematics – Strong problem-solving, algebra, calculus, and theorem applications.
✅ Limited Coding Support – Can handle basic programming tasks but is not optimized for complex software development.

Limitations & Risks

⚠ Hallucinations in Code – Not a coding-specialized model, may produce incorrect or insecure implementations.
⚠ Arithmetic Errors – While highly capable, the model still makes occasional miscalculations.

Usage Disclaimer

Formulae/MITA-V1.2-7B is an experimental mathematical model. For verified accuracy, always cross-check results with reliable tools.

Contribute

We welcome contributions, including quantizations, fine-tuning, and further enhancements.

💡 Support Us: Buy Me a Coffee

📩 Contact: [email protected]

---

Future Development

This is part of the MITA series. Future iterations will integrate MoE (Mixture of Experts) for even more specialized reasoning across multiple domains.

Made possible with MergeKit.