Qwen2.5-R1-Minny-1.5B-v1

🧠 Qwen2.5-R1-Minny-1.5B-v1

Qwen2.5-R1-Minny-1.5B-v1 is a highly plastic, multi-domain "Super-Base" model engineered through a novel hybrid merging topology. It fuses state-of-the-art latent reasoning (GRPO), mathematical logic, and coding proficiency into a single 1.5B parameter manifold without suffering from catastrophic forgetting or RLHF-induced formatting collapse.

This model was created using a custom out-of-core pipeline combining Model Stock (geometric anchoring) and Sens-Merging (gradient-based sensitivity scaling), informed by extensive layer-wise ablation studies.

🔬 Model Topology

This model is the result of the "Refined Golden Triangle" merging topology, which anchors on a Group Relative Policy Optimization (GRPO) reasoning model and injects pure Supervised Fine-Tuned (SFT) domain experts.

• The Base Anchor (GRPO): deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
• Expert 1 (Math SFT): RLinf/RLinf-math-1.5B
• Expert 2 (Code SFT): agentica-org/DeepCoder-1.5B-Preview
• Expert 3 (Reasoning SFT): mobiuslabsgmbh/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1

⚙️ The Methodology: Why this Merge Works

During the development of this model, extensive forward-pass perplexity benchmarking revealed several fundamental laws regarding the merging of modern RLHF models:

1. The GRPO Outlier Theorem: Standard sparsification methods (like DARE, DELLA, or Model Breadcrumbs) fundamentally destroy models trained via GRPO (like DeepSeek-R1). GRPO encodes deep logical routing in extreme, sparse weight outliers. Magnitude pruning amputates these reasoning capabilities.
2. The RL Paradigm Collision: When mixing models trained via different Reinforcement Learning paradigms (e.g., PPO vs. GRPO), cross-task logit alignment fails catastrophically due to fundamentally incompatible probability distributions.

To bypass these limitations, Qwen2.5-R1-Minny-1.5B-v1 utilizes a custom hybrid algorithm:

• Model Stock: Acts as the foundational geometric anchor, preventing the injected vectors from pushing the model out of the pre-trained loss basin.
• Sens-Merging (Liu et al., 2025): Because the injected experts (Math and Code) are pure SFT models, we utilized Sens-Merging to compute the layer-wise gradient sensitivities ($\alpha$) and cross-task logit transferability ($\tau$). This allowed the SFT mathematical and coding logic to perfectly weave into the unshielded intermediate layers of the DeepSeek base without destroying the RLHF outliers.

📊 Empirical Benchmarking (Forward-Pass Loss)

Prior to minting this model, it was evaluated against raw baseline algorithms using a strict, zero-shot Forward-Pass Cross-Entropy Loss evaluation across three domains (lower loss = better absorption of the latent space):

By combining the geometric stability of Model Stock with the gradient-aware routing of Sens-Merging, this model achieves the strict Pareto-optimal frontier for 1.5B reasoning architectures.

🚀 Intended Use and Downstream Capabilities

Qwen2.5-R1-Minny-1.5B-v1 is primarily intended to serve as a Super-Base for downstream alignment.

While it can be used for zero-shot inference, its true potential is unlocked when used as the initialization checkpoint for downstream Supervised Fine-Tuning (SFT) or Direct Preference Optimization (DPO). Because the Base Anchor's foundational neurons were mathematically shielded during the merge, the model retains maximum plasticity. It holds deep latent representations of Python/C++ coding syntax, mathematical routing, and DeepSeek <think> traces, ready to be aligned to your specific conversational formatting or agentic workflow.

⚠️ Limitations

• Formatting Schizophrenia: Because this is a merged foundation model combining various SFT datasets, its prompt template adherence (e.g., <|im_start|> vs <|eot_id|>) may be slightly unstable out-of-the-box prior to downstream instruct-tuning.
• Size Constraints: At 1.5 billion parameters, while punching heavily above its weight class in logic, it is still subject to the standard hallucination and knowledge-retrieval limitations inherent to small language models.

📚 Acknowledgements

• The geometry of this merge relies heavily on the theories proposed in Editing Models with Task Arithmetic (Ilharco et al., 2023) and Sens-Merging: Sensitivity-Guided Parameter Balancing (Liu et al., 2025).
• Compute and evaluation orchestration powered by custom implementations of mergekit (Goddard et al.).