Downloads · 30 days
7
7% of all-time downloads
Qwen2.5-Math-1.5B-RoPE-300k
waleko/Qwen2.5-Math-1.5B-RoPE-300k
Qwen2.5-Math-1.5B-RoPE-300k is a machine learning model from waleko. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Weights are identical to Qwen/Qwen2.5-Math-1.5B. Only config.json is changed.
All-time downloads
94
Public
Parameters
1.5B
3.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.1 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- qwen2
- Access
- Public
- Created
- Jun 28, 2026
- Updated
- Jun 28, 2026
- SHA
- 4f840c2a
Base models
Model card
From the Hugging Face model README
Qwen2.5-Math-1.5B-RoPE-300k
Weights are identical to Qwen/Qwen2.5-Math-1.5B. Only config.json is changed.
Mirrors what open-r1/Qwen2.5-Math-7B-RoPE-300k did for the 7B variant.
Config changes vs base
| Field | Base (Qwen2.5-Math-1.5B) | This model |
|---|---|---|
rope_theta | 10000 | 300000.0 |
max_position_embeddings | 4096 | 32768 |
sliding_window | 4096 | null |
use_mrope | false | removed |
rope_scaling | — | null |
Why
Increasing rope_theta 10k → 300k compresses RoPE frequencies so that the angular range seen during training (positions 0–4096) maps to positions 0–32768 at inference. No fine-tuning required — the model has seen all the relevant rotation angles; they are just stretched across a wider position range.
The 1.5B model has head_dim=128 (hidden_size=1536 / 12 heads). The compression ratio for the slowest frequency dimension is ~28×, placing rope300k position 32768 at an effective base-training position of ~1152 — well within the original training distribution.
- Type
- qwen2
- License
- apache-2.0
- Created
- Jun 28, 2026
- Updated
- Jun 28, 2026