Downloads · 30 days
12
34% of all-time downloads
ismamNur/qwen2.5-math-1.5b-jepa-pretrain-amcaime32k-openr1-step400
qwen2.5-math-1.5b-jepa-pretrain-amcaime32k-openr1-step400 is a machine learning model from ismamNur. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Stage-1 checkpoint of the two-stage JEPA→GRPO training pipeline. The model was JEPA-pretrained (LLM-JEPA cosine alignment + SIGReg, official recipe, no repulsion) on verified solved CoT traces, then serves as the init…
Downloads · 30 days
12
34% of all-time downloads
All-time downloads
35
Public
Parameters
1.5B
3.1 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors3.1 GB · 99%
From the Hugging Face model README
Stage-1 checkpoint of the two-stage JEPA→GRPO training pipeline. The model was JEPA-pretrained (LLM-JEPA cosine alignment + SIGReg, official recipe, no repulsion) on verified solved CoT traces, then serves as the init for stage-2 plain GRPO RL.
Qwen/Qwen2.5-Math-1.5B-Instruct (hidden_size 1536, 28 layers)(1−λ)·mean(1−cos(p,z)) + λ·SIGReg(p), λ=0.1 (official LLM-JEPA default)<|predictor_1|> token, learnable embedding delta (seed 1234 init)model.safetensors — merged full fine-tuned weightspredictor_delta.pt — learnable embedding delta for <|predictor_1|> (1×1536)manifest.json — full run config