Downloads · 30 days
11
21% of all-time downloads
MuToR-gemma-2B-1M_MATH-dmax_3_a_02
nasos10/MuToR-gemma-2B-1M_MATH-dmax_3_a_02
MuToR-gemma-2B-1M_MATH-dmax_3_a_02 is a text generation model from nasos10. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
All-time downloads
52
Public
Parameters
2.5B
5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- gemma
- Access
- Public
- Created
- Jun 27, 2025
- Updated
- Jun 27, 2025
- SHA
- 77b368c7
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
MuToR: Multi-Token prediction with Registers
Arxiv: https://arxiv.org/abs/2505.10518
TL;DR: MuToR is a simple, plug-and-play approach for multi-token prediction. It leverages dummy register tokens to predict multiple targets in the future, enriching the supervisory signal and improving performance across diverse settings and modalities. The register tokens are discarded on inference, leaving generation speed unchanged.
Model Description
This model is a finetuned version of Gemma-2B. It was finetuned using the MuToR method for 5 epochs on the 1M-MATH training split. Please refer to our code for guidelines on how to use the models to reproduce our results.
- Task
- Text Generation
- Library
- transformers
- Type
- gemma
- License
- mit
- Created
- Jun 27, 2025
- Updated
- Jun 27, 2025