Downloads · 30 days
0
gemma4-mtp-bench
Blongo1/gemma4-mtp-bench
gemma4-mtp-bench is a machine learning model from Blongo1. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Custom handler for HF Inference Endpoints that loads google/gemma-4-31B-it (target) and google/gemma-4-31B-it-assistant (drafter), then per-request flips speculative decoding via the usemtp parameter.
Access
Public
Updated May 7, 2026
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.py5.4 KB · 70%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- May 7, 2026
- Updated
- May 7, 2026
- SHA
- aa949651
Model card
From the Hugging Face model README
Gemma 4 31B MTP-on-vs-off bench handler
Custom handler for HF Inference Endpoints that loads google/gemma-4-31B-it (target) and google/gemma-4-31B-it-assistant (drafter), then per-request flips speculative decoding via the use_mtp parameter.
Request schema
{
"inputs": "your prompt",
"parameters": {
"max_new_tokens": 300,
"use_mtp": true,
"do_sample": true,
"temperature": 0.7
}
}
Response
Each call returns timing fields (elapsed_seconds, tokens_per_second, generated_tokens) so you can bench MTP-on vs MTP-off on identical hardware/weights.
- License
- apache-2.0
- Created
- May 7, 2026
- Updated
- May 7, 2026