Downloads · 30 days
5
19% of all-time downloads
dynamic-qwen-mod-gamma0.5
fredericowieser/dynamic-qwen-mod-gamma0.5
dynamic-qwen-mod-gamma0.5 is a machine learning model from fredericowieser. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This model is a dynamically-computed version of outputs/PRETRAIN-TEST-qwen2.5-0.5B-mod-pretrainmix-2025-08-2814-02-45-gamma=0.5/finalmodel, fine-tuned using the MOD architecture.
All-time downloads
27
Public
Parameters
494M
2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors2 GB · 99%
At a glance
- License
- apache-2.0
- Model type
- dynamic_qwen
- Access
- Public
- Created
- Aug 29, 2025
- Updated
- Aug 29, 2025
- SHA
- bbe540ac
Model card
From the Hugging Face model README
Model Card for dynamic-qwen-mod-gamma0.5
This model is a dynamically-computed version of outputs/PRETRAIN-TEST-qwen2.5-0.5B-mod-pretrain_mix-2025-08-28_14-02-45-gamma=0.5/final_model, fine-tuned
using the MOD architecture.
- Dynamic Architecture:
MOD - Capacity Gamma (γ):
0.5
The MOD architecture enables the model to conditionally skip parts of its
computation, aiming for improved efficiency. The capacity_gamma parameter
controls the portion of tokens processed by the dynamic components.
How to Use
This model requires trust_remote_code=True to load the custom architecture.
from transformers import AutoModelForCausalLM, AutoTokenizer
# It is recommended to load in bfloat16 for efficiency
model = AutoModelForCausalLM.from_pretrained(
"fredericowieser/dynamic-qwen-mod-gamma0.5",
trust_remote_code=True,
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("fredericowieser/dynamic-qwen-mod-gamma0.5")
# Example usage
prompt = "The capital of the United Kingdom is"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=10)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Evaluation
Results on standard benchmarks:
| Task | Metric | Value |
|---|---|---|
| arc_challenge | acc,none | 0.2065 |
| arc_challenge | acc_norm,none | 0.2432 |
| arc_challenge | acc_norm_stderr,none | 0.0125 |
| arc_challenge | acc_stderr,none | 0.0118 |
| hellaswag | acc,none | 0.2945 |
| hellaswag | acc_norm,none | 0.3262 |
| hellaswag | acc_norm_stderr,none | 0.0047 |
| hellaswag | acc_stderr,none | 0.0045 |
| mmlu | acc,none | 0.2332 |
| mmlu | acc_stderr,none | 0.0036 |
| mmlu_abstract_algebra | acc,none | 0.2000 |
| mmlu_abstract_algebra | acc_stderr,none | 0.0402 |
| mmlu_anatomy | acc,none | 0.2667 |
| mmlu_anatomy | acc_stderr,none | 0.0382 |
| mmlu_astronomy | acc,none | 0.1776 |
| mmlu_astronomy | acc_stderr,none | 0.0311 |
| mmlu_business_ethics | acc,none | 0.2800 |
| mmlu_business_ethics | acc_stderr,none | 0.0451 |
| mmlu_clinical_knowledge | acc,none | 0.2075 |
| mmlu_clinical_knowledge | acc_stderr,none | 0.0250 |
| mmlu_college_biology | acc,none | 0.2917 |
| mmlu_college_biology | acc_stderr,none | 0.0380 |
| mmlu_college_chemistry | acc,none | 0.2200 |
| mmlu_college_chemistry | acc_stderr,none | 0.0416 |
| mmlu_college_computer_science | acc,none | 0.2200 |
| mmlu_college_computer_science | acc_stderr,none | 0.0416 |
| mmlu_college_mathematics | acc,none | 0.2400 |
| mmlu_college_mathematics | acc_stderr,none | 0.0429 |
| mmlu_college_medicine | acc,none | 0.1965 |
| mmlu_college_medicine | acc_stderr,none | 0.0303 |
| mmlu_college_physics | acc,none | 0.2255 |
| mmlu_college_physics | acc_stderr,none | 0.0416 |
| mmlu_computer_security | acc,none | 0.2400 |
| mmlu_computer_security | acc_stderr,none | 0.0429 |
| mmlu_conceptual_physics | acc,none | 0.2723 |
| mmlu_conceptual_physics | acc_stderr,none | 0.0291 |
| mmlu_econometrics | acc,none | 0.2018 |
| mmlu_econometrics | acc_stderr,none | 0.0378 |
| mmlu_electrical_engineering | acc,none | 0.2483 |
| mmlu_electrical_engineering | acc_stderr,none | 0.0360 |
| mmlu_elementary_mathematics | acc,none | 0.2090 |
| mmlu_elementary_mathematics | acc_stderr,none | 0.0209 |
| mmlu_formal_logic | acc,none | 0.2143 |
| mmlu_formal_logic | acc_stderr,none | 0.0367 |
| mmlu_global_facts | acc,none | 0.2400 |
| mmlu_global_facts | acc_stderr,none | 0.0429 |
| mmlu_high_school_biology | acc,none | 0.2387 |
| mmlu_high_school_biology | acc_stderr,none | 0.0243 |
| mmlu_high_school_chemistry | acc,none | 0.1921 |
| mmlu_high_school_chemistry | acc_stderr,none | 0.0277 |
| mmlu_high_school_computer_science | acc,none | 0.2600 |
| mmlu_high_school_computer_science | acc_stderr,none | 0.0441 |
| mmlu_high_school_european_history | acc,none | 0.2182 |
| mmlu_high_school_european_history | acc_stderr,none | 0.0323 |
| mmlu_high_school_geography | acc,none | 0.2020 |
| mmlu_high_school_geography | acc_stderr,none | 0.0286 |
| mmlu_high_school_government_and_politics | acc,none | 0.1710 |
| mmlu_high_school_government_and_politics | acc_stderr,none | 0.0272 |
| mmlu_high_school_macroeconomics | acc,none | 0.2128 |
| mmlu_high_school_macroeconomics | acc_stderr,none | 0.0208 |
| mmlu_high_school_mathematics | acc,none | 0.2444 |
| mmlu_high_school_mathematics | acc_stderr,none | 0.0262 |
| mmlu_high_school_microeconomics | acc,none | 0.2017 |
| mmlu_high_school_microeconomics | acc_stderr,none | 0.0261 |
| mmlu_high_school_physics | acc,none | 0.1722 |
| mmlu_high_school_physics | acc_stderr,none | 0.0308 |
| mmlu_high_school_psychology | acc,none | 0.1633 |
| mmlu_high_school_psychology | acc_stderr,none | 0.0158 |
| mmlu_high_school_statistics | acc,none | 0.1806 |
| mmlu_high_school_statistics | acc_stderr,none | 0.0262 |
| mmlu_high_school_us_history | acc,none | 0.2549 |
| mmlu_high_school_us_history | acc_stderr,none | 0.0306 |
| mmlu_high_school_world_history | acc,none | 0.2321 |
| mmlu_high_school_world_history | acc_stderr,none | 0.0275 |
| mmlu_human_aging | acc,none | 0.2691 |
| mmlu_human_aging | acc_stderr,none | 0.0298 |
| mmlu_human_sexuality | acc,none | 0.2595 |
| mmlu_human_sexuality | acc_stderr,none | 0.0384 |
| mmlu_humanities | acc,none | 0.2425 |
| mmlu_humanities | acc_stderr,none | 0.0063 |
| mmlu_international_law | acc,none | 0.2562 |
| mmlu_international_law | acc_stderr,none | 0.0398 |
| mmlu_jurisprudence | acc,none | 0.2593 |
| mmlu_jurisprudence | acc_stderr,none | 0.0424 |
| mmlu_logical_fallacies | acc,none | 0.2147 |
| mmlu_logical_fallacies | acc_stderr,none | 0.0323 |
| mmlu_machine_learning | acc,none | 0.2589 |
| mmlu_machine_learning | acc_stderr,none | 0.0416 |
| mmlu_management | acc,none | 0.1650 |
| mmlu_management | acc_stderr,none | 0.0368 |
| mmlu_marketing | acc,none | 0.2906 |
| mmlu_marketing | acc_stderr,none | 0.0297 |
| mmlu_medical_genetics | acc,none | 0.3000 |
| mmlu_medical_genetics | acc_stderr,none | 0.0461 |
| mmlu_miscellaneous | acc,none | 0.2439 |
| mmlu_miscellaneous | acc_stderr,none | 0.0154 |
| mmlu_moral_disputes | acc,none | 0.2601 |
| mmlu_moral_disputes | acc_stderr,none | 0.0236 |
| mmlu_moral_scenarios | acc,none | 0.2425 |
| mmlu_moral_scenarios | acc_stderr,none | 0.0143 |
| mmlu_nutrition | acc,none | 0.2549 |
| mmlu_nutrition | acc_stderr,none | 0.0250 |
| mmlu_other | acc,none | 0.2391 |
| mmlu_other | acc_stderr,none | 0.0076 |
| mmlu_philosophy | acc,none | 0.2476 |
| mmlu_philosophy | acc_stderr,none | 0.0245 |
| mmlu_prehistory | acc,none | 0.2191 |
| mmlu_prehistory | acc_stderr,none | 0.0230 |
| mmlu_professional_accounting | acc,none | 0.2730 |
| mmlu_professional_accounting | acc_stderr,none | 0.0266 |
| mmlu_professional_law | acc,none | 0.2458 |
| mmlu_professional_law | acc_stderr,none | 0.0110 |
| mmlu_professional_medicine | acc,none | 0.1360 |
| mmlu_professional_medicine | acc_stderr,none | 0.0208 |
| mmlu_professional_psychology | acc,none | 0.2598 |
| mmlu_professional_psychology | acc_stderr,none | 0.0177 |
| mmlu_public_relations | acc,none | 0.2000 |
| mmlu_public_relations | acc_stderr,none | 0.0383 |
| mmlu_security_studies | acc,none | 0.2735 |
| mmlu_security_studies | acc_stderr,none | 0.0285 |
| mmlu_social_sciences | acc,none | 0.2187 |
| mmlu_social_sciences | acc_stderr,none | 0.0074 |
| mmlu_sociology | acc,none | 0.2537 |
| mmlu_sociology | acc_stderr,none | 0.0308 |
| mmlu_stem | acc,none | 0.2277 |
| mmlu_stem | acc_stderr,none | 0.0075 |
| mmlu_us_foreign_policy | acc,none | 0.2400 |
| mmlu_us_foreign_policy | acc_stderr,none | 0.0429 |
| mmlu_virology | acc,none | 0.2651 |
| mmlu_virology | acc_stderr,none | 0.0344 |
| mmlu_world_religions | acc,none | 0.2632 |
| mmlu_world_religions | acc_stderr,none | 0.0338 |
| truthfulqa_mc2 | acc,none | 0.4355 |
| truthfulqa_mc2 | acc_stderr,none | 0.0155 |
| winogrande | acc,none | 0.5264 |
| winogrande | acc_stderr,none | 0.0140 |
- Type
- dynamic_qwen
- License
- apache-2.0
- Created
- Aug 29, 2025
- Updated
- Aug 29, 2025