Downloads · 30 days
11
18% of all-time downloads
noneUsername/Impish_Magic_24B-W8A8
Impish_Magic_24B-W8A8 is a machine learning model from noneUsername. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
vllm (pretrained=/root/autodl-tmp/ImpishMagic24B,addbostoken=true,maxmodellen=3096,dtype=bfloat16,trustremotecode=true,gpumemoryutilization=0.8), genkwargs: (None), limit: 250.0, numfewshot: 5, batchsize: auto
Downloads · 30 days
11
18% of all-time downloads
All-time downloads
60
Public
Parameters
23.6B
24.9 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors24.9 GB · 100%
How the weights are stored.
I822.2B · 94%
From the Hugging Face model README
vllm (pretrained=/root/autodl-tmp/Impish_Magic_24B,add_bos_token=true,max_model_len=3096,dtype=bfloat16,trust_remote_code=true,gpu_memory_utilization=0.8), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| gsm8k | 3 | flexible-extract | 5 | exact_match | ↑ | 0.908 | ± | 0.0183 |
| strict-match | 5 | exact_match | ↑ | 0.904 | ± | 0.0187 |
vllm (pretrained=/root/autodl-tmp/Impish_Magic_24B,add_bos_token=true,max_model_len=3048,dtype=bfloat16,trust_remote_code=true,gpu_memory_utilization=0.9), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: auto
| Groups | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| mmlu | 2 | none | acc | ↑ | 0.7545 | ± | 0.0034 | |
| - humanities | 2 | none | acc | ↑ | 0.6782 | ± | 0.0064 | |
| - other | 2 | none | acc | ↑ | 0.8169 | ± | 0.0067 | |
| - social sciences | 2 | none | acc | ↑ | 0.8469 | ± | 0.0064 | |
| - stem | 2 | none | acc | ↑ | 0.7168 | ± | 0.0077 |
vllm (pretrained=/root/autodl-tmp/90-512-2048-9999999,add_bos_token=true,max_model_len=3096,dtype=bfloat16,trust_remote_code=true,gpu_memory_utilization=0.8), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| gsm8k | 3 | flexible-extract | 5 | exact_match | ↑ | 0.908 | ± | 0.0183 |
| strict-match | 5 | exact_match | ↑ | 0.908 | ± | 0.0183 |
vllm (pretrained=/root/autodl-tmp/90-512-2048-9999999,add_bos_token=true,max_model_len=3048,dtype=bfloat16,trust_remote_code=true,gpu_memory_utilization=0.9), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: auto
| Groups | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| mmlu | 2 | none | acc | ↑ | 0.7527 | ± | 0.0034 | |
| - humanities | 2 | none | acc | ↑ | 0.6810 | ± | 0.0064 | |
| - other | 2 | none | acc | ↑ | 0.8072 | ± | 0.0068 | |
| - social sciences | 2 | none | acc | ↑ | 0.8463 | ± | 0.0064 | |
| - stem | 2 | none | acc | ↑ | 0.7146 | ± | 0.0077 |