Downloads · 30 days
14
23% of all-time downloads
elastix-ai/Qwen2.5-0.5B-maskllm-2to4-calib4096-blade
Qwen2.5-0.5B-maskllm-2to4-calib4096-blade is a machine learning model from elastix-ai. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is a compressed version of Qwen/Qwen2.5-0.5B.
Downloads · 30 days
14
23% of all-time downloads
All-time downloads
61
Public
Parameters
494M
1000 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors988 MB · 99%
From the Hugging Face model README
This is a compressed version of Qwen/Qwen2.5-0.5B.
| Field | Value |
|---|---|
| Original model | Qwen/Qwen2.5-0.5B |
| Pruning method | blade |
| Calibration samples | 4096 |
| Calibration sequence length | 2048 |
| Calibration dataset | {"path": "DKYoon/SlimPajama-6B", "split": "validation", "streaming": true} |
| Symmetric calibration | False |
| BEAM fine-tuning | False |
| Convert experts to linear | False |
| GFP checkpoint format | False |
| MLflow run | a0fa646d01a14fc4818c4981255ff55a |
| Pattern | Sparsity | Quantize | Bits | Type | Groupsize | Quant Dim |
|---|---|---|---|---|---|---|
.*embed_tokens$ | 0 | no | 16 | gfp | 32 | -1 |
.*lm_head$ | 0 | no | 16 | gfp | 32 | -1 |
| `(?:^ | .)linear_attn.conv1d$` | 0 | no | 16 | gfp | 32 |
| `(?:^ | .)(?:self_attn | linear_attn).` | 2:4 | no | 16 | gfp |
| `(?:^ | .)mlp.router$` | 0 | no | 16 | gfp | 32 |
.* | 2:4 | no | 16 | gfp | 32 | -1 |
[
{
"pattern": ".*embed_tokens$",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": 0,
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": ".*lm_head$",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": 0,
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": "(?:^|\\.)linear_attn\\.conv1d$",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": 0,
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": "(?:^|\\.)(?:self_attn|linear_attn)\\.",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": "2:4",
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": "(?:^|\\.)mlp\\.router$",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": 0,
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": ".*",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": "2:4",
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
}
]
</details>
<!-- COMPRESSION_EVAL_RESULTS:BEGIN -->
| Dataset | Avg KL | Total KL | Tokens |
|---|---|---|---|
| wikitext2 | 0.439969 | 131459.1317 | 298,792 |
| c4 | 0.365359 | 686803.2745 | 1,879,804 |
| slimpajama_calib | 0.270190 | 2265413.3834 | 8,384,512 |
| Task | acc, None | stderr, None |
|---|---|---|
| arc_challenge | 0.2193 | 0.0121 |
| arc_easy | 0.5320 | 0.0102 |
| hellaswag | 0.3326 | 0.0047 |
| mmlu | 0.2294 | 0.0035 |
| openbookqa | 0.1960 | 0.0178 |
| piqa | 0.6415 | 0.0112 |
| race | 0.3301 | 0.0146 |
| winogrande | 0.5201 | 0.0140 |
max_length=2048)| Dataset | Word PPL | Byte PPL |
|---|---|---|
| WikiText-2 | 30.9934 | 1.9005 |
| C4 (en) | 74.5297 | 2.0570 |
BLADE-Eval: lm-eval 0.4.10, torch 2.13.0+cu130, MLflow run
<!-- COMPRESSION_EVAL_RESULTS:END -->