Downloads · 30 days
10
21% of all-time downloads
elastix-ai/Qwen2.5-0.5B-maskllm-2to4-calib512-weakscale-blade
Qwen2.5-0.5B-maskllm-2to4-calib512-weakscale-blade is a machine learning model from elastix-ai. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is a compressed version of Qwen/Qwen2.5-0.5B.
Downloads · 30 days
10
21% of all-time downloads
All-time downloads
47
Public
Parameters
494M
1000 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors988 MB · 99%
From the Hugging Face model README
This is a compressed version of Qwen/Qwen2.5-0.5B.
| Field | Value |
|---|---|
| Original model | Qwen/Qwen2.5-0.5B |
| Pruning method | blade |
| Calibration samples | 512 |
| Calibration sequence length | 2048 |
| Calibration dataset | {"path": "DKYoon/SlimPajama-6B", "split": "validation", "streaming": true} |
| Symmetric calibration | False |
| BEAM fine-tuning | False |
| Convert experts to linear | False |
| GFP checkpoint format | False |
| MLflow run | 852bf83aa68e46be9af888a1d8fb2386 |
| Pattern | Sparsity | Quantize | Bits | Type | Groupsize | Quant Dim |
|---|---|---|---|---|---|---|
.*embed_tokens$ | 0 | no | 16 | gfp | 32 | -1 |
.*lm_head$ | 0 | no | 16 | gfp | 32 | -1 |
| `(?:^ | .)linear_attn.conv1d$` | 0 | no | 16 | gfp | 32 |
| `(?:^ | .)(?:self_attn | linear_attn).` | 2:4 | no | 16 | gfp |
| `(?:^ | .)mlp.router$` | 0 | no | 16 | gfp | 32 |
.* | 2:4 | no | 16 | gfp | 32 | -1 |
[
{
"pattern": ".*embed_tokens$",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": 0,
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": ".*lm_head$",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": 0,
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": "(?:^|\\.)linear_attn\\.conv1d$",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": 0,
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": "(?:^|\\.)(?:self_attn|linear_attn)\\.",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": "2:4",
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": "(?:^|\\.)mlp\\.router$",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": 0,
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": ".*",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": "2:4",
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
}
]
</details>
<!-- COMPRESSION_EVAL_RESULTS:BEGIN -->
| Dataset | Avg KL | Total KL | Tokens |
|---|---|---|---|
| wikitext2 | 0.572866 | 3147.3253 | 5,494 |
| c4 | 0.542486 | 19991.6859 | 36,852 |