Downloads · 30 days
27
35% of all-time downloads
elastix-ai/Qwen2.5-0.5B-maskllm-beam-2to4-calib4096-blade
Qwen2.5-0.5B-maskllm-beam-2to4-calib4096-blade is a machine learning model from elastix-ai. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is a compressed version of Qwen/Qwen2.5-0.5B.
Downloads · 30 days
27
35% of all-time downloads
All-time downloads
78
Public
Parameters
494M
1000 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors988 MB · 99%
From the Hugging Face model README
This is a compressed version of Qwen/Qwen2.5-0.5B.
| Field | Value |
|---|---|
| Original model | Qwen/Qwen2.5-0.5B |
| Pruning method | blade |
| Calibration samples | 4096 |
| Calibration sequence length | 2048 |
| Calibration dataset | {"path": "DKYoon/SlimPajama-6B", "split": "validation", "streaming": true} |
| Symmetric calibration | False |
| BEAM fine-tuning | True |
| Convert experts to linear | False |
| GFP checkpoint format | False |
| MLflow run | cea16c355591432a8346db4808f36816 |
| Pattern | Sparsity | Quantize | Bits | Type | Groupsize | Quant Dim |
|---|---|---|---|---|---|---|
.*embed_tokens$ | 0 | no | 16 | gfp | 32 | -1 |
.*lm_head$ | 0 | no | 16 | gfp | 32 | -1 |
| `(?:^ | .)linear_attn.conv1d$` | 0 | no | 16 | gfp | 32 |
| `(?:^ | .)(?:self_attn | linear_attn).` | 2:4 | no | 16 | gfp |
| `(?:^ | .)mlp.router$` | 0 | no | 16 | gfp | 32 |
.* | 2:4 | no | 16 | gfp | 32 | -1 |
[
{
"pattern": ".*embed_tokens$",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": 0,
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": ".*lm_head$",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": 0,
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": "(?:^|\\.)linear_attn\\.conv1d$",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": 0,
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": "(?:^|\\.)(?:self_attn|linear_attn)\\.",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": "2:4",
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": "(?:^|\\.)mlp\\.router$",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": 0,
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
},
{
"pattern": ".*",
"quant_config": {
"bits": 16,
"type": "gfp",
"groupsize": 32,
"exp_bits": 5,
"exp_bias": 18,
"expfact_m_bits": 3,
"expfact_m_range_oct": 2.0,
"scale_search": {
"grid_search_steps": 5,
"grid_search_min_ratio": 0.3,
"ls_polish_steps": 2,
"clip_ratio": 1.0
},
"qtip": {
"L": 16,
"T": null,
"hash_bytes": 4,
"kappa": null,
"hash_seed": 0,
"boundary": "prologue",
"candidates": 4
},
"rotate_seed": 0
},
"sparsity": "2:4",
"quantize": false,
"quantize_before_pruning": false,
"quant_dim": -1
}
]
</details>
| Field | Value |
|---|---|
| Ray Tune enabled | False |
| Num samples | 20 |
| Max steps | 100 |
| Min steps | 10 |
| Reduction factor | 3 |
| Search algorithm | optuna |
| Search space | {'lr': {'type': 'loguniform', 'min': 1e-05, 'max': 0.01}, 'batch_size': {'type': 'choice', 'values': [8, 16, 32]}, 'weight_decay': {'type': 'uniform', 'min': 0.0, 'max': 0.1}, 'scheduler_warmup_ratio': {'type': 'uniform', 'min': 0.0, 'max': 0.2}, 'steps': 100, 'optimizer': 'adamw', 'scheduler': 'cosine', 'scheduler_min_lr_ratio': 0.1} |
| Dataset | Avg KL | Total KL | Tokens |
|---|---|---|---|
| wikitext2 | 0.351580 | 105049.3556 | 298,792 |
| c4 | 0.283643 | 533193.6334 | 1,879,804 |
| slimpajama_calib | 0.172846 | 1449229.0732 | 8,384,512 |
| Task | acc, None | stderr, None |
|---|---|---|
| arc_challenge | 0.2423 | 0.0125 |
| arc_easy | 0.5593 | 0.0102 |
| hellaswag | 0.3444 | 0.0047 |
| mmlu | 0.2316 | 0.0036 |
| openbookqa | 0.2140 | 0.0184 |
| piqa | 0.6469 | 0.0112 |
| race | 0.3340 | 0.0146 |
| winogrande | 0.5438 | 0.0140 |
max_length=2048)| Dataset | Word PPL | Byte PPL |
|---|---|---|
| WikiText-2 | 27.8316 | 1.8627 |
| C4 (en) | 68.0583 | 2.0259 |
BLADE-Eval: lm-eval 0.4.10, torch 2.13.0+cu130, MLflow run
<!-- COMPRESSION_EVAL_RESULTS:END -->