Downloads · 30 days
20
38% of all-time downloads
Qwen-27B-Pruned-Retrained
Columbidae/Qwen-27B-Pruned-Retrained
Qwen-27B-Pruned-Retrained is a machine learning model from Columbidae. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is ToastyPigeon/qwen2.5-32b-unnamed-test-model pruned down from 32b - 27b.
All-time downloads
53
Public
Parameters
26.9B
53.8 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors53.8 GB · 100%
At a glance
- Model type
- qwen2
- Access
- Public
- Created
- Feb 18, 2025
- Updated
- Mar 8, 2025
- SHA
- b0f80589
Base models
Model card
From the Hugging Face model README
Pruned Qwen (Epoch 1)
This is ToastyPigeon/qwen2.5-32b-unnamed-test-model pruned down from 32b -> 27b.
Using PruneMe to find layers to remove resulted in the removal of layers [25, 29) and [36, 43) for a reduction from 64 -> 52 layers.
Trained on 1 epoch of mixed data from the datasets that went into the pre-pruned model (I'll document that later), totaling about ~10M tokens so far of retraining.
Coherent but a little dumb. Likely needs more than 10M tokens of retraining to re-align the layers.
- Type
- qwen2
- Created
- Feb 18, 2025
- Updated
- Mar 8, 2025