Downloads · 30 days
0
ra225/Qwen2.5-7B-pruned-70p-rmsnorm-finetuned
Qwen2.5-7B-pruned-70p-rmsnorm-finetuned is a machine learning model from ra225. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This model is a pruned and finetuned version of Qwen/Qwen2.5-7B, retaining approximately 70% of parameters while maintaining strong performance through genetic algorithm pruning and RMSNorm fine-tuning.
Downloads · 30 days
0
Access
Public
Updated Jan 21, 2026
Repo size
15.2 GB
Likes
0
Public
Click a slice to open those files.
.pt15.2 GB · 100%
From the Hugging Face model README
This model is a pruned and finetuned version of Qwen/Qwen2.5-7B, retaining approximately 70% of parameters while maintaining strong performance through genetic algorithm pruning and RMSNorm fine-tuning.
| Metric | Value |
|---|---|
| PPL (Before Fine-tuning) | 15.23 |
| PPL (After Fine-tuning) | 11.12 |
| Improvement | 27.00% |
This 70% model is part of a family of pruned models with different compression ratios:
| Model | PPL (After FT) |
|---|---|
| 50% params | 25.81 |
| 70% params | 11.12 |
| 80% params | 8.71 |
| 90% params | 7.64 |
import torch
from transformers import AutoTokenizer
# Load tokenizer (standard Qwen2.5-7B tokenizer)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B")
# Note: This model uses a custom pruned architecture
# Custom loading code is required to use this model
model_weights.pt: Full model state dictREADME.md: This documentationApache 2.0 (inherited from base model)