Downloads · 30 days
8
20% of all-time downloads
ThomasTheMaker/Llama3.1-6B-ReplaceMe-Healed-rkllm-v1.2.0
Llama3.1-6B-ReplaceMe-Healed-rkllm-v1.2.0 is a machine learning model from ThomasTheMaker. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
[](https://arxiv.org/abs/2505.02819) [](https://opensource.org/licenses/Apache-2.0)
Downloads · 30 days
8
20% of all-time downloads
All-time downloads
41
Public
Repo size
6.9 GB
Likes
0
Public
Click a slice to open those files.
.rkllm6.8 GB · 100%
From the Hugging Face model README

ReplaceMe is a novel method for transformer model compression that enables training-free block/layer pruning while maintaining model performance through linear transformations. The approach:
| Method | num_pruned_layers | Dataset | State | race 🏁 | winogrande 🎲 | piqa 🧠 | boolq ❓ | openbookqa 📖 | sciq 🔬 | lambada_openai 🦙 | ppl | Avg-acc 📊 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| acc | acc | acc_norm | acc | acc_norm | acc_norm | acc | ||||||
| Llama 3.1 (baseline) | - | - | - | 0.450 | 0.779 | 0.810 | 0.842 | 0.430 | 0.961 | 0.732 | 3.404 | 0.712 |
| UIDL* | 8 | slim_orca | no training | 0.341 | 0.719 | 0.690 | 0.773 | 0.310 | 0.719 | 0.087 | 932.000 | 0.592 |
| ReplaceMe (Ours) ✅ | 8 | slim_orca | no training | 0.406 | 0.742 🏆 | 0.706 | 0.830 | 0.338 | 0.901 | 0.471 | 16.760 | 0.654 |
| ReplaceMe (Ours) ❌ | 8 | slim_orca | SFT | 0.431 🏆 | 0.716 | 0.728 🏆 | 0.849 🏆 | 0.378 🏆 | 0.912 🏆 | 0.697 🏆 | 4.04 🏆 | 0.669 🏆 |
Key:
Metrics Explained:
🔥 Our Healed model can acheive 94.0% of baseline performance after healing on 1B tokens!
pip install replaceme
# or
git clone https://github.com/mts-ai/ReplaceMe
cd ReplaceMe
pip install -e .
# LSTSQ method (recommended)
run_replaceme --config ./reproduce/Replace_Me_pipeline_lstsq.yaml
# Cosine similarity method
run_replaceme --config ./reproduce/Replace_Me_pipeline_cosine.yaml
There are many parameters you can play with, visit our repo and dscover 🔥🔥
As we said we are merging the LTs with the original transformer architecture so you just do it as usual
## EXAMPLE
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "MTSAIR/Llama3.1-6B-ReplaceMe-Healed"
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
prompt = "What is ReplaceME pruning method?!"
messages = [
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
output = model.generate(
**model_inputs,
max_new_tokens=512
)
response = tokenizer.batch_decode(output, skip_special_tokens=True)[0]
If you use ReplaceMe in your research, please cite our paper:
@article{shopkhoev2025replaceme0,
title = {ReplaceMe: Network Simplification via Layer Pruning and Linear Transformations},
author = {Dmitriy Shopkhoev and Ammar Ali and Magauiya Zhussip and Valentin Malykh and Stamatios Lefkimmiatis and Nikos Komodakis and Sergey Zagoruyko},
year = {2025},
journal = {arXiv preprint arXiv: 2505.02819}
}