Downloads · 30 days
4
4% of all-time downloads
SLAB-Llama-350M
OpenEfficientAI/SLAB-Llama-350M
SLAB-Llama-350M is a machine learning model from OpenEfficientAI. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
An unofficial reproduced PRepBN-Llama-350M checkpoints for SLAB.
All-time downloads
100
Public
Repo size
2.9 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin1.5 GB · 100%
At a glance
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Aug 23, 2024
- Updated
- Aug 23, 2024
- SHA
- e3301184
Model card
From the Hugging Face model README
<!-- Provide a quick summary of what the model is/does. -->
An unofficial reproduced PRepBN-Llama-350M checkpoints for SLAB.
Model Sources [optional]
<!-- Provide the basic links for the model. -->- Repository: [https://github.com/xinghaochen/SLAB/]
- Paper [optional]: [https://arxiv.org/abs/2405.11582]
Evaluation
<!-- This section describes the evaluation protocols and provides the results. -->https://github.com/xinghaochen/SLAB/tree/main/llama
python evaluation.py --ckpt <checkpoint-path>
BibTeX:
@inproceedings{guo2024slab,
title={SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization},
author={Guo, Jialong and Chen, Xinghao and Tang, Yehui and Wang, Yunhe},
booktitle={International Conference on Machine Learning},
year={2024}
}
- Type
- llama
- License
- mit
- Created
- Aug 23, 2024
- Updated
- Aug 23, 2024