Downloads · 30 days
6
4% of all-time downloads
FAT5-xl-flan-en
Thalesian/FAT5-xl-flan-en
FAT5-xl-flan-en is a machine learning model from Thalesian. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Adaptation of the flan-t5-xl weights to make it compatible with the FAT5 framework (Flash Attention T5). This adaptation should enable the user to efficiently continue the pre-training of the flan-t5 to adapt it to mo…
All-time downloads
169
Public
Parameters
2.8B
11.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors11.4 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- flash_t5
- Access
- Public
- Created
- Nov 21, 2025
- Updated
- Nov 21, 2025
- SHA
- 370e54b3
Model card
From the Hugging Face model README
Description
Adaptation of the flan-t5-xl weights to make it compatible with the FAT5 framework (Flash Attention T5).
This adaptation should enable the user to efficiently continue the pre-training of the flan-t5 to adapt it to more recent data, or to specialize it in a specific domain, for example.
Usage
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("CATIE-AQ/FAT5-xl-flan-en", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-xl")
Note: we forgot to push a file we need to find (cf. https://huggingface.co/CATIE-AQ/FAT5-xl-flan-en/discussions/1 🙃
- Type
- flash_t5
- License
- apache-2.0
- Languages
- en
- Created
- Nov 21, 2025
- Updated
- Nov 21, 2025