fr_en-t5-large

Model Card for fr_en-t5-large

<!-- Provide a quick summary of what the model is/does. -->

This model has been optimized for French and English language processing while minimizing overall size. To achieve this, I only retained relevant parameters and tokens specific to these two languages, ensuring that performance remains as good as the original mt5.

Model Details

I used a method outlined in a blog post by David Dale to downsize the multilingual T5 model for French and English use cases specifically. By utilizing the giga_fren dataset, I was able to successfully reduce the total number of tokens and decrease both the model and tokenizer sizes by 38% and 80% respectively.

Model Description

• Developed by: Korventenn
• Model type: mt5
• Language(s) (NLP): French and English
• License: Apache 2.0
• Generated from model: mt5-large

Model Sources [optional]

<!-- Provide the basic links for the model. -->

• Repository: https://colab.research.google.com/drive/1cDWtO5BqWMm_nxnM7lHmPEKMWMejHdBJ#scrollTo=s6ebzRxA1VGv

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

You can use the raw model for any sequence to sequence task that is focused on either french, english or both.

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("Korventenn/fr_en-t5-large")

model = AutoModelForSeq2SeqLM.from_pretrained("Korventenn/fr_en-t5-large")

Training Data

<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

giga_fren