t5-small-custom

Model Card for t5_small Summarization Model

Model Details

This model is Sentiment Analysis model based on T5 model designed by Google Research.

Training Data

Used the CNN/DailyMail dataset for train, validation. Train : 2871 Validation : 134

Training Procedure

batch_size = 4, lr = 2e-5, epochs = 1, weight_decay = 0.01

How to Use

simply use the tranformers library to load the model and tokenizer.

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

input_text = "This is a test sentence" input = tokenizer.encode(input_text, return_tensors="pt") outputs = model(input) print(outputs.logits)

Evaluation

accuracy : 0.84 eval_loss : 0.21156078577041626 BLEU-1 : 38.46

Limitations

Due to small dataset and small epoch, the model may not be able to generalize well to other datasets.

Ethical Considerations

The model is trained on the CNN/DailyMail dataset which is a public dataset.