t5-small-custom

Model Card for t5_small Summarization Model

Model Details

This model is a fine-tuned version of t5_small for summarization tasks.

Training Data

The model was trained on the cnn-dailymail.

Training Procedure

• Learning Rate: 2e-5
• Epochs: 1
• Batch Size: 4

How to Use

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("t5-small")
model = AutoModelForSeq2SeqLM.from_pretrained("t5-small")

input_text = "The movie was fantastic with a gripping storyline!"
inputs = tokenizer.encode(input_text, return_tensors="pt")
outputs = model(inputs)
print(outputs.logits)

Evaluation

Epoch 1 Training Loss: 0.257800 Validation Loss: 0.202847 Rouge1: 29.958418 Rouge2: 12.682260 Rougel: 22.047841 Bleu1: 8.886142 Bleu2: 0.237862 Bleu4: 0.000000

Limitations

The model may generate biased or inappropriate content due to the nature of the training data. It is recommended to use the model with caution and apply necessary filters.

Ethical Considerations

• Bias: The model may inherit biases present in the training data.
• Misuse: The model can be misused to generate misleading or harmful content.