t5-small-custom

Model Card for t5-small based Text Summarization Model

(boostcamp ai tech huggingface utilization task test upload)

Model Details

This model is a fine-tuned version of t5-small-base for summarization tasks.

Training Data

The model was trained on the CNN/Daily Mail dataset.

Training Procedure

• Learning Rate: 2e-5
• Epochs: 1
• Batch Size: 4

How to Use

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("test_work3")
model = AutoModelForSeq2SeqLM.from_pretrained("test_work3")

input_text = "Your input text here"
inputs = tokenizer(input_text, return_tensors="pt", max_length=512, truncation=True)
outputs = model.generate(**inputs, max_length=100)
decoded_output = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(decoded_output)

Evaluation

• Training Loss: 0.2661
• Validation Loss: 0.2014
• ROUGE-1: 23.25
• ROUGE-2: 8.76
• ROUGE-L: 18.92
• BLEU-1: 40.37
• BLEU-2: 15.69
• BLEU-4: 4.58

Limitations

This is the test model created for the assignment. The model may generate biased or inappropriate content due to the nature of the training data. It is recommended to use the model with caution and apply necessary filters.

Ethical Considerations

• Bias: The model may inherit biases present in the training data.
• Misuse: The model can be misused to generate misleading or harmful content.

Copyright and License

This model is licensed under the MIT License.