finetune_T5_small_title_generation_NLP_cours

Model Card for T5 Small Fine-Tuned for Title Generation

This model fine-tunes the T5-small model from Hugging Face for the task of news article title generation. It has been trained on a dataset where each input sequence is a body of text, and the task is to generate a coherent and meaningful title that summarizes the content.

The dataset for Title Generation comprises a collection of articles along with their corresponding titles and subjects. Each entry in the dataset includes the full text of an article, its subject category, and the title that summarizes its content. This batchset serves as a valuable resource for natural language processing tasks, particularly in the domain of text summarization and headline generation. Researchers, developers, and enthusiasts can leverage this batchset to train machine learning models for automatic title generation, headline summarization, and related applications. With a diverse range of subjects and article texts, this batchset offers ample opportunities for experimentation and innovation in the field of natural language processing.

link : https://www.kaggle.com/datasets/thejas2002/titlegen

Model Details

Model Description

This fine-tuned model is based on google/t5-small and has been trained to generate titles for textual inputs. The model was fine-tuned using the 🤗 Transformers library, and ROUGE was used as an evaluation metric to assess performance during training.

• Developed by: Ivanhoe9
• Model type: T5 (sequence-to-sequence)
• Language(s) (NLP): English
• License: Apache-2.0
• Finetuned from model: google/t5-small

Model Sources

• Repository: Hugging Face Hub - T5 Title Generation

Uses

Direct Use

The model can be directly used for generating titles from text. It is well-suited for summarizing articles, blogs, or any content that requires a concise title.

Out-of-Scope Use

The model might not perform well on texts that are highly technical, ambiguous, or outside its training domain.

Bias, Risks, and Limitations

This model may generate biased or inappropriate titles if the input text contains sensitive or biased content. Users should validate generated titles before using them in production.

How to Get Started with the Model

You can use the following code to start generating titles:

from transformers import T5ForConditionalGeneration, T5Tokenizer

tokenizer = T5Tokenizer.from_pretrained("Ivanhoe9/finetune_T5_small_title_generation_NLP_cours")
model = T5ForConditionalGeneration.from_pretrained("Ivanhoe9/finetune_T5_small_title_generation_NLP_cours")

article = """Neutral Switzerland is in touch with Spain and Catalonia about resolving a row over the region s moves towards
independence but conditions for promoting talks are not yet ripe, the foreign ministry in Bern said on Friday. Facilitation
can only be provided if both parties request it. Switzerland is in contact with both parties, but the conditions for facilitation
are not in place at this stage, a spokesman said by email in response to a query. He described the situation in Catalonia as an internal
Spanish political matter and said Switzerland respected the sovereignty of Spain."""

input_text = "Generate a title: " + article
input_ids = tokenizer.encode(input_text, return_tensors="pt")

# Generate title
generated_ids = model.generate(input_ids, max_length=30, num_beams=4, early_stopping=True)
title = tokenizer.decode(generated_ids[0], skip_special_tokens=True)
print(title)

Training Details

Training Data

The model was trained on a dataset of texts and their respective titles, preprocessed to include the prefix "Generate a title:" to guide the T5-small architecture. The data was tokenized using Hugging Face's T5 tokenizer.

Training Procedure

Preprocessing

• Input texts were cleaned using a custom clean_text function.
• A prefix "Generate a title:" was added to guide the model.

Training Hyperparameters

• Batch size: 32
• Number of epochs: 9
• Learning rate: 1e-4
• Optimizer: Adam
• Loss function: Cross-Entropy Loss

Checkpoints and Early Stopping

• Checkpoints were saved based on validation loss improvements.
• Early stopping was used with patience set to avoid overfitting.

Evaluation

Testing Data, Factors & Metrics

Testing Data

Evaluation was performed using a portion of the dataset held out during training.

Metrics

• ROUGE-1 and ROUGE-2 scores were computed during training to assess how well the generated titles captured relevant content from the input text.

Environmental Impact

The carbon footprint and energy consumption can be estimated using tools like the Machine Learning Impact Calculator.

• Hardware Type: GPU A100 (NVIDIA)
• Hours used: 0.5 H
• Compute Region: GLICID HPC

Technical Specifications

Model Architecture and Objective

• T5-small with a sequence-to-sequence objective.

Citation

If you use this model, please cite the following:

@misc{ivanhoe2025t5tune,
  title={Fine-Tuning T5 Small for Title Generation},
  author={Ivanhoe9},
  year={2025},
  url={https://huggingface.co/Ivanhoe9/finetune_T5_small_title_generation_NLP_cours}
}