flan-t5-small-title

Flan-T5 Small Title Generator

Model Description

This model is a fine-tuned version of the Flan-T5 small model, specifically adapted for generating attention-grabbing titles based on given text. Flan-T5 is an improved version of the T5 (Text-To-Text Transfer Transformer) model developed by Google, which has been instruction-tuned on a diverse set of tasks.

• Architecture: Flan-T5 small
• Purpose: Generate engaging titles from input text
• Base Model: google/flan-t5-small

Intended Uses & Limitations

Intended Uses

• Generating catchy titles for articles, blog posts, or news stories
• Summarizing key points of a text passage into a concise headline
• Assisting content creators in brainstorming title ideas

Limitations

• Requires clear context from the input paragraph to generate relevant titles
• May produce exaggerated or off-topic titles if the context is ambiguous
• Outputs should always be reviewed by a human before use
• Not suitable for generating titles for sensitive or critical content without human oversight

Training Details

Training Data

The model was fine-tuned on the "Wikipedia Paragraphs and AI-Generated Titles Dataset" (agentlans/wikipedia-paragraph-titles), which contains:

• Pairs of Wikipedia paragraphs and corresponding AI-generated titles
• A mix of human-written content and machine-generated titles
• Diverse topics from Wikipedia articles

<details> <summary>Training details</summary>

Training Procedure

• Base Model: google/flan-t5-small
• Fine-tuning Approach: Further trained on the title generation task
• Input Format: topic || text
• Output Format: Attention-grabbing title based on the input text

Training Hyperparameters

• Learning rate: 5e-05
• Train batch size: 8
• Eval batch size: 8
• Seed: 42
• Optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• LR scheduler type: linear
• Number of epochs: 10.0

The model was trained using the following framework versions:

• Transformers 4.45.1
• PyTorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.0

</details>

Ethical Considerations & Biases

• The model may inherit biases present in the Wikipedia content used for training
• There's a risk of generating sensationalized or misleading titles, especially for ambiguous content
• Users should be aware of potential biases in title generation, particularly for sensitive topics
• The model should not be used as the sole source for generating titles in professional or journalistic contexts without human review

Usage

To use the model, follow these steps:

1. Input format: topic||text
2. The model will generate an attention-grabbing title based on the input text
3. Always review the output for relevance and appropriateness

Example Usage

Here's a code example demonstrating how to use the Flan-T5 small model for title generation:

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_name = "agentlans/flan-t5-small-title"
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Prepare the input text
topic = "The Serenity of Nature" # a cue to establish context (not necessary but recommended)
text = "As dawn breaks, the world awakens to a symphony of colors and sounds. The golden rays of sunlight filter through the leaves, casting playful shadows on the forest floor. Birds chirp melodiously, their songs weaving through the crisp morning air, while a gentle breeze rustles the branches overhead. Dew-kissed flowers bloom in vibrant hues, their fragrant scents mingling with the earthy aroma of damp soil. In this tranquil setting, one can’t help but feel a profound sense of peace and connection to the natural world, reminding us of the simple joys that life has to offer."

input_text = f"{topic}||{text}"

# Tokenize the input
inputs = tokenizer(input_text, return_tensors="pt", max_length=512, truncation=True)

# Generate the title
outputs = model.generate(**inputs, max_length=30, num_return_sequences=1)

# Decode and print the generated title
generated_title = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(generated_title) # The Serenity of Nature: A Symbol of Peace and Harmony

License

This model is released under the Apache 2.0 license.