flan-t5-sarcasm

Model Card for Sarcasm-Enhanced Sentiment Analysis Model

This model performs sentiment analysis with a specific focus on detecting sarcasm in textual content. It is fine-tuned on a combination of standard sentiment datasets and specialized sarcastic data, allowing for more nuanced sentiment classification that accounts for sarcastic language.

Model Details

Model Description

This model leverages the Flan-T5-small transformer architecture, fine-tuned on datasets including IMDB for sentiment analysis and bharatiyabytes/sentimentWithSarcasm for sarcasm detection. By combining these datasets, the model is better equipped to differentiate between sarcastic and genuine sentiment expressions, improving sentiment analysis accuracy in contexts where sarcasm is prevalent.

• Developed by: bharatiyabytes
• Funded by [optional]: Not yet funded
• Model type: Text Classification
• Language(s) (NLP): English (en)
• License: Apache 2.0
• Fine-tuned from model: google/flan-t5-small

Model Sources [optional]

• Repository: https://github.com/sohi-g/lets-talk-the-hype.git
• Paper [optional]: [Link to paper if available]
• Demo [optional]: [Link to demo if available]

Uses

Direct Use

This model can be used directly for sentiment classification on texts where sarcasm may obscure intended sentiment. It provides accurate classifications by considering nuanced expressions, making it ideal for social media analysis, customer feedback processing, and sarcasm-rich content sources.

Downstream Use

This model can serve as a base for further fine-tuning for domains that require sarcasm-aware sentiment analysis, such as customer service, public relations, and social media monitoring applications.

Out-of-Scope Use

The model may not perform well on texts with heavy dialects or informal language that goes beyond the sarcasm in the fine-tuning data. It is not intended for multi-lingual sarcasm detection.

Bias, Risks, and Limitations

As with many sentiment analysis models, there may be inherent biases in the sarcasm and sentiment labels in the training datasets, potentially affecting model performance across different demographic or cultural groups. Users should be cautious when using this model in critical decision-making contexts.

Recommendations

Users should perform additional validation on specific datasets to ensure that model predictions align with intended use cases, especially in high-stakes applications.

How to Get Started with the Model

Use the code below to get started with the model:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("your-username/sarcasm-sentiment-analysis")
model = AutoModelForSequenceClassification.from_pretrained("your-username/sarcasm-sentiment-analysis")

inputs = tokenizer("Your input text here", return_tensors="pt")
outputs = model(**inputs)
predictions = torch.argmax(outputs.logits, dim=-1)

Training Details

Training Data

The model was fine-tuned on a combination of IMDB (a general sentiment analysis dataset) and bharatiyabytes/sentimentWithSarcasm (designed to capture sarcastic sentiment). This blend improves the model’s ability to identify nuanced sentiment.

Training Procedure

Preprocessing

Standard text preprocessing methods were applied, such as tokenization and lowercase transformation.

Training Hyperparameters

Training regime: FP32 Epochs: 3 Batch Size: 16 Learning Rate: 3e-5

Speeds, Sizes, Times

Training took approximately 4 hours on an NVIDIA V100 GPU with a model size of 60M parameters.

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Data, Factors & Metrics

Testing Data

he model was evaluated on the IMDB and bharatiyabytes/sentimentWithSarcasm test splits.

Factors

Evaluation considers sentiment disaggregation by sarcastic vs. non-sarcastic samples.

Metrics

-Accuracy: Measures overall sentiment classification accuracy. -F1 Score (Sarcasm): Evaluates the model’s sarcasm detection capability, which is key for accurate sarcastic sentiment handling.

Results

The model achieved an accuracy of 88% on the sentiment classification task and an F1 score of 0.83 on sarcasm detection.

Summary

The model shows strong performance in sarcasm-sensitive sentiment analysis, making it suitable for applications where nuanced sentiment interpretation is crucial.

Model Examination

The model’s predictions have been examined to ensure that sarcastic content is accurately labeled, using interpretability tools such as SHAP to visualize model attention on sarcastic phrases.

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: NVIDIA V100 GPU
• Hours used: Approximately 4 hours
• Cloud Provider: Google
• Compute Region: USA
• Carbon Emitted: NA

Technical Specifications

Model Architecture and Objective

The model uses the Flan-T5-small architecture fine-tuned for binary sentiment classification with sarcasm detection as an enhancement.

Compute Infrastructure

[More Information Needed]

Hardware

NVIDIA V100 GPU

Software

Hugging Face Transformers Library, PyTorch

Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors

• Shivang sinha
• Garima Sohi
• Parteek

Model Card Contact

[email protected]