multilingual-sentiment-classifier

advexon/multilingual-sentiment-classifier

Multilingual text classification model trained on XLM-RoBERTa base for sentiment analysis across English, Russian, Tajik and other languages

Model Description

This is a multilingual text classification model based on XLM-RoBERTa. It has been fine-tuned for sentiment analysis across multiple languages and can classify text into positive, negative, and neutral categories.

Model Details

• Base Model: XLM-RoBERTa Base
• Model Type: XLMRobertaForSequenceClassification
• Number of Labels: 3 (Negative, Neutral, Positive)
• Languages: Multilingual (English, Russian, Tajik, and others)
• Max Sequence Length: 512 tokens
• Hidden Size: 768
• Attention Heads: 12
• Layers: 12

Performance

Based on training metrics:

• Training Accuracy: 58.33%
• Validation Accuracy: 100%
• Training Loss: 0.94
• Validation Loss: 0.79

Usage

Using the Model

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load the model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("advexon/multilingual-sentiment-classifier")
model = AutoModelForSequenceClassification.from_pretrained("advexon/multilingual-sentiment-classifier")

# Example usage
text = "This product is amazing!"
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
outputs = model(**inputs)
predictions = torch.softmax(outputs.logits, dim=-1)
predicted_class = torch.argmax(predictions, dim=1).item()

# Class mapping: 0=Negative, 1=Neutral, 2=Positive
sentiment_labels = ["Negative", "Neutral", "Positive"]
predicted_sentiment = sentiment_labels[predicted_class]
print(f"Predicted sentiment: {predicted_sentiment}")

Example Predictions

• "I absolutely love this product!" → Positive
• "This is terrible quality." → Negative
• "It's okay, nothing special." → Neutral
• "Отличный сервис!" → Positive (Russian)
• "Хунуки хуб нест" → Negative (Tajik)

Training

This model was trained using:

• Base Model: XLM-RoBERTa Base
• Optimizer: AdamW
• Learning Rate: 2e-5
• Batch Size: 16
• Training Epochs: 2
• Languages: English, Russian, Tajik

Model Architecture

The model uses the XLM-RoBERTa architecture with:

• 12 transformer layers
• 768 hidden dimensions
• 12 attention heads
• 3 classification heads for sentiment analysis

Limitations

• The model's performance may vary across different languages
• It is recommended to fine-tune on domain-specific data for optimal performance
• Maximum input length is 512 tokens
• Performance may be lower on languages not well-represented in the training data

Citation

If you use this model in your research, please cite:

@misc{multilingual-text-classifier,
  title={Multilingual Text Classification Model},
  author={Advexon},
  year={2024},
  publisher={Siyovush Mirzoev},
  journal={Hugging Face Hub},
  howpublished={\url{https://huggingface.co/advexon/multilingual-sentiment-classifier}},
}

Contact me

mirzoevsish@gmail.com / +992 710707777 WhatsApp