xlm-tourism-topic-classifier

XLM-RoBERTa Tourism Topic Classifier

Fine-tuned XLM-RoBERTa model for multilingual tourism comment topic classification.

Model Description

This model classifies multilingual tourism comments into 7 topics:

• scenery (Landscapes, views)
• food (Cuisine, dining)
• service (Staff, customer service)
• pricing (Cost, value)
• facilities (Infrastructure, amenities)
• activities (Things to do)
• accessibility (Transportation, location)

Training Data

• Languages: 39+ languages (English, Korean, Russian, German, French, etc.)
• Dataset: 462 non-Vietnamese tourism comments from social media
• Sources: Facebook, TikTok, YouTube, Google Maps
• Quality: Filtered for meaningful content with rule-based topic labels

Performance

Language Distribution in Training

Usage

import torch
from transformers import AutoTokenizer

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained('xlm-roberta-base')

# Load model
model = torch.load('xlm_best_model.pt')
model.eval()

# Predict (works with any language)
text = "Beautiful scenery, great food, a bit expensive"
encoding = tokenizer(text, return_tensors='pt', max_length=256,
                     padding='max_length', truncation=True)

with torch.no_grad():
    logits = model(encoding['input_ids'], encoding['attention_mask'])
    probs = torch.sigmoid(logits)
    predictions = (probs > 0.3).float()  # Lower threshold for multilingual

# Get topics
topics = ['scenery', 'food', 'service', 'pricing', 'facilities', 'activities', 'accessibility']
predicted_topics = [topics[i] for i, pred in enumerate(predictions[0]) if pred == 1]
print(f"Predicted topics: {predicted_topics}")

Training Details

• Base Model: xlm-roberta-base
• Architecture: XLM-RoBERTa + Classification Head (768 → 7)
• Parameters: ~270M (base) + 5,376 (classifier)
• Training Time: ~15-20 minutes on RTX 4060
• Epochs: 5
• Batch Size: 16
• Learning Rate: 2e-5
• Optimizer: AdamW
• Loss: BCEWithLogitsLoss (multi-label)

Complementary to PhoBERT

This model is designed to work alongside PhoBERT:

• PhoBERT: Vietnamese comments (79% of data, F1: 56.38%)
• XLM-RoBERTa: Non-Vietnamese comments (21% of data, F1: 53.74%)

Limitations

• Limited training data (462 samples)
• Performance lower than PhoBERT due to less data
• May not work well for very rare languages
• Trained on tourism domain only

Citation

@misc{xlm-tourism-classifier,
  author = {Your Name},
  title = {XLM-RoBERTa Tourism Topic Classifier},
  year = {2025},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/Strawberry0604/xlm-tourism-topic-classifier}}
}

Contact

• Repository: tourism-data-monitor