conflict_detection_ROBERTA_based

Model Card for XLM-RoBERTa Spanish Conflict Detection Classifier

<!-- Quick summary of what the model is/does. -->

A fine-tuned XLM-RoBERTa model for detecting social conflict mentions in Spanish news articles. The model is trained on the "Conflicto Social en Noticias" dataset and achieves 91.07% macro-F1 score on test data, making it suitable for automated content classification and conflict-related news filtering.

Model Details

Model Description

<!-- Longer summary of what this model is. -->

This is a binary text classification model based on FacebookAI/xlm-roberta-base fine-tuned to detect whether Spanish news articles discuss social conflict or not. The model was trained using a rigorous multi-seed approach (10 random seeds) to ensure robustness and generalization.

The classification task is binary:

• CONFLICTO (1): News articles that discuss social conflict
• NO_CONFLICTO (0): News articles that do not discuss social conflict

The model achieved strong performance across multiple evaluation runs, with consistent metrics indicating reliable predictions on unseen test data.

Model Details

• Developed by: Germán Rosati (Factor~Data, SICSS-Buenos Aires)
• Model type: Transformer-based text classification
• Language(s) (NLP): Spanish (es)
• License: MIT
• Finetuned from model: FacebookAI/xlm-roberta-base

Model Sources

• Repository: https://github.com/gefero/factor_data_tuto_NLP_SICSS
• Dataset: agusnieto77/conflicto-social-noticias-4034

Uses

Direct Use

This model can be used for:

• Automated news classification: Identify news articles discussing social conflict in Spanish-language sources
• Content moderation: Flag conflict-related content for review or categorization
• News aggregation: Filter and organize news by conflict relevance
• Research and analytics: Systematic analysis of conflict coverage in news media
• Social media monitoring: Detect posts discussing social conflict

Downstream Use [optional]

This model can be integrated into:

• News recommendation systems to provide conflict-focused news feeds
• Content management systems for automated news categorization
• Data pipelines for media analysis research
• Misinformation detection systems (as a conflict-detection component)

Out-of-Scope Use

This model is not suitable for:

• Languages other than Spanish (though XLM-RoBERTa is multilingual, the model was fine-tuned only on Spanish data)
• Content moderation decisions without human review (should be used as a scoring/filtering tool, not final arbiter)
• Real-time moderation of live content streams without performance testing in your specific domain
• Classification of informal text, social media, or user-generated content not resembling news articles (model trained on news)

Bias, Risks, and Limitations

Limitations

1. Language: Model trained exclusively on Spanish news articles. Performance on other languages or dialects is unknown.

2. Domain: Model trained on news articles. Performance on other text types (social media, academic text, etc.) may be degraded.

3. Temporal bias: Dataset represents a specific time period. Linguistic evolution and emerging conflict narratives may not be captured.

4. Class balance: Dataset contains both conflict and non-conflict examples. Performance may vary based on class distribution in your specific use case.

5. Truncation: Text is truncated to 256 tokens (matching model's training setup). Very long articles may lose important context.

6. Context sensitivity: "Conflict" detection is based on textual patterns. Sarcasm, irony, or indirect references may be misclassified.

Risks and Biases

• Labeling bias: Model inherits any biases present in the original dataset annotation process
• Geographic bias: News sources and conflict types in training data may not represent all Spanish-speaking regions equally
• Media bias: Model trained on news articles, which may have their own coverage biases
• Potential overreach: Model might flag articles mentioning conflict in non-concerning contexts (e.g., historical analysis, conflict resolution discussion)

Recommendations

1. Always validate: Test the model on your specific data before production deployment
2. Human review: Use model predictions as a starting point for human review, not as final decisions
3. Monitor performance: Track model performance over time and across different domains
4. Document decisions: Keep clear records of how the model is being used and any adjustments made
5. Consider context: Combine model predictions with other signals for robust classification decisions

How to Get Started with the Model

Installation

pip install transformers torch

Quickstart - Using the Pipeline API

from transformers import pipeline

# Initialize the model
classifier = pipeline(
    "text-classification",
    model="gefero/conflict_detection_ROBERTA_based"
)

# Classify text
texts = [
    "El gobierno anunció nuevas políticas de seguridad social",
    "Miles de personas protestaron en las calles contra las medidas económicas"
]

results = classifier(texts)
for text, result in zip(texts, results):
    print(f"Text: {text[:50]}...")
    print(f"Label: {result['label']} (score: {result['score']:.4f})\n")

Quickstart - Using the Model Directly

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "gefero/conflict_detection_ROBERTA_based"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Prepare input
text = "Manifestantes se enfrentan con la policía"
inputs = tokenizer(
    text,
    truncation=True,
    padding="max_length",
    max_length=256,
    return_tensors="pt"
)

# Get predictions
with torch.no_grad():
    outputs = model(**inputs)
    logits = outputs.logits
    predictions = torch.argmax(logits, dim=-1)
    confidence = torch.softmax(logits, dim=-1).max().item()

labels = {0: "NO_CONFLICTO", 1: "CONFLICTO"}
print(f"Prediction: {labels[predictions.item()]} (confidence: {confidence:.4f})")

Training Details

Training Data

• Dataset: agusnieto77/conflicto-social-noticias-4034
• Language: Spanish (es)
• Domain: News articles
• Splits used:
  • Training set: 70% (~2,823 examples)
  • Development set: 10% (~403 examples)
  • Test set: 20% (~806 examples)

Training Procedure

Preprocessing

• Tokenization: XLM-RoBERTa tokenizer
• Max length: 256 tokens (increased from default 128 to capture longer articles)
• Truncation: Long articles truncated to max_length
• Padding: Padded to max_length for batch processing

Training Hyperparameters

• Base model: FacebookAI/xlm-roberta-base
• Learning rate: 2e-5
• Batch size: 16 (training), 64 (evaluation)
• Epochs: 5
• Weight decay: 0.01
• Warmup steps: 0.1 (proportion of total training steps)
• Evaluation strategy: Evaluate at the end of each epoch
• Best model selection: Based on macro-F1 on development set
• Hardware: GPU with FP16 (mixed precision) when available
• Random seeds: 10 seeds [0, 1, 7, 13, 42, 100, 123, 2024, 31337, 65535] for robust evaluation

Training Details

• Framework: Hugging Face Transformers
• Optimizer: AdamW (default)
• Metric for best model: macro-F1 (average of precision and recall across both classes)
• Multi-seed training: Model was trained 10 times with different random seeds to ensure robustness and to provide confidence intervals on performance metrics

Evaluation

Testing Data, Factors & Metrics

Testing Data

• Split: Test set (20% of original data, stratified split)
• Size: ~806 examples
• Language: Spanish
• Domain: News articles from the original dataset

Metrics

• Macro-F1 (primary metric): Average F1-score across both classes

  • Used for model selection during training
  • Balances precision and recall
  • Better for imbalanced or binary classification tasks

• Accuracy: Overall correctness of predictions

• F1-CONFLICTO: F1-score specifically for the CONFLICTO class (conflict-related news)

• Precision: True positives / (true positives + false positives)

• Recall: True positives / (true positives + false negatives)

Results

Official Test Results (10 runs with different seeds)

Per-Seed Results

Summary

The model demonstrates excellent performance with:

• High macro-F1 (0.91): Balanced and strong predictions on both classes
• High accuracy (0.94): Correct classification in 94% of cases
• Robust across seeds: Low standard deviation indicates consistent generalization
• Strong conflict detection (F1-CONFLICTO: 0.86): Reliably identifies conflict-related news

Environmental Impact

Compute Infrastructure

Hardware

• GPU: NVIDIA GPU (exact model unspecified, but typical for Colab)
• CPU: Supporting processors on Colab infrastructure
• RAM: Standard Colab allocation

Training Time

• Per seed: ~5-10 minutes (5 epochs per training run)
• Total: ~50-100 minutes for 10 complete runs
• Cloud Platform: Google Colaboratory (free tier)

Carbon Emissions

Estimated CO2 emissions for multi-seed training approach: Low to minimal (Colab's data centers use renewable energy sources). Individual training runs are short (~10 min each) and performed on highly optimized infrastructure.

For detailed calculations, see ML Impact Calculator.

Technical Specifications

Model Architecture and Objective

• Architecture: Transformer-based sequence classification

  • Base: XLM-RoBERTa (12 layers, 768 hidden dimensions, 110M parameters)
  • Task-specific layer: Linear classification head for 2 classes

• Objective: Binary cross-entropy loss (standard for text classification)

• Multilingual base: XLM-RoBERTa trained on 100+ languages, fine-tuned here for Spanish-specific conflict detection

Input/Output

• Input: Spanish text (news articles)
• Max length: 256 tokens
• Output: Class probabilities for [NO_CONFLICTO, CONFLICTO]

Citation [optional]

If you use this model in research, please cite:

BibTeX:

@software{rosati2024conflictdetection,
  author = {Rosati, Germán},
  title = {XLM-RoBERTa Spanish Conflict Detection Classifier},
  year = {2024},
  publisher = {Hugging Face Hub},
  url = {https://huggingface.co/gefero/conflict_detection_ROBERTA_based}
}

APA:

Rosati, G. (2024). XLM-RoBERTa Spanish Conflict Detection Classifier [Machine learning model]. Hugging Face Hub. Retrieved from https://huggingface.co/gefero/conflict_detection_ROBERTA_based

Model Card Authors

• Germán Rosati (Factor~Data, SICSS-Buenos Aires)

Model Card Contact

• Email: [email protected]
• GitHub: https://github.com/gefero