multilingual-ai-text-detector-xlmr

Multilingual AI Text Detector (XLM-RoBERTa)

A transformer-based baseline classifier for detecting AI-generated vs human-generated text across multiple languages.

Features

• Automatic dataset discovery from CSV files
• Support for multilingual text classification
• XLM-RoBERTa model fine-tuning
• Comprehensive evaluation metrics (accuracy, F1, ROC-AUC, calibration)
• Per-language performance analysis
• Temperature scaling for calibration

Installation

pip install -r requirements.txt

Usage

Train a model:

python -m baselines.train_transformer --data_dir data --model xlm-roberta-base

Key Arguments

• --data_dir: Path to directory containing CSV files
• --model: HuggingFace model name (default: xlm-roberta-base)
• --epochs: Number of training epochs (default: 3)
• --batch_size: Training batch size (default: 16)
• --lr: Learning rate (default: 2e-5)
• --seed: Random seed (default: 42)
• --temperature_scaling: Enable temperature scaling calibration (default: False)

Data Format

CSV files should contain:

• A text column (aliases: input_text, summary, output_text, text, content)
• A label column (aliases: label, y, target, is_ai, ai, class)
• Optional: language column for multilingual datasets
• Optional: group_id column for group-aware splitting

Labels can be:

• Integers: 0 (human) or 1 (AI)
• Strings: "human"/"ai", "real"/"fake", "generated", etc.

Output

Results are saved to out/transformer/<model>/<run_name>/:

• best_model/: Saved model checkpoint
• report.md: Evaluation report
• metrics_test.json: Test set metrics
• predictions_test.parquet: Predictions on test set
• Plots: ROC curve, PR curve, confusion matrix, reliability diagram