Roberta-base-ai_human-classifier

AI vs Human Text Classification Model

Overview

This repository contains a fine-tuned RoBERTa-based model for binary text classification that distinguishes between human-written and AI-generated text. The model is trained using Parameter-Efficient Fine-Tuning (PEFT) with LoRA (Low-Rank Adaptation) for optimal performance and resource efficiency.

Model Architecture

• Base Model: roberta-base
• Task Type: Sequence Classification (Binary)
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Classes:
  • HUMAN (label 0): Human-written text
  • AI (label 1): AI-generated text

LoRA Configuration

• Rank (r): 8
• Alpha: 16
• Dropout: 0.1
• Target Modules: ["query", "value"] (attention layers)

Model Files

The model directory contains the following files:

• config.json - Model configuration and label mappings
• model.safetensors - Model weights in SafeTensors format
• tokenizer.json - Fast tokenizer configuration
• tokenizer_config.json - Tokenizer configuration
• vocab.json - Vocabulary mappings
• merges.txt - BPE merge operations
• special_tokens_map.json - Special token mappings

Training Script Usage

The model is trained using the trainer_root/trainer/task.py script, designed for Google Cloud Vertex AI training jobs.

Command Line Arguments

python task.py \
  --model-dir gs://your-bucket/model-output \
  --lora-dir gs://your-bucket/lora-output \
  --train-data gs://your-bucket/train-data.csv \
  --valid-data gs://your-bucket/valid-data.jsonl \
  --train-data-percentage 1.0 \
  --valid-data-percentage 1.0 \
  --epochs 3 \
  --batch-size 8 \
  --create-mar

Required Arguments

• --model-dir: GCS path for model output
• --lora-dir: GCS path for LoRA adapter output
• --train-data: GCS path to CSV training dataset
• --valid-data: GCS path to JSONL validation dataset

Optional Arguments

• --train-data-percentage: Percentage of training data to use (0-1, default: 1.0)
• --valid-data-percentage: Percentage of validation data to use (0-1, default: 1.0)
• --epochs: Number of training epochs (default: 3)
• --batch-size: Training batch size (default: 8)
• --create-mar: Flag to create TorchServe .mar file for deployment

Data Format

Training Data (CSV)

The training dataset should be in CSV format with the following columns:

• text: The input text to classify
• label: The ground truth label (HUMAN or AI)

Validation Data (JSONL)

The validation dataset should be in JSONL format with each line containing:

{"text": "Sample text content", "label": "HUMAN"}

Model Performance Metrics

The training script tracks the following metrics:

• Accuracy: Overall classification accuracy
• Precision: Weighted precision across classes
• Recall: Weighted recall across classes
• F1-Score: Weighted F1-score across classes

Deployment Options

1. HuggingFace Transformers (Direct Loading)

from transformers import RobertaTokenizerFast, RobertaForSequenceClassification

# Load model and tokenizer
model = RobertaForSequenceClassification.from_pretrained('./model')
tokenizer = RobertaTokenizerFast.from_pretrained('./model')

# Classify text
def classify_text(text):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
    outputs = model(**inputs)
    predictions = torch.softmax(outputs.logits, dim=-1)
    predicted_class = torch.argmax(predictions, dim=-1)
    
    label_map = {0: "HUMAN", 1: "AI"}
    return label_map[predicted_class.item()]

2. TorchServe Deployment

The training script can generate a .mar file for TorchServe deployment when the --create-mar flag is used. This creates a production-ready model archive with:

• Custom handler (roberta_handler.py)
• Model weights and configuration
• Tokenizer files
• Label mappings

TorchServe Usage

# Start TorchServe with the model
torchserve --start --model-store /path/to/model-store --models roberta_classifier=roberta_classifier.mar

# Make predictions
curl -X POST http://localhost:8080/predictions/roberta_classifier \
  -H "Content-Type: application/json" \
  -d '{"text": "Your text to classify here"}'

3. Google Cloud Vertex AI

The model is designed for deployment on Google Cloud Vertex AI:

• Uses Vertex AI environment variables (AIP_MODEL_DIR, AIP_CHECKPOINT_DIR, AIP_TENSORBOARD_LOG_DIR)
• Supports distributed training
• Compatible with Vertex AI Prediction endpoints

Environment Requirements

Core Dependencies

• transformers
• torch
• datasets
• peft
• scikit-learn
• pandas
• google-cloud-storage

For TorchServe Deployment

• torch-model-archiver
• torchserve

Memory Optimizations

The training script includes several memory optimization techniques:

• Gradient Checkpointing: Reduces memory usage during backpropagation
• LoRA Fine-tuning: Significantly reduces trainable parameters
• Pin Memory Disabled: Prevents potential memory issues
• Dynamic Padding: Uses DataCollatorWithPadding for efficient batching

Model Integration

Input Processing

• Maximum Sequence Length: 512 tokens
• Tokenization: RoBERTa tokenizer with truncation and padding
• Text Preprocessing: Standard RoBERTa preprocessing (no special preprocessing required)

Output Format

The model outputs logits for binary classification:

• Raw logits: [human_score, ai_score]
• Probabilities: Softmax-normalized scores
• Prediction: Argmax of probabilities

License and Usage

This model is designed for research and production use in AI content detection. Please ensure compliance with your organization's AI ethics guidelines when deploying in production environments.

Support and Issues

For training-related issues, refer to the training script documentation in trainer_root/trainer/task.py. For deployment issues, consult the TorchServe or Vertex AI documentation based on your deployment method.