model-to-production-2.1

model-to-production-2.1

Fine-tuned text classification model based on google/gemma-3-1b-it for detecting AI-generated vs human-written text across 85+ languages. Trained using LoRA and merged into a standalone model for simple deployment.

Model Description

• Model type: Binary Sequence Classification (AI vs Human Text Detection)
• Base model: google/gemma-3-1b-it
• Training method: LoRA fine-tuning (r=64, α=128), then merged
• Number of labels: 2 (Binary: 0=Human, 1=AI)
• Languages: 85+ languages from FineWeb2 dataset
• License: Apache 2.0

Key Improvements over v2.0

• ✨ Multilingual: Trained on 85+ languages (vs Swedish-only in v2.0)
• 🎯 High Performance: 98.73% accuracy and F1 score
• 🔨 Robust: Character-level augmentation (noise, punctuation, case variations)
• 📦 Reproducible: Full training code and data included
• ⚡ Optimized: Better LoRA configuration (r=64, α=128)

Label Mapping

Performance

Validation Set Results (2,977 examples, 85 languages):

• Accuracy: 98.73%
• F1 Score: 98.73% (macro)
• Eval Loss: 0.046

Achieved after 41,600 steps (1 epoch) with early stopping.

Languages Supported

The model was trained on 85 languages from the FineWeb2 dataset, covering diverse scripts:

By Script Family

• Latin (40+ langs): English, German, French, Spanish, Italian, Portuguese, Polish, Dutch, Swedish, Finnish, Norwegian, Danish, Czech, Romanian, Croatian, Hungarian, etc.
• Cyrillic (10+ langs): Russian, Ukrainian, Bulgarian, Serbian, Belarusian, Macedonian, Kazakh, Kyrgyz, Tajik, Uzbek
• Han/CJK (3 langs): Mandarin Chinese, Japanese, Korean
• Indic (15+ langs): Hindi, Bengali, Tamil, Telugu, Gujarati, Kannada, Malayalam, Marathi, Punjabi, Oriya, Assamese, Nepali, Sinhala
• Arabic script (5+ langs): Standard Arabic, Persian, Urdu, Pashto, Kurdish
• Other: Greek, Hebrew, Georgian, Armenian, Thai, Khmer, Lao, Myanmar, Amharic, etc.

See datasets/language_distribution.md for the complete list of ISO 639-3 language codes.

Quick Start

Installation

pip install transformers torch

Basic Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model and tokenizer
model_name = "Mohamad-Jaallouk/model-to-production-2.1"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Example text (any of 85+ supported languages)
text = "This is a sample text to classify."

# Tokenize and predict
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)

predicted_class = predictions.argmax().item()
confidence = predictions[0][predicted_class].item()

print(f"Prediction: {'AI' if predicted_class == 1 else 'Human'}")
print(f"Confidence: {confidence:.2%}")

Using Pipeline (Recommended)

from transformers import pipeline

# Create classification pipeline
classifier = pipeline(
    "text-classification",
    model="Mohamad-Jaallouk/model-to-production-2.1",
    device=0  # Use GPU if available, -1 for CPU
)

# Classify text
result = classifier("Your text here", truncation=True, max_length=512)
print(result)
# Output: [{'label': 'LABEL_0', 'score': 0.9873}]
# LABEL_0 = Human, LABEL_1 = AI

Batch Processing

texts = [
    "First sample text in English.",
    "Deuxième exemple de texte en français.",
    "Tercer ejemplo de texto en español.",
]

results = classifier(texts, truncation=True, max_length=512, batch_size=8)
for text, result in zip(texts, results):
    label = "Human" if result['label'] == 'LABEL_0' else "AI"
    print(f"{label} ({result['score']:.2%}): {text[:50]}...")

Training Data

Dataset Composition

• Total Training Samples: 93,372
• Human Text (Label 0): 32,440 (50%)
• AI Text (Label 1): 32,436 (50%)
• Validation Samples: 2,977
• Languages: 85

Data Sources

Human Text:

• Source: FineWeb2 by HuggingFace
• Date Range: Pre-2022 web content (before LLM era)
• Sampling: Random samples across 85 languages
• License: ODC-By 1.0

AI Text:

• Models: GPT-4o-mini and GPT-5-nano (OpenAI)
• Task: Document analysis and generation
• Format: Mixed natural and structured text

Data Augmentation

Training data was augmented with character-level transformations for robustness:

• Random punctuation removal (0.5% rate)
• Character replacement (0.5% rate)
• Random character injection (0.5% rate)
• Extra space injection (0.5% rate)
• Adjacent character swaps (0.5% rate)
• Edge cutting (1-10 chars from start/end)
• Case normalization (lowercase)

See transform.py for implementation details.

Training Procedure

Hyperparameters

• Epochs: 1
• Learning Rate: 5e-5
• Batch Size: 16 (per device)
• Gradient Accumulation: 1 step
• Max Sequence Length: 512 tokens
• Warmup Ratio: 0.1
• Weight Decay: 0.0
• Optimizer: AdamW (fused)
• LR Schedule: Cosine with warmup
• Mixed Precision: bfloat16
• Flash Attention: Enabled

LoRA Configuration

• Rank (r): 64
• Alpha: 128
• Dropout: 0.1
• Target Modules: q_proj, k_proj, v_proj, o_proj, up_proj, down_proj, gate_proj
• Modules Saved: score (classification head)

Training Hardware

• GPU: NVIDIA GPU with bfloat16 support
• Training Time: ~X hours
• Peak Memory: ~X GB VRAM

Training Command

python train.py \
  --model-id google/gemma-3-1b-it \
  --data train.csv \
  --val-data val_small.csv \
  --output-dir gemma-3-1b-it_lora_cls_pt \
  --max-length 512 \
  --epochs 1 \
  --lr 5e-5 \
  --batch-size 16 \
  --eval-batch-size 32 \
  --grad-accum-steps 1 \
  --warmup-ratio 0.1 \
  --eval-every-steps 400 \
  --lora-r 64 \
  --lora-alpha 128 \
  --lora-dropout 0.1 \
  --pad-side left \
  --hf-token $HF_TOKEN

Reproducing Training

All training code and data are included in this repository for full reproducibility.

Setup

# Clone repository
git clone https://huggingface.co/Mohamad-Jaallouk/model-to-production-2.1
cd model-to-production-2.1

# Install dependencies
pip install -r requirements.txt

# Set HuggingFace token (for gated models)
export HF_TOKEN=your_hf_token_here

Train from Scratch

python train.py \
  --model-id google/gemma-3-1b-it \
  --data datasets/train.csv \
  --val-data datasets/val.csv \
  --output-dir my_model \
  --hf-token $HF_TOKEN

Custom Training

# See train.py for all available arguments
python train.py --help

Evaluation

Confusion Matrix (Validation Set)

          Predicted
          Human   AI
Actual
Human    [...]   [...]
AI       [...]   [...]

Per-Language Performance

The model maintains high performance across all 85 languages. See evaluation/per_language_metrics.json for detailed breakdown.

Limitations

• AI Text Format: AI training data includes structured JSON analysis, not just natural prose
• Temporal: Trained on pre-2022 human text; may not generalize to future text styles
• Domain: Web text only (not books, academic papers, code, etc.)
• Detection Evasion: Adversarial or paraphrased AI text may fool the detector
• New Models: AI text generated by newer/different models may not be detected accurately
• Context Length: Limited to 512 tokens; longer texts should be chunked

Intended Use

✅ Recommended:

• Detecting AI-generated text across 85+ languages
• Content moderation and authenticity verification
• Educational tools for AI literacy
• Research on AI text detection
• Multilingual text classification pipelines

❌ Not Recommended:

• Making high-stakes decisions solely based on model predictions
• Detection of adversarially-modified AI text
• Languages outside the 85 training languages
• Texts longer than 512 tokens without chunking
• Real-time streaming applications (use quantized version)

Bias and Ethical Considerations

• The model may perform differently across languages due to data imbalance
• AI text in training data comes from specific models (GPT-4o-mini, GPT-5-nano)
• Model may incorrectly flag human text from underrepresented languages
• Should not be used as sole evidence of academic misconduct
• Consider ensemble with other detection methods for critical applications

Citation

@model{model-to-production-2.1,
  title={model-to-production-2.1: Multilingual AI Text Detection},
  author={Mohamad Jaallouk},
  year={2025},
  publisher={HuggingFace},
  url={https://huggingface.co/Mohamad-Jaallouk/model-to-production-2.1},
  base_model={google/gemma-3-1b-it}
}

If you use the training data, please also cite FineWeb2:

@dataset{fineweb2,
  title={FineWeb2: A sparkling update with 1000s of languages},
  author={HuggingFace},
  year={2024},
  publisher={HuggingFace},
  url={https://huggingface.co/datasets/HuggingFaceFW/fineweb-2}
}

Model Card Contact

• Developed by: Mohamad Jaallouk
• Repository: https://huggingface.co/Mohamad-Jaallouk/model-to-production-2.1
• Issues: Please report issues in the repository

License

This model is released under the Apache 2.0 License, consistent with the base Gemma-3 model license.

The training data includes:

• FineWeb2 content (ODC-By 1.0 License)
• OpenAI-generated content (subject to OpenAI Terms of Use)

Acknowledgements

• Base Model: Google's Gemma-3-1b-it team
• Training Data: HuggingFace FineWeb2 dataset
• AI Generation: OpenAI GPT models
• Framework: HuggingFace Transformers, PEFT (LoRA), PyTorch

---

Version: 2.1 Last Updated: 2025-11-10 Framework Versions: transformers 4.57.0, PEFT 0.17.1, PyTorch 2.x