roberta-large-cv-detector

RoBERTa-Large CV AI Detection Model

Model Overview

This model is a fine-tuned RoBERTa-Large transformer trained to detect whether a curriculum vitae (CV) was written by a human, generated by AI, or produced through a combination of both.

The model was trained using the dataset:

jamal-ibrahim/ai-cv-detection-dataset

The task is formulated as a three-class classification problem:

Label Meaning

---

human CV written entirely by a human mixed CV containing both human-written and AI-generated content ai_generated CV generated primarily by an AI system

The goal of this experiment was to explore whether modern transformer models can identify stylistic differences between human-written and AI-generated professional documents.

---

Experimental Setup

Model Architecture

Base model:

roberta-large

RoBERTa-Large contains approximately 355 million parameters and is known to perform well on tasks involving linguistic style, authorship detection, and semantic classification.

Dataset

Dataset used for training:

jamal-ibrahim/ai-cv-detection-dataset

Total dataset size: approximately 1500 CV documents.

The dataset was split using a stratified strategy to maintain class balance.

Split configuration:

Split Percentage

---

Train 70% Validation 15% Test 15%

Each class (human, mixed, ai_generated) is represented equally in each split.

---

Training Configuration

Training parameters:

Parameter Value

---

Epochs 10 Batch size 16 Learning rate 5e-6 Optimizer AdamW Max sequence length 512 Evaluation strategy per epoch

Training was performed using the Hugging Face Trainer API.

---

Results

Evaluation was performed on the held-out test set.

Confusion Matrix

True / Predicted Human Mixed AI

---

Human 75 0 0 Mixed 0 74 1 AI Generated 0 0 75

Only one misclassification occurred in the test set.

Metrics

Metric Value

---

Accuracy ~0.995 Weighted F1 ~0.996 Macro F1 ~0.995

The model demonstrates near-perfect classification performance on this dataset.

---

Interpretation of Results

The extremely high classification accuracy suggests that the model successfully learned stylistic patterns that differentiate:

• human-authored CVs
• AI-generated CVs
• hybrid documents

Several factors may contribute to this performance.

Structural Patterns in AI-Generated CVs

Large language models often produce documents with consistent structural patterns such as:

• uniform sentence lengths
• repeated phrasing patterns
• symmetrical bullet point structures
• consistent grammatical style

These patterns may create detectable stylistic signatures.

Human Writing Variability

Human-written CVs often contain:

• irregular formatting
• inconsistent sentence structures
• varied vocabulary
• domain-specific phrasing

This variability may make human documents distinguishable from AI-generated ones.

Mixed Documents as a Transitional Style

Documents labeled as mixed appear to occupy an intermediate stylistic space between human and AI-generated text.

The single observed misclassification (mixed → AI-generated) suggests that documents heavily edited by AI may resemble fully generated content.

---

Limitations

Although the results are strong, several limitations should be considered.

Dataset Size

The dataset contains approximately 1500 samples, which is relatively small for training large transformer models.

High performance may partly reflect the limited diversity of the dataset.

Potential Template Effects

If AI-generated CVs were produced using similar prompts or generation patterns, the model may learn those specific templates rather than generalizable AI-writing signals.

Domain Specificity

This model was trained specifically on CV-style documents and may not generalize well to other types of text such as essays, emails, or reports.

---

Future Work

Future experiments could explore several directions:

• cross-validation to measure performance stability
• testing on unseen CV datasets
• evaluating different LLM-generated CV styles
• reducing model size using distillation or quantization
• hybrid approaches combining stylistic features with transformer embeddings

---

Intended Use

Potential applications include:

• research on AI-generated text detection
• analysis of AI assistance in professional writing
• experimentation with authorship attribution in structured documents

The model should not be used as a definitive tool for determining authorship in real-world hiring decisions.

---

Citation

@model{roberta_large_cv_detector, title = {RoBERTa-Large CV AI Detection Model}, author = {Jamal Ibrahim}, year = {2026}, dataset = {jamal-ibrahim/ai-cv-detection-dataset}, publisher = {Hugging Face} }