GIST-small-markov-slop-detector

GIST-small-markov-slop-detector

This BERT-based classifier is trained to distinguish coherent human-written text from text generated by a Markov chain.

As expected, the classifier achieves near-perfect performance (98.2% accuracy on evaluation set), largely because BERT’s attention mechanism captures long-range contextual dependencies, whereas a Markov model relies only on the previous state.

Dataset

Class distribution of the training dataset:

• Real samples from: agentlans/high-quality-text-long sample_k10000
• Markov samples from: agentlans/markov-slop

Model Specification

• Model type: bert
• Problem Type: single_label_classification
• Number of Labels: 2
• Vocabulary Size: 30522
• License: MIT

Use

To get started with this model in Python using the Hugging Face Transformers library, run the following code:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "agentlans/GIST-small-markov-slop-detector"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Replace this with your input text."
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
predicted_class_name = model.config.id2label[predicted_class_id]

print(f"Predicted Class ID: {predicted_class_id}")
print(f"Predicted Class Name: {predicted_class_name}")

Intended Uses & Limitations

Intended Use

This model is designed for sequence classification tasks. Below are the specific class labels mapped to their corresponding IDs:

Training Details

Hyperparameters

The following hyperparameters were used during fine-tuning:

• Learning Rate: 5e-05
• Train Batch Size: 8
• Eval Batch Size: 8
• Optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• Number of Epochs: 3.0
• Mixed Precision: BF16

<details> <summary><b>Show Advanced Training Configuration</b></summary>

Optimization & Regularization

• Gradient Accumulation Steps: 1
• Learning Rate Scheduler: SchedulerType.LINEAR
• Warmup Steps: 0
• Warmup Ratio: None
• Weight Decay: 0.0
• Max Gradient Norm: 1.0

Hardware & Reproducibility

• Number of GPUs: 1
• Seed: 42

</details>

Training Results & Evaluation

During fine-tuning, the model achieved the following results on the evaluation set:

Speed Performance

• Training Runtime: 106.2373 seconds
• Train Samples per Second: 451.762
• Evaluation Runtime: 3.2116 seconds
• Eval Samples per Second: 1245.467

<details> <summary><b>Show Detailed Training Logs</b></summary>

Training Logs History

</details>

Framework Versions

• Transformers: 5.0.0.dev0
• PyTorch: 2.9.1+cu128