modernbert-base-llm-author-attribution

ModernBERT LLM Authorship Attribution

Fine-tuned ModernBERT-base that classifies the author of a short English Q&A answer into five classes: human, Claude (Haiku 4.5), GPT (GPT-5 mini), Gemini (2.5 Flash), Kimi (K2).

From Measuring LLM Self-Edit Resistance Through Writing Behavior (Terra Jiang, Matilda Orona & Rahil Sharma, UC Berkeley MIDS, 2026).

Try it in your browser: Authorship Attribution - inference runs fully client-side via Transformers.js.

Training data

Derived from the HC3 corpus (CC-BY-SA-4.0), extended with answers generated by the four LLMs above across 5,306 questions on finance, medicine, and general knowledge. Split at the question level to prevent leakage.

Evaluation

• PyTorch weights: 98.0% accuracy on a held-out test sample (n=150, chance = 20%). Paper reports per-class F1 0.98-0.99 on the full test split.
• Quantized ONNX (onnx/model_quantized.onnx, int8 per-channel, 151 MB): 98.7% on the same sample, 98.7% agreement with fp32.

Usage

from transformers import pipeline
clf = pipeline("text-classification", model="tildahh/modernbert-base-llm-author-attribution")
clf("your text here", top_k=None)

Or in JavaScript with Transformers.js, dtype: "q8".

Limitations

Research artifact, not a general-purpose AI detector. Trained on short Q&A answers and four specific 2025-era models. Performance beyond the evaluated domains and model versions has not been established. Confidence scores saturate near 100% and are not calibrated probabilities. A prediction does not establish who wrote a passage. Provided as-is, no warranties.