distilbert-misinfo-lora

DistilBERT — Text-Only Misinformation Detection on FakeTT

Authors: Andrei-Gabriel Radu, Ciprian-Octavian Truică, Elena-Simona Apostol

National University of Science and Technology POLITEHNICA Bucharest

LoRA adapter fine-tuned from distilbert-base-uncased for binary text-only misinformation classification on the FakeTT social-media video dataset.

This model accompanies the bachelor thesis Misinformation Detection in Social Media Videos.

Results

Model

• Base model: distilbert-base-uncased
• Task: Binary misinformation classification
• Modality: Text-only
• Fine-tuning: LoRA / PEFT
• Dataset: FakeTT
• Number of classes: 2
• Primary metric: Macro-F1

Training

• LoRA rank (r): 8
• LoRA alpha: 16
• LoRA dropout: 0.05
• Target modules: k_lin, out_lin, q_lin, v_lin
• Bias: none

Usage

import torch
from peft import PeftModel
from transformers import AutoTokenizer, AutoModelForSequenceClassification

repo_id = "DS4AI-UPB/distilbert-misinfo-lora"
base_model_id = "distilbert-base-uncased"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
base_model = AutoModelForSequenceClassification.from_pretrained(base_model_id, num_labels=2)
model = PeftModel.from_pretrained(base_model, repo_id).eval()

text = "Example social media video description."
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)

with torch.no_grad():
    logits = model(**inputs).logits

print(logits.argmax(dim=-1).item())

Use the class-to-label mapping from the original FakeTT training pipeline.

Intended Use

Research and benchmarking of English-language text-only misinformation detection for social-media video content.

Limitations

This is a classification model, not a factual verification system. It cannot inspect the associated video and can degrade under domain shift.

Citation

@thesis{radu2026misinformation,
    author = {Radu, Andrei-Gabriel and Truică, Ciprian-Octavian and Apostol, Elena-Simona},
    title  = {Misinformation Detection in Social Media Videos},
    school = {National University of Science and Technology POLITEHNICA Bucharest},
    year   = {2026}
}