Downloads · 30 days
6
13% of all-time downloads
abhishekkshah12987/fakeddit-bert-fake-news
fakeddit-bert-fake-news is a text classification model from abhishekkshah12987. Use it when you need a label for a piece of text. The card lists the license as mit.
Fine-tuned on a balanced subsample of Fakeddit for binary (real/fake) headline classification, with validation-based checkpoint selection and post-hoc temperature-scaled calibration.
Downloads · 30 days
6
13% of all-time downloads
All-time downloads
47
Public
Parameters
67M
268 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors268 MB · 100%
From the Hugging Face model README
Fine-tuned on a balanced subsample of Fakeddit for binary (real/fake) headline classification, with validation-based checkpoint selection and post-hoc temperature-scaled calibration.
This model classifies headline style, not claim veracity. It is not a general-purpose fact-checker. See Limitations below.
| Metric | Value |
|---|---|
| Accuracy | 0.7980 |
| F1 (weighted) | 0.7975 |
| Precision (weighted) | 0.8013 |
| Recall (weighted) | 0.7980 |
| ECE (calibrated) | 0.0656 |
Multi-seed comparison (mean +/- std across 3 seeds):
| Model | Accuracy | F1 |
|---|---|---|
| bert-base-uncased | 0.7983 +/- 0.0078 | 0.7982 +/- 0.0079 |
| distilbert-base-uncased | 0.7897 +/- 0.0125 | 0.7885 +/- 0.0134 |
from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch, torch.nn.functional as F
model = AutoModelForSequenceClassification.from_pretrained("abhinav-29/fakeddit-bert-fake-news")
tokenizer = AutoTokenizer.from_pretrained("abhinav-29/fakeddit-bert-fake-news")
text = "Local city council approves new budget for public libraries"
enc = tokenizer(text, max_length=64, padding="max_length", truncation=True, return_tensors="pt")
with torch.no_grad():
logits = model(**enc).logits
probs = F.softmax(logits / 1.1099, dim=-1) # calibrated confidence
print(probs)
Nakamura, K., Levy, S., & Wang, W. Y. (2020). r/Fakeddit: A New Multimodal Benchmark Dataset for Fine-grained Fake News Detection. Proceedings of LREC 2020.