Downloads · 30 days
0
vinukatashidu/evidence-aware-fake-news
evidence-aware-fake-news is a text classification model from vinukatashidu. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as mit.
A specialized deep learning pipeline designed to detect fake news by incorporating contextual metadata and external evidence into the textual input. This model was fine-tuned on the LIAR dataset.
Downloads · 30 days
0
Access
Public
Updated Apr 16, 2026
Repo size
1.1 GB
Likes
0
Public
Click a slice to open those files.
.pt1.1 GB · 100%
From the Hugging Face model README
A specialized deep learning pipeline designed to detect fake news by incorporating contextual metadata and external evidence into the textual input. This model was fine-tuned on the LIAR dataset.
Standard approaches to the LIAR dataset often treat fake news detection as a pure sentence-classification task, isolating the claim from its origin. However, the veracity of a statement is heavily dependent on context and source.
To address this, we developed a metadata enrichment strategy:
speaker, party_affiliation, and context.Context: {context}. Speaker: {speaker} ({party}). Statement: {claim}justification variable up to the maximum sequence limit: ... Statement: {claim} [SEP] Evidence: {justification}This approach allows the Transformer’s self-attention layers to map the latent relationships between a speaker's historical credibility and the claim being analyzed, mirroring human fact-checking heuristics.
To maximize performance within strict hardware constraints, we upgraded the backbone architecture from standard BERT to microsoft/deberta-v3-small. DeBERTa-v3 uses a disentangled attention mechanism, allowing it to significantly outperform standard BERT on Nuanced Language Understanding (NLU) tasks, while keeping the parameter count low enough to fit within an RTX 3050 (4GB VRAM) running at a massive 512-token sequence length.
External Evidence Significantly Improves F1-Score and Fake News Detection Capability.
| Metric | Model A (Claim-Only | Model B (Claim+Evidence) | Delta (Change) |
|---|---|---|---|
| Accuracy | 60.41% | 60.33% | -0.08% |
| Precision | 60.90% | 59.46% | -1.45% |
| Recall | 60.41% | 60.33% | -0.08% |
| F1 Score | 0.5464 | 0.5875 | +0.0411 |
While both datasets achieved roughly 60% accuracy, looking at Accuracy alone is highly deceptive due to class imbalance.
When we analyze the F1-score—which balances precision and recall—Model B outperformed Model A by a significant +0.04 margin. Furthermore, adding evidence bumped the 'FAKE' class recall from a dangerous 21% up to 38%. This proves the core hypothesis: while a model can memorize metadata to achieve baseline accuracy, true contextual understanding and reliable classification of Fake News absolutely requires external empirical evidence.
models/: Contains the large .pt weights for both Model A and Model B.notebooks/: Contains the Jupyter Notebook showing the end-to-end execution.results/: Contains the generated training curve comparisons, Confusion Matrices, and Radar charts.src/: Contains the modularized PyTorch pipeline design.