misinfo_roberta_fine_tune

Misinfo Fine-tuning Notebook

Fine-tune transformer models (DistilBERT, RoBERTa) for misinformation detection using LoRA.

Quick Start

Local

pip install -q transformers datasets accelerate torch tokenizers evaluate scikit-learn huggingface_hub peft
jupyter notebook misinfo_agent/notebooks/misinfo_finetune.ipynb

Colab

1. Upload notebook to Colab
2. Run cells 1-3 (install & mount Drive)
3. Upload CSV to Google Drive
4. Run remaining cells

---

Dataset Format

CSV with columns: title (optional), text, label (0=fake, 1=legit)

title,text,label
"News","Content...",1
"Claim","False...",0

---

Config (Cell 6)

config = {
    'model_name': 'distilbert-base-uncased',  # or roberta-base
    'max_samples': 500,
    'num_train_epochs': 1,
    'per_device_train_batch_size': 8,
    'output_dir': './misinfo_model',
}

Key Settings:

• max_samples: Reduce to 250 if out of memory
• per_device_train_batch_size: 4-8 (lower if OOM)
• model_name: distilbert (fast) or roberta (better accuracy)

---

Training Pipeline

---

Model Weights

Saved to ./misinfo_model/:

• adapter_model.bin (~5-10 MB)
• adapter_config.json
• config.json
• tokenizer.json

Performance:

• Training: 2-5 min (GPU) vs 15-30 min (CPU)
• Size: 5-10 MB (LoRA) vs 500+ MB (full)
• Accuracy: 90-95%

---

Upload to Hugging Face

from huggingface_hub import login, HfApi
login()
api = HfApi()
api.upload_folder(folder_path="./misinfo_model", repo_id="username/model-name", repo_type="model")

---

Use Trained Model

from peft import PeftModel
from transformers import AutoTokenizer, AutoModelForSequenceClassification

base = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased")
model = PeftModel.from_pretrained(base, "username/model-name")
tokenizer = AutoTokenizer.from_pretrained("username/model-name")

inputs = tokenizer("Your claim", return_tensors="pt")
outputs = model(**inputs)
print(outputs.logits.softmax(dim=-1))

---

Troubleshooting

---

Requirements

• GPU: 2+ GB VRAM
• RAM: 4+ GB
• Python 3.8+

Last Updated: November 17, 2025