longformer-framing-gold

Media Framing Detector with Longformer

A multi-label transformer model for detecting media frames in news articles and political text. This model identifies up to 15 distinct framing strategies used in communication, based on the Media Frames Corpus taxonomy (Card et al., 2015).

Model Description

This Longformer-based classifier detects media frames in text up to ~1,600 words, addressing key limitations in existing approaches:

• No hallucination: Unlike GenAI prompt-engineering approaches, this discriminative model cannot hallucinate non-existent frames
• Long context: Handles full articles (2048 tokens) using sparse attention, unlike sentence-level classifiers
• Human-aligned: Fine-tuned on gold-standard human annotations from 19 trained annotators, combining best available data for the task

Performance: Weighted F1 of 0.686 on held-out test set, outperforming GenAI approaches while being 47x smaller than mm-framing's Mistral-7B. Trained on 385,000 total articles across political spectrum from 24 independent news providers.

Read the full write-up: https://ry-rousseau.github.io/2026/02/09/longformer-framing-classifier.html

Intended Uses

• Automated detection of argumentation strategies in news media
• Analysis of political communication and polarization
• Social media content analysis (posts, threads, articles)
• Bias detection

How to Use

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model and tokenizer
model_name = "ry-rousseau/media-framing-longformer"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Prepare input with topic injection (highly recommended)
topic = "immigration" # or use topic classifier
text = "Your article text here..."
input_text = f"TOPIC:{topic}\n{text}"

# Tokenize and predict
inputs = tokenizer(input_text, return_tensors="pt", truncation=True, max_length=2048)
outputs = model(**inputs)

# Get predictions (apply optimized thresholds per class)
probs = torch.sigmoid(outputs.logits)
predictions = (probs > 0.5).int() # Use class-specific thresholds for better performance

# Frame labels
frames = [
"Economic", "Capacity and Resources", "Morality", "Fairness and Equality",
"Legality, Constitutionality and Jurisprudence", "Policy Prescription and Evaluation",
"Crime and Punishment", "Security and Defense", "Health and Safety",
"Quality of Life", "Cultural Identity", "Public Opinion", "Political",
"External Regulation and Reputation", "Other"
]

detected_frames = [frames[i] for i, pred in enumerate(predictions[0]) if pred == 1]
print("Detected frames:", detected_frames)

The topic classifier is RoBERTa with a head-tail truncation strategy for long articles, trained on 64,000 examples:

• Topic Classifier: Use this Model

Training Data

Gold Standard (Human Annotations)

• Media Frames Corpus (MFC): 2,224 NYT articles on immigration, same-sex marriage, and smoking (1990-2012)
• SemEval 2023 Task 3: 516 articles on Ukraine-Russia war, COVID-19, migration, abortion, climate change (2020-2022)
• Total: 2,740 articles with annotations from 19 trained human annotators
• Annotation strategy: Union of annotations (mean 4.14 frames per article)

Silver Training Data

• Source: copenlu/mm-framing dataset
• Size: ~380,000 articles labeled by Mistral-7B
• Purpose: Pre-training to learn frame patterns before gold fine-tuning

Training Procedure

Phase 1: Silver Training

Trained on machine-labeled data to learn broad frame patterns:

• Model: allenai/longformer-base-4096
• Loss: Binary cross-entropy with class weights
• Epochs: 4
• Batch size: 16 (effective 32 with gradient accumulation)
• Learning rate: 2e-5
• Duration: 72 hours on NVIDIA A40 (48GB VRAM)

Phase 2: Gold Fine-Tuning

Fine-tuned on human annotations to align with expert reasoning:

• Loss: Focal loss (gamma=2.0) to focus on difficult examples
• Epochs: 10
• Batch size: 2 (effective 16 with gradient accumulation)
• Learning rate: 2e-5 with scheduler
• Validation: 90/10 split

Key Features

• Topic injection: Prepending article topic improves Micro F1 by +2.7%
• Global attention: Applied to [CLS] token and topic token for mixture-of-experts effect
• Threshold optimization: Post-training per-class threshold tuning for class imbalance

Evaluation Results

Overall Performance (Test Set)

Per-Frame Performance

Note: "Other" frame excluded from evaluation metrics, as per SemEval method.

More Info

• GitHub Repository: https://github.com/Ry-Rousseau/frame-delta
• Project Write-Up: https://ry-rousseau.github.io/2026/02/09/longformer-framing-classifier.html

Model will be iterated in future. It should run on most mid-range GPUs locally with gradient accumulation.

@misc{rousseau2026framing, title={15-Class Media Framing Detector with Longformer}, author={Rousseau, Ryan}, year={2026}, howpublished={\url{https://huggingface.co/ry-rousseau/media-framing-longformer}} }