Downloads · 30 days
19
2% of all-time downloads
kangelamw/RoBERTa-political-bias-classifier-softmax
RoBERTa-political-bias-classifier-softmax is a machine learning model from kangelamw. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Downloads · 30 days
19
2% of all-time downloads
All-time downloads
773
Public
Parameters
125M
499 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors499 MB · 99%
From the Hugging Face model README
IN PROGRESS
A classifier model for analyzing news articles to reveal their political slant. Using machine learning, the system provides probabilistic scores across the political spectrum: liberal, center, and conservative.
[More Information Needed]
Not Suitable For:
Users should avoid using this model for political campaigning, propaganda, or any application that might promote bias or misinformation.
Data Bias:
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
Use the code below to get started with the model.
model_id = "kangelamw/RoBERTa-political-bias-classifier-softmax"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
# Example input text
text = "Your sample text goes here."
inputs = tokenizer(text, return_tensors="pt")
# Get predictions
outputs = model(**inputs)
print(outputs)
[More Information Needed]
Model Initialization:
Start from a pre-trained RoBERTa model that has general language understanding capabilities.
Fine-Tuning Approach:
The model was adapted to political bias classification by appending a softmax classification layer and training on the specialized dataset. Key training parameters such as learning rate, batch size, and the number of epochs were optimized during the fine-tuning process.
Validation and Optimization:
A portion of the dataset was set aside for validation to monitor performance and avoid overfitting.
training_args = TrainingArguments(
output_dir=model_path,
do_train=True,
do_eval=True,
do_predict=True,
eval_strategy="steps",
eval_steps=150,
eval_accumulation_steps=4,
logging_strategy="steps",
logging_steps=300,
save_strategy="steps",
save_steps=300,
num_train_epochs=5,
learning_rate=2e-5,
lr_scheduler_type="linear",
warmup_ratio=0.1,
weight_decay=0.01,
load_best_model_at_end=True,
metric_for_best_model="f1",
greater_is_better=True,
report_to="tensorboard",
resume_from_checkpoint=True,
per_device_eval_batch_size=8,
per_device_train_batch_size=8,
gradient_accumulation_steps=4,
gradient_checkpointing=True
)
| Metric | Value |
|---|---|
| eval_accuracy | 0.9204 |
| eval_f1 | 0.9206 |
| eval_cross_entropy | 0.2789 |
| eval_kl_divergence | 0.2789 |
| epoch | 4.9875 |
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
@article{DBLP:journals/corr/abs-1907-11692,
author = {Yinhan Liu and
Myle Ott and
Naman Goyal and
Jingfei Du and
Mandar Joshi and
Danqi Chen and
Omer Levy and
Mike Lewis and
Luke Zettlemoyer and
Veselin Stoyanov},
title = {RoBERTa: {A} Robustly Optimized {BERT} Pretraining Approach},
journal = {CoRR},
volume = {abs/1907.11692},
year = {2019},
url = {http://arxiv.org/abs/1907.11692},
archivePrefix = {arXiv},
eprint = {1907.11692},
timestamp = {Thu, 01 Aug 2019 08:59:33 +0200},
biburl = {https://dblp.org/rec/journals/corr/abs-1907-11692.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}