Downloads · 30 days
43
1% of all-time downloads
kornosk/bert-political-election2020-twitter-mlm
bert-political-election2020-twitter-mlm is a fill-mask model from kornosk. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as gpl-3.0.
Pre-trained weights for Knowledge Enhance Masked Language Model for Stance Detection, NAACL 2021.
Downloads · 30 days
43
1% of all-time downloads
All-time downloads
7.6K
Public
Repo size
1.3 GB
Likes
4
Public
Click a slice to open those files.
.bin441 MB · 50%
From the Hugging Face model README
Pre-trained weights for Knowledge Enhance Masked Language Model for Stance Detection, NAACL 2021.
We use the initialized weights from BERT-base (uncased) or bert-base-uncased.
This model is pre-trained on over 5 million English tweets about the 2020 US Presidential Election.
This model is initialized with BERT-base and trained with normal MLM objective.
This pre-trained language model can be fine-tunned to any downstream task (e.g. classification).
Please see the official repository for more detail.
from transformers import BertTokenizer, BertForMaskedLM, pipeline
import torch
# Choose GPU if available
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
# Select mode path here
pretrained_LM_path = "kornosk/bert-political-election2020-twitter-mlm"
# Load model
tokenizer = BertTokenizer.from_pretrained(pretrained_LM_path)
model = BertForMaskedLM.from_pretrained(pretrained_LM_path)
# Fill mask
example = "Trump is the [MASK] of USA"
fill_mask = pipeline('fill-mask', model=model, tokenizer=tokenizer)
# Use following line instead of the above one does not work.
# Huggingface have been updated, newer version accepts a string of model name instead.
fill_mask = pipeline('fill-mask', model=pretrained_LM_path, tokenizer=tokenizer)
outputs = fill_mask(example)
print(outputs)
# See embeddings
inputs = tokenizer(example, return_tensors="pt")
outputs = model(**inputs)
print(outputs)
# OR you can use this model to train on your downstream task!
# Please consider citing our paper if you feel this is useful :)
@inproceedings{kawintiranon2021knowledge,
title={Knowledge Enhanced Masked Language Model for Stance Detection},
author={Kawintiranon, Kornraphop and Singh, Lisa},
booktitle={Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
year={2021},
publisher={Association for Computational Linguistics},
url={https://www.aclweb.org/anthology/2021.naacl-main.376}
}