Downloads · 30 days
24
0% of all-time downloads
SocBERT-base
sarkerlab/SocBERT-base
SocBERT-base is a fill-mask model from sarkerlab. Use it when you need the model to fill a missing word. It is set up for transformers.
Pretrained model on 20GB English tweets and 72GB Reddit comments using a masked language modeling (MLM) objective. The tweets are from Archive and collected from Twitter Streaming API. The Reddit comments are ramdonly…
All-time downloads
8.4K
Public
Repo size
2.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin572 MB · 99%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- Model type
- roberta
- Access
- Public
- Created
- Feb 9, 2023
- Updated
- Mar 21, 2023
- SHA
- e00ef164
Try a prompt
Spaces
Model card
From the Hugging Face model README
SocBERT model
Pretrained model on 20GB English tweets and 72GB Reddit comments using a masked language modeling (MLM) objective. The tweets are from Archive and collected from Twitter Streaming API. The Reddit comments are ramdonly sampled from all subreddits from 2015-2019. SocBERT-base was pretrained on 819M sequence blocks for 100K steps. SocBERT-final was pretrained on 929M (819M+110M) sequence blocks for 112K (100K+12K) steps. We benchmarked SocBERT, on 40 text classification tasks with social media data.
The experiment results can be found in our paper:
@inproceedings{socbert:2023,
title = {{SocBERT: A Pretrained Model for Social Media Text}},
author = {Yuting Guo and Abeed Sarker},
booktitle = {Proceedings of the Fourth Workshop on Insights from Negative Results in NLP},
year = {2023}
}
- Task
- Fill-Mask
- Library
- transformers
- Type
- roberta
- Created
- Feb 9, 2023
- Updated
- Mar 21, 2023