distilroberta-base-climate-s

Model Card for distilroberta-base-climate-s

Model Description

This is the ClimateBERT language model based on the SIM-SELECT sample selection strategy.

Note: We generally recommend choosing the distilroberta-base-climate-f language model over this language model (unless you have good reasons not to).

Using the DistilRoBERTa model as starting point, the ClimateBERT Language Model is additionally pre-trained on a text corpus comprising climate-related research paper abstracts, corporate and general news and reports from companies. The underlying methodology can be found in our language model research paper.

Climate performance model card

Citation Information

@inproceedings{wkbl2022climatebert,
    title={{ClimateBERT: A Pretrained Language Model for Climate-Related Text}},
    author={Webersinke, Nicolas and Kraus, Mathias and Bingler, Julia and Leippold, Markus},
    booktitle={Proceedings of AAAI 2022 Fall Symposium: The Role of AI in Responding to Climate Challenges},
    year={2022},
    doi={https://doi.org/10.48550/arXiv.2212.13631},
}