Downloads · 30 days
8
31% of all-time downloads
BERToli
mattiaferrarini/BERToli
BERToli is a fill-mask model from mattiaferrarini. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as mit.
BERToli is a BERT model for Italian song lyrics. It was obtained via continued pretraining of dbmdz/bert-base-italian-xxl-cased on ~106k Italian song lyrics from the Genius Song Lyrics Dataset.
All-time downloads
26
Public
Parameters
111M
443 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors443 MB · 100%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- mit
- Model type
- bert
- Access
- Public
- Created
- Aug 26, 2025
- Updated
- Sep 16, 2025
- SHA
- b9debc7c
Try a prompt
Base models
Model card
From the Hugging Face model README
BERToli 🎶🇮🇹
About the model
BERToli is a BERT model for Italian song lyrics. It was obtained via continued pretraining of dbmdz/bert-base-italian-xxl-cased on ~106k Italian song lyrics from the Genius Song Lyrics Dataset.
The objective was Masked Language Modeling (MLM).
The training code is available on GitHub.
Evaluation
The base model and the adapted model were tested on a held-out set of ~6k songs with the following results:
| Model | MLM Loss | Perplexity |
|---|---|---|
| Base | 1.94 | 6.95 |
| BERToli | 1.45 | 4.26 |
Evaluation of the learned representations will be made available in the future, once a suitable dataset has been created / identified.
Why BERToli?
Pierangelo Bertoli (5 November 1942 – 7 October 2002) was an Italian singer-songwriter and poet.
- Task
- Fill-Mask
- Library
- transformers
- Type
- bert
- License
- mit
- Languages
- it
- Created
- Aug 26, 2025
- Updated
- Sep 16, 2025