Downloads · 30 days
6
2% of all-time downloads
lmvasque/readability-es-benchmark-bertin-es-paragraphs-2class
readability-es-benchmark-bertin-es-paragraphs-2class is a text classification model from lmvasque. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as cc-by-4.0.
This project is part of a series of models from the paper "A Benchmark for Neural Readability Assessment of Texts in Spanish". You can find more details about the project in our GitHub.
Downloads · 30 days
6
2% of all-time downloads
All-time downloads
395
Public
Repo size
997 MB
Likes
0
Public
Click a slice to open those files.
.bin499 MB · 99%
From the Hugging Face model README
This project is part of a series of models from the paper "A Benchmark for Neural Readability Assessment of Texts in Spanish". You can find more details about the project in our GitHub.
Our models were fine-tuned in multiple settings, including readability assessment in 2-class (simple/complex) and 3-class (basic/intermediate/advanced) for sentences and paragraph datasets. You can find more details in our paper. These are the available models you can use (current model page in bold):
| Model | Granularity | # classes |
|---|---|---|
| BERTIN (ES) | paragraphs | 2 |
| BERTIN (ES) | paragraphs | 3 |
| mBERT (ES) | paragraphs | 2 |
| mBERT (ES) | paragraphs | 3 |
| mBERT (EN+ES) | paragraphs | 3 |
| BERTIN (ES) | sentences | 2 |
| BERTIN (ES) | sentences | 3 |
| mBERT (ES) | sentences | 2 |
| mBERT (ES) | sentences | 3 |
| mBERT (EN+ES) | sentences | 3 |
For the zero-shot setting, we used the original models BERTIN and mBERT with no further training.
These are our results for all the readability models in different settings. Please select your model based on the desired performance:
| Granularity | Model | F1 Score (2-class) | Precision (2-class) | Recall (2-class) | F1 Score (3-class) | Precision (3-class) | Recall (3-class) |
|---|---|---|---|---|---|---|---|
| Paragraph | Baseline (TF-IDF+LR) | 0.829 | 0.832 | 0.827 | 0.556 | 0.563 | 0.550 |
| Paragraph | BERTIN (Zero) | 0.308 | 0.222 | 0.500 | 0.227 | 0.284 | 0.338 |
| Paragraph | BERTIN (ES) | 0.924 | 0.923 | 0.925 | 0.772 | 0.776 | 0.768 |
| Paragraph | mBERT (Zero) | 0.308 | 0.222 | 0.500 | 0.253 | 0.312 | 0.368 |
| Paragraph | mBERT (EN) | - | - | - | 0.505 | 0.560 | 0.552 |
| Paragraph | mBERT (ES) | 0.933 | 0.932 | 0.936 | 0.776 | 0.777 | 0.778 |
| Paragraph | mBERT (EN+ES) | - | - | - | 0.779 | 0.783 | 0.779 |
| Sentence | Baseline (TF-IDF+LR) | 0.811 | 0.814 | 0.808 | 0.525 | 0.531 | 0.521 |
| Sentence | BERTIN (Zero) | 0.367 | 0.290 | 0.500 | 0.188 | 0.232 | 0.335 |
| Sentence | BERTIN (ES) | 0.900 | 0.900 | 0.900 | 0.699 | 0.701 | 0.698 |
| Sentence | mBERT (Zero) | 0.367 | 0.290 | 0.500 | 0.278 | 0.329 | 0.351 |
| Sentence | mBERT (EN) | - | - | - | 0.521 | 0.565 | 0.539 |
| Sentence | mBERT (ES) | 0.893 | 0.891 | 0.896 | 0.688 | 0.686 | 0.691 |
| Sentence | mBERT (EN+ES) | - | - | - | 0.679 | 0.676 | 0.682 |
If you use our results and scripts in your research, please cite our work: "A Benchmark for Neural Readability Assessment of Texts in Spanish" (to be published)
@inproceedings{vasquez-rodriguez-etal-2022-benchmarking,
title = "A Benchmark for Neural Readability Assessment of Texts in Spanish",
author = "V{\'a}squez-Rodr{\'\i}guez, Laura and
Cuenca-Jim{\'\e}nez, Pedro-Manuel and
Morales-Esquivel, Sergio Esteban and
Alva-Manchego, Fernando",
booktitle = "Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022), EMNLP 2022",
month = dec,
year = "2022",
}