Downloads · 30 days
101
8% of all-time downloads
convbert-small-spanish
mrm8488/convbert-small-spanish
convbert-small-spanish is a feature extraction model from mrm8488. Use it when you need embeddings to search or compare text. It is set up for transformers. The card lists the license as mit.
The ConvBERT architecture is presented in the "ConvBERT: Improving BERT with Span-based Dynamic Convolution" paper.
All-time downloads
1.3K
Public
Parameters
13.1M
211 MB on disk
Likes
3
Public
Repo makeup
Click a slice to open those files.
.h552.9 MB · 33%
Parameter types
How the weights are stored.
F3213.1M · 100%
Datasets
Model card
From the Hugging Face model README
ConvBERT small pre-trained on large_spanish_corpus
The ConvBERT architecture is presented in the "ConvBERT: Improving BERT with Span-based Dynamic Convolution" paper.
Metrics on evaluation set
disc_accuracy = 0.95163906
disc_auc = 0.9405496
disc_loss = 0.13658184
disc_precision = 0.80829453
disc_recall = 0.49316448
global_step = 1000000
loss = 9.12079
masked_lm_accuracy = 0.53505784
masked_lm_loss = 2.3028736
sampled_masked_lm_accuracy = 0.44047198
Usage
from transformers import AutoModel, AutoTokenizer
model_name = "mrm8488/convbert-small-spanish"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)
Created by Manuel Romero/@mrm8488 with the support of Narrativa
Made with <span style="color: #e25555;">♥</span> in Spain
- Task
- Feature Extraction
- Library
- transformers
- Type
- convbert
- License
- mit
- Languages
- es
- Created
- Mar 2, 2022
- Updated
- Mar 22, 2023