Downloads · 30 days
6
1% of all-time downloads
roberta-base-wikimovies
thatdramebaazguy/roberta-base-wikimovies
roberta-base-wikimovies is a fill-mask model from thatdramebaazguy. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as cc-by-4.0.
Language model: roberta-base Language: English Downstream-task: Fill-Mask Training data: wikimovies Eval data: wikimovies Infrastructure: 2x Tesla v100 Code: See example
All-time downloads
923
Public
Repo size
2.2 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.h5655 MB · 40%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- cc-by-4.0
- Model type
- roberta
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- May 20, 2021
- SHA
- b32788c7
Try a prompt
Datasets
Model card
From the Hugging Face model README
roberta-base for MLM
model_name = "thatdramebaazguy/roberta-base-wikimovies"
pipeline(model=model_name, tokenizer=model_name, revision="v1.0", task="Fill-Mask")
Overview
Language model: roberta-base
Language: English
Downstream-task: Fill-Mask
Training data: wikimovies
Eval data: wikimovies
Infrastructure: 2x Tesla v100
Code: See example
Hyperparameters
num_examples = 4346
batch_size = 16
n_epochs = 3
base_LM_model = "roberta-base"
learning_rate = 5e-05
max_query_length=64
Gradient Accumulation steps = 1
Total optimization steps = 816
evaluation_strategy=IntervalStrategy.NO
prediction_loss_only=False
per_device_train_batch_size=8
per_device_eval_batch_size=8
adam_beta1=0.9
adam_beta2=0.999
adam_epsilon=1e-08,
max_grad_norm=1.0
lr_scheduler_type=SchedulerType.LINEAR
warmup_ratio=0.0
seed=42
eval_steps=500
metric_for_best_model=None
greater_is_better=None
label_smoothing_factor=0.0
Performance
perplexity = 4.3808
Some of my work:
- Task
- Fill-Mask
- Library
- transformers
- Type
- roberta
- License
- cc-by-4.0
- Created
- Mar 2, 2022
- Updated
- May 20, 2021