Downloads · 30 days
12
0% of all-time downloads
opt-125M-pt-br-finetuned
Mirelle/opt-125M-pt-br-finetuned
opt-125M-pt-br-finetuned is a text generation model from Mirelle. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Fine-tuning the OPT-125M model on a reduced corpus of mc4-Portuguese with approximately 300M tokens.
All-time downloads
2.4K
Public
Repo size
1 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin501 MB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- opt
- Access
- Public
- Created
- May 21, 2023
- Updated
- May 21, 2023
- SHA
- 6210d866
Try a prompt
Datasets
Model card
From the Hugging Face model README
OPT-125M finetuned Portuguese
Fine-tuning the OPT-125M model on a reduced corpus of mc4-Portuguese with approximately 300M tokens.
Hyper-parameters
- learning_rate = 5e-5
- batch_size = 32
- warmup = 500
- seq_length = 512
- num_train_epochs = 2.0
With an A100 with 40GB of RAM, the training took around 3 hours
Perplexity: 9.4
Sample Use
from transformers import pipeline
generator = pipeline('text-generation', model='Mirelle/opt-125M-pt-br-finetuned', max_length=100, do_sample=True)
generator("Em uma bela manhã de")
- Task
- Text Generation
- Library
- transformers
- Type
- opt
- License
- mit
- Languages
- pt
- Created
- May 21, 2023
- Updated
- May 21, 2023