Downloads · 30 days
278
3% of all-time downloads
gpt-neo-romanian-780m
dumitrescustefan/gpt-neo-romanian-780m
gpt-neo-romanian-780m is a text generation model from dumitrescustefan. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This model is a GPT-Neo transformer decoder model designed using EleutherAI's replication of the GPT-3 architecture.
All-time downloads
9.2K
Public
Repo size
6.5 GB
Likes
13
Public
Repo makeup
Click a slice to open those files.
.bin3.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- gpt_neo
- Access
- Public
- Created
- Aug 29, 2022
- Updated
- Sep 17, 2022
- SHA
- 8b19795e
Model card
From the Hugging Face model README
GPT-Neo Romanian 780M
This model is a GPT-Neo transformer decoder model designed using EleutherAI's replication of the GPT-3 architecture.
It was trained on a thoroughly cleaned corpus of Romanian text of about 40GB composed of Oscar, Opus, Wikipedia, literature and various other bits and pieces of text, joined together and deduplicated. It was trained for about a month, totaling 1.5M steps on a v3-32 TPU machine.
Authors:
- Dumitrescu Stefan
- Mihai Ilie
Evaluation
Evaluation to be added soon, also on https://github.com/dumitrescustefan/Romanian-Transformers
Acknowledgements
Thanks TPU Research Cloud for the TPUv3 machine needed to train this model!
- Task
- Text Generation
- Library
- transformers
- Type
- gpt_neo
- License
- mit
- Languages
- ro
- Created
- Aug 29, 2022
- Updated
- Sep 17, 2022