Downloads · 30 days
186
2% of all-time downloads
iliemihai/gpt-neo-romanian-125m
gpt-neo-romanian-125m is a text generation model from iliemihai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This model is a GPT-Neo transformer decoder model designed using EleutherAI's replication of the GPT-3 architecture.
Downloads · 30 days
186
2% of all-time downloads
All-time downloads
8.4K
Public
Repo size
3.4 GB
Likes
3
Public
Click a slice to open those files.
.msgpack2.2 GB · 78%
From the Hugging Face model README
This model is a GPT-Neo transformer decoder model designed using EleutherAI's replication of the GPT-3 architecture.
It was trained on a thoroughly cleaned corpus of Romanian text of about 40GB composed of Oscar, Opus, Wikipedia, literature and various other bits and pieces of text, joined together and deduplicated. It was trained for about a month, totaling 5.8M steps on a v3 TPU machine.
from transformers import GPTNeoForCausalLM, GPT2Tokenizer
model = GPTNeoForCausalLM.from_pretrained("iliemihai/gpt-neo-romanian-125m")
tokenizer = GPT2Tokenizer.from_pretrained("iliemihai/gpt-neo-romanian-125m")
prompt = "Cine a fost mihai eminescu"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
output = model.generate(input_ids, penalty_alpha=0.6, top_k=4, max_length=64)
result = tokenizer.decode(output[0], skip_special_tokens=True)
print(result)
Evaluation to be added soon, also on https://github.com/dumitrescustefan/Romanian-Transformers
Thanks TPU Research Cloud for the TPUv3 machine needed to train this model!