rugpt3medium-tathagata

RuGPT3Medium-tathagata

Model description

This is the model for text generation for Russian based on rugpt3medium_based_on_gpt2.

Intended uses & limitations

Тhis model was trained and run to generate text on RTX 3080

How to use

from transformers import GPT2LMHeadModel, GPT2Tokenizer
import torch
DEVICE = torch.device("cuda:0")

model_name_or_path = "radm/rugpt3medium-tathagata"
tokenizer = GPT2Tokenizer.from_pretrained("sberbank-ai/rugpt3medium_based_on_gpt2")
model = GPT2LMHeadModel.from_pretrained(model_name_or_path).to(DEVICE)

text = "В чем смысл жизни?\n"
input_ids = tokenizer.encode(text, return_tensors="pt").to(DEVICE)
model.eval()
with torch.no_grad():
    out = model.generate(input_ids, 
                        do_sample=True,
                        num_beams=4,
                        temperature=1.1,
                        top_p=0.9,
                        top_k=50,
                        max_length=250,
                        min_length=50,
                        early_stopping=True,
                        no_repeat_ngram_size=2
                        )

generated_text = list(map(tokenizer.decode, out))[0]
print()
print(generated_text)

Dataset

Dataset based on summaries of major Buddhist, Hindu and Advaita texts such as:

• Diamond Sutra
• Lankavatara Sutra
• Sri Nisargadatta Maharaj quotes
• Quotes from the Bhagavad Gita

Dataset link: tathagata