Downloads · 30 days
20
3% of all-time downloads
fewshot-goes-multilingual/mTk-AdversarialQA_en-SberQuAD_ru-1B
mTk-AdversarialQA_en-SberQuAD_ru-1B is a text generation model from fewshot-goes-multilingual. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This model is a generative in-context few-shot learner specialized in Russian. It was trained on a combination of English AdversarialQA and Russian SberQuAD datasets.
Downloads · 30 days
20
3% of all-time downloads
All-time downloads
748
Public
Parameters
1.2B
9.9 GB on disk
Likes
1
Public
Click a slice to open those files.
.bin4.9 GB · 50%
From the Hugging Face model README
This model is a generative in-context few-shot learner specialized in Russian. It was trained on a combination of English AdversarialQA and Russian SberQuAD datasets.
You can find detailed information on Project Github & the referenced paper.
This model is intended to be used in a few-shot in-context learning format in the target language (Russian), or in the source language (English, see below). It was evaluated for unseen task learning (with k=3 demonstrations) in Russian: see the referenced paper for details.
Use the code below to get started with the model.
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
model = AutoModelForSeq2SeqLM.from_pretrained("{this model path}")
tokenizer = AutoTokenizer.from_pretrained("{this model path}")
# Instead, use keywords "Вопрос", "Контекст" and "Отвечать" for Russian few-shot prompts
input_text = """
Question: What is the customer's name?
Context: Origin: Barrack Obama, Customer id: Bill Moe.
Answer: Bill Moe,
Question: What is the customer's name?
Context: Customer id: Barrack Obama, if not deliverable, return to Bill Clinton.
Answer:
"""
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
print("Answer:")
print(tokenizer.decode(outputs))
Training this model can be reproduced by running pip install -r requirements.txt && python train_mt5_qa_en_AQA+ru_info.py .
See the referenced script for hyperparameters and other training configurations.
If you use our models or other resources in your research, please cite our work as follows.
BibTeX:
@inproceedings{stefanik2023resources,
author = {\v{S}tef\'{a}nik, Michal and Kadlčík, Marek and Gramacki, Piotr and Sojka, Petr},
title = {Resources and Few-shot Learners for In-context Learning in Slavic Languages},
booktitle = {Proceedings of the 9th Workshop on Slavic Natural Language Processing},
publisher = {ACL},
numpages = {9},
url = {https://arxiv.org/abs/2304.01922},
}