question-answering-generative-t5-v1-base-s-q-c

Question Answering Generative

The model is intended to be used for Q&A task, given the question & context, the model would attempt to infer the answer text.<br> Model is generative (t5-v1-base), fine-tuned from question-generation-auto-hints-t5-v1-base-s-q-c with - Loss: 0.6751 & Rougel: 0.8022 performance scores.

Live Demo: Question Answering Encoders vs Generative

Encoder based Question Answering V1 <br>Encoder based Question Answering V2

Example code:

from transformers import (
    AutoModelForSeq2SeqLM,
    AutoTokenizer
)

def _generate(query, context, model, device):
    
    FT_MODEL = AutoModelForSeq2SeqLM.from_pretrained(model).to(device)
    FT_MODEL_TOKENIZER = AutoTokenizer.from_pretrained(model)
    input_text = "question: " + query + "</s> question_context: " + context
    
    input_tokenized = FT_MODEL_TOKENIZER.encode(input_text, return_tensors='pt', truncation=True, padding='max_length', max_length=1024).to(device)
    _tok_count_assessment = FT_MODEL_TOKENIZER.encode(input_text, return_tensors='pt', truncation=True).to(device)

    summary_ids = FT_MODEL.generate(input_tokenized, 
                                       max_length=30, 
                                       min_length=5, 
                                       num_beams=2,
                                       early_stopping=True,
                                   )
    output = [FT_MODEL_TOKENIZER.decode(id, clean_up_tokenization_spaces=True, skip_special_tokens=True) for id in summary_ids] 
    
    return str(output[0])

device = [0 if torch.cuda.is_available() else 'cpu'][0]
_generate(query, context, model="consciousAI/t5-v1-base-s-q-c-multi-task-qgen-v2", device=device)   

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 3
• eval_batch_size: 3
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Framework versions

• Transformers 4.23.0.dev0
• Pytorch 1.12.1+cu113
• Datasets 2.5.2
• Tokenizers 0.13.0