Flan-T5-Large-LoRA-Rephraser-Merged

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

The model is a LoRA trained Flan-T5-Large. The shift in performace came from the LoRA Adapter Layer that was introduced to the model. It helped in training the model using cheaper hardware while not losing any of the accuracy, on the contrary it increased the accuracy. The takes as input 3 elements: Previous_Question, Previous_Answer and Current_Question. It will rephrase the Current_Question based on all the previouses.

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

• Model type: Language model
• Language(s) (NLP): English
• Finetuned from model: Flan-T5-Large

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

The primary use of the model is to generate a rephrased version of a follow-up question. The rephrased sentence will be a standalone sentence, that can be understood independantaly from the previous context. The output is supposed to be a sentence that has the same meaning/context as the combination of (Previous_Question, Previous_Answer, Current_Question)

Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

The model is not intended to be used in fixing grammar or rephrasing using a format not similar to the one the model was trained on.

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

The data used for rephrasing was mainly data that roams around the context that can be seen when dealing with the HR Chatbot for Azadea or the News Chatbot for AL Qabas.

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

It is always recommended to follow the input format that the model was trained on. The format is the following:

text = """
Question:
What about Mondays?

History:
ASSISTANT: No it is not allowed to work remotely on Fridays per company policy.
HUMAN: Can we work remotely on Fridays?
"""

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import pipeline

pipe = pipeline("text2text-generation", model="EuriskoMobility/Flan-T5-Large-LoRA-Rephraser-Merged", max_new_tokens=64)

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

The data used will be provided shortly.

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Using a LoRA adapter while still using 32-bit precision, the model managed to generalize better. With a mix of dropout and weight decay, overfitting was reduced. Using both approaches the model reached performances never reached previously.

Training Hyperparameters

• Training regime:
  • max_input_length: 512
  • max_target_length: 64
  • learning_rate: 1e-4
  • per_device_train_batch_size: 8
  • per_device_eval_batch_size: 8
  • num_train_epochs: 10
  • fp16: False
  • seed: 42
  • metric_for_best_model: "rougel"

Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

The model needs to be tested using pipeline for single input and batched input as well as deployment on Triton and TGI.

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Data, Factors & Metrics

The model was tested on both a test set and a list of key examples to tackle that were gathered around by the team. While "rougel" was taken into consideration, the aim of course was also to decrease training and validation loss as much as possible. Then the model was tested on the 63 examples gathered aside where he only rephrased incorrerctly 6 sentences. Some of them also being in a condition acceptable for the retriever system. as in the retriever would return the desired chunk.

Testing Data

<!-- This should link to a Dataset Card if possible. -->

The data used for testing was augmented data generated mainly from GPT-4 based on a few examples gathered by the team. It is a mix of HR related inputs as well as News related input. It has some conversational examples as well with the goal of generalizing more. Then there is a 63 input list that were manually tested by the team to judge the performance of the model.

Metrics

During training the following metrics were mainly taken into consideration:

• rouge1
• rougel
• training_loss
• evaluation_loss

Results

The best performing checkpoint that we got during training had the followng results:

• rouge1: 0.642358
• rougel: 0.604891
• training_loss: 0.657600
• evaluation_loss: 0.840903

Summary

The model still has room for improvement, but it would require an extensive amount of data that is much more diversified.

Technical Specifications [optional]

Model Architecture and Objective

The model uses Flan-T5-Large architecture as a base but adds to it a LoRA Adapter to help in generalization.

Hardware

1 x A100 GPU