llm-course-hw3-lora

LoRA

<!-- Provide a quick summary of what the model is/does. -->

Model Details

This repository features a LoRA fine-tuned model for tweet sentiment classification trained as part of VK's LLM course.

Model Description

<!-- Provide a longer summary of what this model is. -->

PEFT (Parameter-Efficient Fine-Tuning) and LoRA (Low-Rank Adaptation) are two techniques used in machine learning to efficiently adapt large pre-trained neural networks to specific tasks without requiring extensive computational resources.

PEFT (Parameter-Efficient Fine-Tuning): PEFT is a technique that focuses on updating only a small subset of the model’s parameters during fine-tuning, rather than the entire network. This approach reduces computational costs and memory usage, making it feasible to adapt large models to new tasks on devices with limited resources. By targeting specific layers or parameters that are most relevant for the task, PEFT achieves significant improvements in efficiency while maintaining performance.

LoRA (Low-Rank Adaptation): LoRA specifically decomposes the weight updates required for fine-tuning into low-rank representations. During fine-tuning, instead of updating the full weight matrices of a neural network, LoRA updates a low-rank decomposition of these matrices. This reduces the number of trainable parameters and computational demands, preserving the original pre-trained weights while still achieving effective task-specific adaptation. LoRA is especially useful in scenarios where memory and computational resources are limited.

In this case, OuteAI/Lite-Oute-1-300M-Instruct is used as the pre-trained model. This model is further fine-tuned using LoRA to classify tweet's sentiment.

Examples

Before fine-tuning

Tweet: QT @user In the original draft of the 7th book, Remus Lupin survived the Battle of Hogwarts. #HappyBirthdayRemusLupin

True: positive

Predicted: The sentiment of the text is negative.

==================================================================================================================================================

Tweet: "Ben Smith / Smith (concussion) remains out of the lineup Thursday, Curtis #NHL #SJ"

True: neutral

Predicted: The sentiment of the text is negative.

==================================================================================================================================================

Tweet: Sorry bout the stream last night I crashed out but will be on tonight for sure. Then back to Minecraft in pc tomorrow night.

True: neutral

Predicted: The sentiment of the text is negative.

==================================================================================================================================================

Tweet: Chase Headley's RBI double in the 8th inning off David Price snapped a Yankees streak of 33 consecutive scoreless innings against Blue Jays

True: neutral

Predicted: The sentiment of the text is negative.

==================================================================================================================================================

Tweet: @user Alciato: Bee will invest 150 million in January, another 200 in the Summer and plans to bring Messi by 2017"

True: positive

Predicted: The sentiment of the text is negative.

==================================================================================================================================================

After fine-tuning

Tweet: "QT @user In the original draft of the 7th book, Remus Lupin survived the Battle of Hogwarts. #HappyBirthdayRemusLupin"

True: positive

Predicted: positive

==================================================================================================================================================

Tweet: "Ben Smith / Smith (concussion) remains out of the lineup Thursday, Curtis #NHL #SJ"

True: neutral

Predicted: neutral

==================================================================================================================================================

Tweet: Sorry bout the stream last night I crashed out but will be on tonight for sure. Then back to Minecraft in pc tomorrow night.

True: neutral

Predicted: positive

==================================================================================================================================================

Tweet: Chase Headley's RBI double in the 8th inning off David Price snapped a Yankees streak of 33 consecutive scoreless innings against Blue Jays

True: neutral

Predicted: neutral

==================================================================================================================================================

Tweet: @user Alciato: Bee will invest 150 million in January, another 200 in the Summer and plans to bring Messi by 2017"

True: positive

Predicted: neutral

Analysis

References

• Model: OuteAI/Lite-Oute-1-300M-Instruct
• Dataset: cardiffnlp/tweet_eval
• Original Article: https://arxiv.org/pdf/2106.09685