llm-course-hw3-dora

DoRA

<!-- Provide a quick summary of what the model is/does. -->

Model Details

This repository features a DoRA fine-tuned model for tweet sentiment classification trained as part of VK's LLM course.

Model Description

<!-- Provide a longer summary of what this model is. -->

PEFT (Parameter-Efficient Fine-Tuning) and DoRA (Weight-Decomposed Low-Rank Adaptation) are two techniques used in machine learning to efficiently adapt large pre-trained neural networks to specific tasks without requiring extensive computational resources.

PEFT (Parameter-Efficient Fine-Tuning): PEFT is a technique that focuses on updating only a small subset of the model’s parameters during fine-tuning, rather than the entire network. This approach reduces computational costs and memory usage, making it feasible to adapt large models to new tasks on devices with limited resources. By targeting specific layers or parameters that are most relevant for the task, PEFT achieves significant improvements in efficiency while maintaining performance.

DoRA (Weight-Decomposed Low-Rank Adaptation): DoRA is a technique for efficiently fine-tuning large pre-trained models by decomposing weight updates into low-rank components. This method separates the magnitude and direction of weight updates, allowing for focused and parameter-efficient adaptations. By targeting only the most influential components, DoRA reduces computational and memory demands while maintaining model performance. This approach is particularly useful for adapting models in resource-constrained environments, enabling scalable fine-tuning with minimal resource usage.

In this case, OuteAI/Lite-Oute-1-300M-Instruct is used as the pre-trained model. This model is further fine-tuned using DoRA to classify tweet's sentiment.

Examples

Before fine-tuning

Tweet: QT @user In the original draft of the 7th book, Remus Lupin survived the Battle of Hogwarts. #HappyBirthdayRemusLupin

True: positive

Predicted: The sentiment of the text is negative.

==================================================================================================================================================

Tweet: "Ben Smith / Smith (concussion) remains out of the lineup Thursday, Curtis #NHL #SJ"

True: neutral

Predicted: The sentiment of the text is negative.

==================================================================================================================================================

Tweet: Sorry bout the stream last night I crashed out but will be on tonight for sure. Then back to Minecraft in pc tomorrow night.

True: neutral

Predicted: The sentiment of the text is negative.

==================================================================================================================================================

Tweet: Chase Headley's RBI double in the 8th inning off David Price snapped a Yankees streak of 33 consecutive scoreless innings against Blue Jays

True: neutral

Predicted: The sentiment of the text is negative.

==================================================================================================================================================

Tweet: @user Alciato: Bee will invest 150 million in January, another 200 in the Summer and plans to bring Messi by 2017"

True: positive

Predicted: The sentiment of the text is negative.

==================================================================================================================================================

After fine-tuning

Tweet: "QT @user In the original draft of the 7th book, Remus Lupin survived the Battle of Hogwarts. #HappyBirthdayRemusLupin"

True: positive

Predicted: positive

==================================================================================================================================================

Tweet: "Ben Smith / Smith (concussion) remains out of the lineup Thursday, Curtis #NHL #SJ"

True: neutral

Predicted: neutral

==================================================================================================================================================

Tweet: Sorry bout the stream last night I crashed out but will be on tonight for sure. Then back to Minecraft in pc tomorrow night.

True: neutral

Predicted: neutral

==================================================================================================================================================

Tweet: Chase Headley's RBI double in the 8th inning off David Price snapped a Yankees streak of 33 consecutive scoreless innings against Blue Jays

True: neutral

Predicted: neutral

==================================================================================================================================================

Tweet: @user Alciato: Bee will invest 150 million in January, another 200 in the Summer and plans to bring Messi by 2017"

True: positive

Predicted: neutral

Analysis

References

• Model: OuteAI/Lite-Oute-1-300M-Instruct
• Dataset: cardiffnlp/tweet_eval
• Original Article: https://arxiv.org/pdf/2402.09353