llm-course-hw3-lora

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Model Details

In this work, we fine tuned the base model OuteAI/Lite-Oute-1-300M-Instruct on a tweet sentiment dataset cardiffnlp/tweet_eval dataset to determine tweets tonality in one of the three classes: positive, neutral or negative.

Model Description

We used a system prompt to instruct the model:

SYSTEM PROMPT:

You are a tweet sentiment classifier. For each tweet input, analyze its sentiment and output exactly one word: "negative", "neutral", or "positive". Do not include any extra text.

But the model is not trained to return only the sentiment name.
So we designed a custom LoRA Linear layer to achive PEFT of this model, by replacing the k_proj and v_proj layers to modify the initial model.

Training Details

This model was trained with batch_size=16, rank = 8, alpha = 16, learning_rate = 5e-6 for 1 epoch.

The model achieved 0.51 macro f1-score on the test dataset, comparing with the initial model which is 0.06.

Comparison

==========
User Prompt: "Ben Smith / Smith (concussion) remains out of the lineup Thursday, Curtis #NHL #SJ"
Label: neutral
Before: The tweet "Ben Smith / Smith (concussion) remains out of the
After: neutral neutral ralph neutral ral

==========
User Prompt: @user Alciato: Bee will invest 150 million in January, another 200 in the Summer and plans to bring Messi by 2017"
Label: positive
Before: The tweet "Alciato: Bee will invest 150
After: neutral ralitive ralitive ralitive

Summary

LoRA fine-tuning allows the model to learn in a subspace, thereby adapting the model to new tasks.