distilled-bert-imdb-lora-merged

LoRA-Finetuned DistilBERT for Sentiment Classification on the IMDB Dataset

• This repository contains a distilbert-base-uncased model that has been fine-tuned for sentiment classification on the IMDB movie review dataset.

• The fine-tuning was performed efficiently using the LoRA (Low-Rank Adaptation) method from the PEFT library and trained with the PyTorch Lightning framework. This model is a merged version, meaning the LoRA adapter weights have been combined with the base model weights.

• This allows for direct use as a standard AutoModelForSequenceClassification without needing the PEFT library for inference.

You can use the following Python code to perform inference:

from transformers import pipeline, AutoModelForSequenceClassification, AutoTokenizer

repo_id = "Qndhm/distilled-bert-imdb-lora-merged"

# Load the model and tokenizer from the Hub
model = AutoModelForSequenceClassification.from_pretrained(repo_id)
tokenizer = AutoTokenizer.from_pretrained(repo_id)

# Create a sentiment analysis pipeline
sentiment_classifier = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer)

# Example 1: Negative review
text_negative = "This movie was not good at all. The acting was terrible and the plot was boring."
result_neg = sentiment_classifier(text_negative)
print(f"Text: '{text_negative}'")
print(f"Prediction: {result_neg[0]['label']}")
# Expected output: LABEL_0 (or NEGATIVE)

# Example 2: Positive review
text_positive = "I loved this movie, it was fantastic! The visuals were stunning and the story was compelling."
result_pos = sentiment_classifier(text_positive)
print(f"\nText: '{text_positive}'")
print(f"Prediction: {result_pos[0]['label']}")
# Expected output: LABEL_1 (or POSITIVE)