distilbert-lora-sentiment-amazon

DistilBERT-LoRA Sentiment Classifier (Amazon Reviews)

Model Description

Developed by: sunil9938
Model type: Text Classification (Sentiment Analysis)
Language: English
License: MIT
Finetuned from model: distilbert-base-uncased

This model is a fine-tuned version of DistilBERT using LoRA (Low-Rank Adaptation) for sentiment analysis on Amazon product reviews. It achieves 87.8% accuracy on the test set with only 1.09% trainable parameters.

Key Features

• Efficient: Only 739,586 trainable parameters
• Accurate: 87.8% accuracy
• Fast: ~50ms inference time on CPU
• Lightweight: LoRA adapter is only 2.96 MB

How to Use

from transformers import AutoTokenizer, AutoModelForSequenceClassification
from peft import PeftModel

tokenizer = AutoTokenizer.from_pretrained("sunil9938/distilbert-lora-sentiment-amazon")
base_model = AutoModelForSequenceClassification.from_pretrained(
    "distilbert-base-uncased",
    num_labels=2,
    ignore_mismatched_sizes=True
)
model = PeftModel.from_pretrained(base_model, "sunil9938/distilbert-lora-sentiment-amazon")
model.eval()

def predict(text):
    inputs = tokenizer(text, truncation=True, padding=True, max_length=256, return_tensors="pt")
    outputs = model(**inputs)
    probs = outputs.logits.softmax(dim=1)
    pred = probs.argmax().item()
    return "POSITIVE" if pred == 1 else "NEGATIVE", probs[0][pred].item()

print(predict("This product is amazing!"))



## Limitations

- English only
- Binary classification (no neutral)
- Trained on Amazon reviews only

## Citation

```bibtex
@misc{sunil9938-distilbert-lora-sentiment,
  author = {Sunil Kumar},
  title = {DistilBERT-LoRA Sentiment Classifier for Amazon Reviews},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/sunil9938/distilbert-lora-sentiment-amazon}
}