Downloads · 30 days
2
8% of all-time downloads
ajiayi/bert-peft-singapore-digi-banks
bert-peft-singapore-digi-banks is a text classification model from ajiayi. Use it when you need a label for a piece of text. It is set up for peft. The card lists the license as apache-2.0.
This model is a Parameter-Efficient Fine-Tuned (PEFT) version of distilbert-base-uncased using LoRA (Low-Rank Adaptation) for 5-class star rating prediction (1-5 stars) on Singapore digital bank reviews.
Downloads · 30 days
2
8% of all-time downloads
All-time downloads
25
Public
Repo size
6.5 MB
Likes
0
Public
Click a slice to open those files.
.safetensors3.6 MB · 79%
From the Hugging Face model README
This model is a Parameter-Efficient Fine-Tuned (PEFT) version of distilbert-base-uncased using LoRA (Low-Rank Adaptation) for 5-class star rating prediction (1-5 stars) on Singapore digital bank reviews.
| Metric | Score |
|---|---|
| Test Accuracy | 0.7619 |
| Precision (Macro) | 0.3064 |
| Recall (Macro) | 0.3597 |
| F1-Score (Macro) | 0.3307 |
| MAE (Mean Absolute Error) | 0.6000 stars |
from peft import PeftModel, PeftConfig
from transformers import AutoModelForSequenceClassification, AutoTokenizer
# Load configuration
config = PeftConfig.from_pretrained("ajiayi/bert-peft-singapore-digi-banks")
# Load base model
base_model = AutoModelForSequenceClassification.from_pretrained(
config.base_model_name_or_path,
num_labels=5
)
# Load PEFT adapter
model = PeftModel.from_pretrained(base_model, "ajiayi/bert-peft-singapore-digi-banks")
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
# Make predictions
text = "Love this bank! Best interest rates in Singapore."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
outputs = model(**inputs)
predicted_stars = outputs.logits.argmax(dim=1).item() + 1 # Convert 0-4 to 1-5
print(f"Predicted rating: {predicted_stars} stars")
distilbert-base-uncased (66M parameters)q_lin, v_lin (attention query and value layers)The model was evaluated on a held-out test set with the following results:
from peft import PeftModel
from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch
# Load model and tokenizer
model = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased", num_labels=5)
model = PeftModel.from_pretrained(model, "ajiayi/bert-peft-singapore-digi-banks")
tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")
# Predict
def predict_rating(text):
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
model.eval()
with torch.no_grad():
outputs = model(**inputs)
prediction = outputs.logits.argmax(dim=1).item()
return prediction + 1 # Convert 0-indexed to 1-5 stars
# Example
review = "Great banking app! Very user-friendly interface."
stars = predict_rating(review)
print(f"Predicted: {stars} stars")
@misc{peft-singapore-banks,
author = {Your Name},
title = {PEFT DistilBERT for Singapore Bank Review Classification},
year = {2025},
publisher = {HuggingFace},
howpublished = {\url{https://huggingface.co/ajiayi/bert-peft-singapore-digi-banks}}
}
This model is released under the Apache 2.0 License.
Generated: 2025-10-14 23:08:12