Bert-User-Review-Rating

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

This model performs sentiment analysis on user reviews, rating them on a scale from 1 (Very Bad) to 5 (Very Good).

Model Details

Model Description

The Bert-User-Review-Rating model is trained on a dataset of 1,300,000 reviews of public places and points of interest. It is capable of classifying user ratings from 1 to 5, where:

5 = Very Good

4 = Good

3 = Neutral

2 = Bad

1 = Very Bad

Model type: BERT-based sentiment analysis model

Language(s) (NLP): English

Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

The model can be used directly to classify the sentiment of user reviews for public places and points of interest, providing a rating from 1 to 5.

Bias, Risks and Limitations

The model may reflect biases present in the training data, such as cultural or regional biases, as training data reflects public places in Singapore.

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users should be aware of potential biases and limitations in the model’s performance, particularly when applied to reviews from different regions or contexts.

How to Get Started with the Model

Use the code below to get started with the model.

Use a pipeline as a high-level helper

from transformers import pipeline

pipe = pipeline("text-classification", model="mekes/Bert-User-Review-Rating")

result = pipe("The food was super tasty, I enjoyed every bite.")

print(result)

Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

The subjective performance of the model is better than the metrices, because it was evaluated regarding meeting the exact correct prediction, so if it reflects 4 instead of 5 it was counted as wrong prediciton. Even though the prediction was way better than predicting 1 for a 5 review.

Test Accuracy: 0.714 Test F1 Score: 0.695 Test Loss: 0.698 Test Recall: 0.714

Environmental Impact

Carbon emissions were estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019)

Calculations were done with Nvidia RTX 3090 instead of the used Nvidia RTX 4090.

For one training run it emmited approximately 1,5 kg CO2