berturk-turkish-sentiment

BERTurk Turkish Sentiment Analysis

This is a Turkish sentiment classification model fine-tuned from dbmdz/bert-base-turkish-cased.

The model classifies Turkish texts into three sentiment categories:

• Negative
• Notr
• Positive

Model Details

• Developed by: Gamze Nur Aslan
• Model type: BERT-based sequence classification model
• Language: Turkish
• Task: Multiclass sentiment classification
• Framework: PyTorch and Hugging Face Transformers
• Fine-tuned from: dbmdz/bert-base-turkish-cased
• Number of classes: 3

Label Mapping

Intended Uses

The model can be used for:

• Turkish product review analysis
• Customer feedback classification
• Social media sentiment analysis
• Educational and research projects
• Sentiment-based text analysis applications

Out-of-Scope Uses

The model should not be used as the sole decision-making system for:

• Medical decisions
• Legal decisions
• Recruitment decisions
• Financial or credit decisions
• Other high-stakes applications

The model was designed for Turkish texts. Its performance on other languages has not been evaluated.

How to Use

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="gmzx/berturk-turkish-sentiment"
)

texts = [
    "Bu ürün gerçekten harika, çok memnun kaldım.",
    "Kargo çok geç geldi ve ürün bozuk çıktı.",
    "Toplantı yarın saat üçte yapılacak."
]

results = classifier(texts)

for text, result in zip(texts, results):
    print(text)
    print(result)

Example output:

Bu ürün gerçekten harika, çok memnun kaldım.
{'label': 'Positive', 'score': 0.99}

Kargo çok geç geldi ve ürün bozuk çıktı.
{'label': 'Negative', 'score': 0.99}

Toplantı yarın saat üçte yapılacak.
{'label': 'Notr', 'score': 0.99}

Training Data

The model was fine-tuned using the winvoker/turkish-sentiment-analysis-dataset.

The original dataset contains Turkish texts collected from several sources:

• Product reviews
• Store reviews
• Wikipedia sentences
• Tweets
• HUMIR
• Random text samples

A balanced subset of 30,000 examples was selected.

The balanced dataset was split into:

• Training set: 24,000 examples
• Validation set: 6,000 examples

Each class contained 8,000 training examples and 2,000 validation examples.

Training Procedure

Preprocessing

The tokenizer associated with dbmdz/bert-base-turkish-cased was used.

The following preprocessing steps were applied:

• Labels were converted to numerical IDs.
• Text truncation was enabled.
• Maximum sequence length was set to 128 tokens.
• Dynamic padding was applied with DataCollatorWithPadding.
• The dataset was split using stratified sampling.

Training Hyperparameters

Evaluation

The model was evaluated on a balanced validation set containing 6,000 examples.

The following metrics were used:

• Accuracy
• Macro precision
• Macro recall
• Macro F1-score

Macro averaging gives equal importance to the Negative, Notr, and Positive classes.

Overall Results

Class-Level Results

Confusion Matrix

The model performs especially well on the Notr class. Most errors occur between the Positive and Negative classes.

Limitations and Bias

The model may have difficulty processing:

• Sarcasm and irony
• Mixed-sentiment sentences
• Slang and spelling mistakes
• Context-dependent expressions
• Very long texts
• Domain-specific terminology

For example, a sentence containing both positive and negative expressions may be classified according to whichever sentiment the model considers dominant.

The training dataset combines texts from different sources. Wikipedia sentences are mostly neutral and stylistically different from product reviews and social media texts. The model may therefore learn source-specific writing patterns in addition to sentiment.

The validation examples were selected from the same collection of sources as the training examples. Performance on completely different real-world datasets may be lower than the reported results.

The model may reproduce offensive expressions, biases, or undesirable associations found in its training data.

Ethical Considerations

Model predictions should be reviewed by a human when used in sensitive contexts.

The model should not be considered an objective measure of a person's emotions, opinions, intentions, or personality.

Author

Gamze Nur Aslan

Computer Engineering Student

Interested in artificial intelligence, natural language processing, and machine learning.