Llama-3.2-3B-tr-ABSA

Llama-3.2-3B Turkish ABSA

This model is a LoRA fine-tuned version of meta-llama/Llama-3.2-3B-Instruct specifically trained for end-to-end aspect-based sentiment analysis on Turkish e-commerce product reviews.

Model Description

• Base Model: meta-llama/Llama-3.2-3B-Instruct
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Task: End-to-End Aspect-Based Sentiment Analysis
• Language: Turkish
• Domain: E-commerce product reviews
• Output Format: JSON with aspect terms and their polarities

Training Details

Training Data

• Dataset Size: 16,000 reviews
• Data Source: Private e-commerce product review dataset
• Domain: E-commerce product reviews in Turkish

LoRA Configuration

• Rank (r): 16
• LoRA Alpha: 32
• LoRA Dropout: 0.05
• Bias: none
• Task Type: CAUSAL_LM
• Target Modules: Automatically determined linear layers

Training Configuration

• Training Epochs: 1
• Batch Size: 1 (per device)
• Gradient Accumulation Steps: 2
• Optimizer: paged_adamw_32bit
• Learning Rate: 2e-4
• Max Sequence Length: 1024
• Evaluation Strategy: steps (every 0.2 of training)
• Warmup Steps: 10
• Logging Steps: 10
• Precision: FP32 (fp16=False, bf16=False)
• Group by Length: True

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-3B-Instruct",
    torch_dtype=torch.float16,
    low_cpu_mem_usage=True
)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")

# Load LoRA adapter
peft_model = PeftModel.from_pretrained(base_model, "opdullah/Llama-3.2-3B-tr-ABSA")

# Example review
review = "Bu telefonun arka kamerasını beğendim ama bataryası yetersiz."

# Prepare input
messages = [{"role": "user", "content": review}]
inp = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
input_ids = tokenizer(inp, return_tensors="pt")["input_ids"].to("cuda")

# Generate output
outputs = peft_model.generate(input_ids, max_new_tokens=1024)
result = tokenizer.decode(outputs[0]).split("<|start_header_id|>assistant<|end_header_id|>")[-1]

print(result)

Expected Output:

[{"term": "arka kamerasını", "polarity": "positive"}, {"term": "bataryası", "polarity": "negative"}]

Output Format

The model outputs JSON format with the following structure:

[
    {
        "term": "aspect_term_in_turkish",
        "polarity": "positive|negative|neutral"
    }
]

Example outputs:

• [{"term": "arka kamerasını", "polarity": "positive"}, {"term": "bataryası", "polarity": "negative"}]
• [{"term": "fiyatı", "polarity": "positive"}, {"term": "kalitesi", "polarity": "negative"}]
• [{"term": "teslimat hızı", "polarity": "positive"}, {"term": "ambalaj", "polarity": "positive"}]

Advantages

• End-to-End: Performs both aspect extraction and sentiment analysis in one step
• JSON Output: Structured output that's easy to parse and integrate
• Contextual Understanding: Leverages LLM's contextual understanding for better accuracy
• Flexible: Can handle complex aspects and nuanced sentiments
• Efficient: Single model inference instead of two separate models

Requirements

torch>=2.0.0
transformers>=4.36.0
peft>=0.7.0
accelerate>=0.25.0

Intended Use

This model is designed for:

• End-to-end aspect-based sentiment analysis of Turkish e-commerce reviews
• Extracting both aspects and their sentiments in a single inference
• Building review analysis systems and recommendation engines
• Research in Turkish NLP and sentiment analysis

Limitations

• Trained specifically on e-commerce domain data
• Performance may vary on other domains or text types
• Requires GPU for practical inference speeds
• Limited to Turkish language
• Based on private dataset, so reproducibility may be limited
• Output format is JSON, requires parsing for integration

Citation

If you use this model, please cite:

@misc{llama-turkish-absa,
  title={Llama-3.2-3B Turkish ABSA},
  author={Abdullah Koçak},
  year={2025},
  url={https://huggingface.co/opdullah/Llama-3.2-3B-tr-ABSA}
}

Base Model Citation

@misc{llama3.2,
  title={Llama 3.2: Revolutionizing edge AI and vision with open, customizable models},
  author={Meta},
  year={2024},
  publisher={Meta AI},
  url={https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/}
}

Related Models

• opdullah/bert-turkish-ecomm-aspect-extraction - BERT-based aspect extraction
• opdullah/bert-turkish-ecomm-absa - BERT-based aspect sentiment analysis