mmbert32k-feedback-detector-lora

mmBERT-32K Feedback Detector (LoRA)

A 4-class user feedback classifier fine-tuned from mmbert-32k-yarn using LoRA (Low-Rank Adaptation).

Model Description

This model classifies user messages into 4 feedback categories to help conversational AI systems understand user satisfaction and respond appropriately:

Performance

Validation Results (2,985 samples):

Per-Class Performance:

Usage

With PEFT (Recommended)

from transformers import AutoModelForSequenceClassification, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForSequenceClassification.from_pretrained(
    "vllm-sr/mmbert-32k-yarn",
    num_labels=4
)
tokenizer = AutoTokenizer.from_pretrained("vllm-sr/mmbert-32k-yarn")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "vllm-sr/mmbert32k-feedback-detector-lora")
model.eval()

# Inference
labels = ["SAT", "NEED_CLARIFICATION", "WRONG_ANSWER", "WANT_DIFFERENT"]
text = "I don't understand your explanation, can you clarify?"

inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
outputs = model(**inputs)
prediction = outputs.logits.argmax(-1).item()

print(f"Feedback: {labels[prediction]}")  # Output: NEED_CLARIFICATION

Using Merged Model (No PEFT required)

For easier deployment, use the merged version:

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained(
    "vllm-sr/mmbert32k-feedback-detector-merged"
)
tokenizer = AutoTokenizer.from_pretrained(
    "vllm-sr/mmbert32k-feedback-detector-merged"
)

Training Details

Hyperparameters

Training Data

Trained on vllm-sr/feedback-detector-dataset:

• Training samples: 17,896 (balanced across 4 classes)
• Validation samples: 2,985

Hardware

• GPU: AMD Instinct MI300X (192GB HBM3)
• Training Time: ~10 minutes
• Framework: PyTorch 2.x with ROCm

Multilingual Support

The model inherits multilingual capabilities from mmbert-32k-yarn (Glot500 tokenizer supporting 1800+ languages). Best performance on:

• English (primary)
• Chinese (Simplified/Traditional)
• French
• Spanish

Limitations

• The SAT class has the strongest performance; some edge cases between WRONG_ANSWER and WANT_DIFFERENT may be ambiguous
• Phrases like "That's perfect, no more questions" may sometimes be misclassified
• Best suited for conversational AI feedback detection, not general sentiment analysis

Citation

@misc{mmbert32k-feedback-detector,
  title={mmBERT-32K Feedback Detector},
  author={LLM Semantic Router Team},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/vllm-sr/mmbert32k-feedback-detector-lora}
}

License

Apache 2.0

Framework Versions

• PEFT: 0.18.1
• Transformers: 4.48+
• PyTorch: 2.6+