mmbert-feedback-detector-merged

mmBERT Feedback Detector (Merged)

A multilingual 4-class user feedback classifier built on jhu-clsp/mmBERT-base. This model classifies user responses into satisfaction categories to help understand user intent in conversational AI systems.

Model Description

This is the merged model (LoRA weights merged into base model) for direct inference without PEFT. For the LoRA adapter version, see vllm-sr/mmbert-feedback-detector-lora.

Labels

Performance

Per-Class Performance

Multilingual Support

Thanks to mmBERT's multilingual pretraining (256k vocabulary, 100+ languages), this model achieves excellent cross-lingual transfer:

Usage

With Transformers

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

model_name = "vllm-sr/mmbert-feedback-detector-merged"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Example: Classify user feedback
text = "Thanks, that's exactly what I needed!"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)

with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.softmax(outputs.logits, dim=-1)
    pred = probs.argmax().item()

labels = ["SAT", "NEED_CLARIFICATION", "WRONG_ANSWER", "WANT_DIFFERENT"]
print(f"Prediction: {labels[pred]} ({probs[0][pred]:.1%})")
# Output: Prediction: SAT (100.0%)

With Pipeline

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="vllm-sr/mmbert-feedback-detector-merged"
)

# English
result = classifier("Thanks, that's helpful!")
print(result)  # [{'label': 'SAT', 'score': 0.999...}]

# Spanish (cross-lingual transfer)
result = classifier("¡Gracias, eso es muy útil!")
print(result)  # [{'label': 'SAT', 'score': 0.999...}]

# Chinese
result = classifier("谢谢，这很有帮助！")
print(result)  # [{'label': 'SAT', 'score': 0.98...}]

Training Details

• Base Model: jhu-clsp/mmBERT-base
• Method: LoRA fine-tuning + merge
• LoRA Rank: 16
• LoRA Alpha: 32
• Learning Rate: 2e-5
• Batch Size: 32
• Epochs: 5
• Max Length: 512
• Dataset: vllm-sr/feedback-detector-dataset

Use Cases

• Conversational AI: Understand if users are satisfied with chatbot responses
• Customer Support: Route dissatisfied users to human agents
• Quality Monitoring: Track response quality across languages
• Feedback Analysis: Categorize user feedback automatically

Related Models

• vllm-sr/mmbert-feedback-detector-lora - LoRA adapter version
• vllm-sr/mmbert-intent-classifier-merged - Intent classification
• vllm-sr/mmbert-fact-check-merged - Fact checking
• vllm-sr/mmbert-jailbreak-detector-merged - Security

Citation

@misc{mmbert-feedback-detector,
  title={mmBERT Feedback Detector},
  author={vLLM Semantic Router Team},
  year={2025},
  publisher={Hugging Face},
  url={https://huggingface.co/vllm-sr/mmbert-feedback-detector-merged}
}

License

Apache 2.0