bert-end-of-turn-pt

BERT End-of-Turn Detection (PT-BR)

Fine-tuned version of neuralmind/bert-base-portuguese-cased specialized in End-of-Utterance (EoU) / End-of-Turn (EoT) Detection for Brazilian Portuguese conversational agents.

• Author: Jaime Guimarães (UFMG)
• Model Type: Binary Sequence Classification (BertForSequenceClassification)
• Language: Portuguese (pt-BR)
• Repository: JamesGuima/bert-end-of-turn-pt

---

About the model

The Problem

In conversational AI and voice/chat agents, determining when a user has finished their sentence is challenging:

1. High Latency: Waiting for a fixed silence/inactivity timeout (e.g., 3–5 seconds) before querying an LLM makes conversations feel sluggish.
2. Wasted Tokens & Interrupted Context: Processing partial or fragmented inputs immediately sends incomplete thoughts to expensive LLMs (e.g., GPT-5, Claude), degrading response quality and multiplying token costs.

The Solution

This lightweight BERT model acts as a real-time Gatekeeper:

• Analyzes incoming messages in milliseconds (~15–30 ms on CPU).
• Returns LABEL_1 if the utterance is semantically and syntactically complete.
• Returns LABEL_0 if the user is likely still typing or mid-sentence.

---

Classes & Labels

---

How to Use

1. Using Hugging Face pipeline (Easiest)

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="JamesGuima/bert-end-of-turn-pt",
    tokenizer="JamesGuima/bert-end-of-turn-pt"
)

# Complete sentence (End of turn)
res1 = classifier("Quero consultar o limite do meu cartão.")
print(res1)
# [{'label': 'LABEL_1', 'score': 0.9998}]

# Incomplete sentence (Still typing)
res2 = classifier("Eu gostaria de saber sobre o")
print(res2)
# [{'label': 'LABEL_0', 'score': 0.9999}]

2. Using PyTorch & Transformers

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_name = "JamesGuima/bert-end-of-turn-pt"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
model.eval()

def check_end_of_turn(text: str) -> dict:
    inputs = tokenizer(
        text,
        return_tensors="pt",
        padding="max_length",
        truncation=True,
        max_length=64
    )
    
    with torch.no_grad():
        outputs = model(**inputs)
        probs = torch.softmax(outputs.logits, dim=-1)
        pred_label = int(torch.argmax(probs, dim=-1).item())
        confidence = float(probs[0][pred_label].item())
        
    return {
        "is_complete": bool(pred_label == 1),
        "label": pred_label,
        "confidence": confidence
    }

print(check_end_of_turn("como faço para redefinir minha senha"))
# {'is_complete': True, 'label': 1, 'confidence': 0.9997}

print(check_end_of_turn("mas se eu não conseguir"))
# {'is_complete': False, 'label': 0, 'confidence': 0.9989}

---

Practical Use Cases

1. Smart LLM Router (Cost & Quality Optimizer)

• LABEL_1 → Forward message directly to high-capacity reasoning model (e.g., GPT-4o, Claude 3.5 Sonnet).
• LABEL_0 → Forward to a smaller, economical model (e.g., Gemini Flash, GPT-4o-mini) or acknowledge with a quick conversational placeholder.

2. Adaptive Debouncing (Zero-Interruption Chat)

• Instead of an arbitrary fixed timeout on user inputs, trigger a wait buffer (e.g., 3–5 seconds) only when LABEL_0 is returned. If the user stops typing after the buffer expires, dispatch to the agent.

---

Training & Methodology

Dataset Preparation & Augmentation

• Base Dataset: RichardSakaguchiMS/brazilian-customer-service-conversations.
• Data Augmentation: An intentional heuristic cutter (extract_and_break_smart) truncated conversational turns between 20% and 70% of word length to generate realistic negative samples (LABEL_0).
• Domain Adaptation: Injected realistic short conversational patterns (greetings, confirmations, banking queries) with oversampling to ensure stability on short chat inputs.

Training Configuration

• Base Model: neuralmind/bert-base-portuguese-cased
• Sequence Length: max_length = 64 (optimized for rapid chat turn inference)
• Optimizer / LR: AdamW with dynamic learning rate adaptation (5e-5)
• Early Stopping: Monitored on eval_f1 with early stopping patience to prevent overfitting.
• Metric Priority: High precision prioritized on LABEL_1 to minimize false positive turn completions (avoiding unwanted agent interruptions).

---

Evaluation & Latency

• Inference Speed: Average CPU latency of ~15–25 ms per sequence (batch size = 1, max_length = 64), making it suitable for streaming and real-time WebSocket conversational loops.
• Target Metric: F1-score → 0.85 on held-out test splits.

---

Author & Citation

Jaime Guimarães
AI & Machine Learning Researcher
Laboratório de Inteligência e Tecnologia Computacional (LITC) - Universidade Federal de Minas Gerais (UFMG)