qwen2.5-1.5b-eou-detector

Arabic End-of-Utterance (EOU) Detection

Fine-tuning Qwen2.5-1.5B using LoRA

Overview

This repository contains a fine-tuned LoRA adapter for Arabic End-of-Utterance (EOU) detection. The goal is to predict whether a speaker is likely to finish their turn, enabling smoother real-time AI voice agents and more natural conversational interaction.

This project was created as part of a technical assessment to demonstrate my ability to:

• Prepare and clean custom datasets
• Build an EOU classification pipeline
• Fine-tune a large language model with LoRA
• Evaluate and serve the model
• Document and package the work professionally

The model, dataset format, and outputs are not reliable for production use. They are included only to show technical skills, not for deployment in real products or systems.

---

Dataset

Training data follows a .jsonl format with the following fields:

• session_id: Unique conversation/session identifier
• type: Type of sample (positive/negative EOU)
• original_text: Full utterance
• ground_truth: Label indicating whether the utterance ends the turn
• formatted_text: Preprocessed version used as model input

The dataset used here is custom, limited, and only intended to support the demonstration of workflow and model training.

---

Model

The base model used is Qwen2.5-1.5B, fine-tuned using LoRA adapters via HuggingFace Transformers.

Contents included in this repository / HF Hub upload:

• adapter_model.safetensors
• adapter_config.json
• tokenizer.json
• tokenizer_config.json
• special_tokens_map.json

Why LoRA

LoRA was chosen to:

• Reduce compute and GPU memory requirements
• Speed up experimentation
• Keep the final model lightweight and shareable

The LoRA adapter captures task-specific behavior without modifying the full base model.

---

Training

Training was handled via a custom script that:

• Loads Qwen2.5-1.5B
• Applies LoRA configuration
• Tokenizes dataset with appropriate truncation
• Trains for several epochs
• Evaluates on a validation split
• Saves only adapter weights for portability

Key libraries used:

• HuggingFace Transformers
• HuggingFace Accelerate
• PEFT (LoRA)
• PyTorch

---

Usage

To load the model with the LoRA adapter:

from transformers import AutoModelForSequenceClassification, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen2.5-1.5B"
adapter = "Habibaaaaa/qwen2.5-1.5b-eou-detector"

tokenizer = AutoTokenizer.from_pretrained(adapter)
model = AutoModelForSequenceClassification.from_pretrained(base)
model = PeftModel.from_pretrained(model, adapter)

inputs = tokenizer("النص هنا", return_tensors="pt")
outputs = model(**inputs)

The output is a probability indicating whether the utterance is likely an end-of-turn.

---

Important Notice

This model, dataset, and results are:

• Not reliable
• Not thoroughly validated
• Not intended for use in production environments
• Provided solely for demonstrating applied machine learning and NLP engineering skills in an assessment context

---