bert-churn-prediction

BERT Fine-Tuned Model for Churn Prediction

This repository hosts a fine-tuned version of the BERT model optimized for churn prediction using the provided dataset. The model is designed to analyze textual data and predict customer churn with high accuracy.

Model Details

• Model Architecture: BERT (Bidirectional Encoder Representations from Transformers)
• Task: Churn Prediction
• Dataset: Custom Dataset (processed and structured for binary classification)
• Quantization: FP16
• Fine-tuning Framework: Hugging Face Transformers

🚀 Usage

Installation

pip install transformers torch pandas scikit-learn

Loading the Model

from transformers import BertTokenizer, BertForSequenceClassification
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"

model_name = "AventIQ-AI/bert-churn-prediction"
model = BertForSequenceClassification.from_pretrained(model_name).to(device)
tokenizer = BertTokenizer.from_pretrained(model_name)

Churn Prediction Inference

def predict_churn(text):
    inputs = tokenizer(text, padding=True, truncation=True, max_length=512, return_tensors="pt").to(device)
    with torch.no_grad():
        outputs = model(**inputs)
        predicted_label = torch.argmax(outputs.logits, dim=1).item()
    return "Churn" if predicted_label == 1 else "Not Churn"

# Example usage
customer_review = "I am unhappy with the service and want to cancel."
print(predict_churn(customer_review))

📊 Evaluation Results

After fine-tuning the BERT model for churn prediction, we evaluated the model's performance on the test set. The following results were obtained:

Fine-Tuning Details

Model: Fine-tuned BERT for churn prediction using a custom dataset. Training: Run for 3 epochs with a batch size of 8, using the AdamW optimizer and a learning rate of 2e-5.

Dataset

The dataset consists of customer interactions, reviews, and metadata used to determine churn likelihood. Textual features like title, features, description, and average rating were merged to create input text samples.

Training

• Number of epochs: 3
• Batch size: 8
• Evaluation strategy: Epochs

Quantization

Post-training quantization was applied using PyTorch's built-in quantization framework to reduce the model size and improve inference efficiency.

📂 Repository Structure

.
├── model/               # Contains the quantized model files
├── tokenizer_config/    # Tokenizer configuration and vocabulary files
├── model.safetensors/   # Quantized Model
├── README.md            # Model documentation

⚠️ Limitations

• May struggle with ambiguous or very short text inputs.
• Quantization may slightly impact model accuracy.
• Performance may vary across different industries and customer segments.

🤝 Contributing

Contributions are welcome! Feel free to open an issue or submit a pull request if you have suggestions or improvements.