bert-base-uncased-adapter-quora-question-pairs

Prefix-Tuned BERT for Quora Duplicate Question Detection

This repository contains a Prefix-Tuned BERT model fine-tuned on the Quora Duplicate Questions Dataset.
The task is binary classification: given two questions, predict whether they are duplicates.

---

Model Details

• Base Model: tomaarsen/bert-base-nq-prompts
• Fine-tuning Method: PEFT with Prefix-Tuning
• Task: Sequence Classification (Duplicate vs. Not Duplicate)
• Frameworks: Hugging Face Transformers, PEFT, PyTorch

---

Training Configuration

• Epochs: 15
• Batch Size: 32 (train) / 16 (eval)
• Learning Rate: 5e-5 (AdamW fused optimizer)
• Scheduler: Cosine
• Prefix-Tuning Parameters:
  • Virtual tokens: 20
  • Projection: Enabled
  • Layers: All encoder layers

---

Dataset

• Name: Quora Duplicate Questions Dataset
• Train/Test Split: 80/20
• Task: Identify if two questions are semantically duplicate.

---

Results

---

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_name = "mia-project-2025/bert-base-uncased-adapter-quora-question-pairs"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Example input
questions = ["How do I learn Python?", "What is the best way to learn Python?"]
inputs = tokenizer(questions[0], questions[1], return_tensors="pt", padding=True, truncation=True)

# Prediction
outputs = model(**inputs)
pred = outputs.logits.argmax(-1).item()
print("Duplicate" if pred == 1 else "Not Duplicate")