bert-base-uncased-mrpc

bert-base-uncased fine-tuned on GLUE MRPC

This model is a fine-tuned version of bert-base-uncased on the GLUE MRPC dataset (paraphrase detection — predict whether two sentences are equivalent).

Evaluation results

On the MRPC validation set:

Training hyperparameters

• Optimizer: AdamW (lr=5e-5, weight_decay=0.01)
• LR scheduler: linear, 0 warmup steps
• Epochs: 3
• Batch size: 8
• Gradient clipping: max_norm=1.0
• Hardware: GPU (CUDA)

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("shimogerald/bert-base-uncased-mrpc")
model = AutoModelForSequenceClassification.from_pretrained("shimogerald/bert-base-uncased-mrpc")

inputs = tokenizer("The cat sat on the mat.", "A cat is sitting on a mat.",
                   return_tensors="pt", truncation=True)
with torch.no_grad():
    logits = model(**inputs).logits
pred = torch.argmax(logits, dim=1).item()
print("equivalent" if pred == 1 else "not equivalent")