roberta-base-squadv1-adam-pol-c8e8

roberta-base fine-tuned on SQuAD

This model is a fine-tuned version of roberta-base on the SQuAD dataset.

Training Details

Training Hyperparameters

• Model: roberta-base
• Dataset: SQuAD
• Optimizer: adamw
• Learning Rate Scheduler: polynomial
• Learning Rate: 2e-05
• Batch Size: 16 per device
• Total Batch Size: 128
• Epochs: 6 (with early stopping)
• Weight Decay: 0.01
• Warmup Ratio: 0.03
• Max Gradient Norm: 1.0

Early Stopping

• Patience: 6
• Metric: f1
• Best Epoch: 5

Usage

from transformers import AutoTokenizer, AutoModelForQuestionAnswering

tokenizer = AutoTokenizer.from_pretrained("HariomSahu/roberta-base-squadv1-adam-pol-c8e8")
model = AutoModelForQuestionAnswering.from_pretrained("HariomSahu/roberta-base-squadv1-adam-pol-c8e8")

# Example usage
question = "What is the capital of France?"
context = "France is a country in Europe. Its capital city is Paris."

inputs = tokenizer(question, context, return_tensors="pt")
outputs = model(**inputs)

# Get answer
start_scores, end_scores = outputs.start_logits, outputs.end_logits
start_index = start_scores.argmax()
end_index = end_scores.argmax()
answer = tokenizer.decode(inputs["input_ids"][0][start_index:end_index+1])
print(f"Answer: {answer}")

Evaluation Results

The model achieved the following results on the evaluation set:

• Exact Match: 85.3359
• F1 Score: 91.8861

Training Configuration Hash

Config Hash: c8e833b8

This hash can be used to reproduce the exact training configuration.