xlmr-viquad2.0-qa

🇻🇳 XLM-R fine-tuned on UIT-ViQuAD 2.0 for Vietnamese Question Answering

This repository contains an XLM-RoBERTa model fine-tuned for extractive Question Answering (QA) on UIT-ViQuAD 2.0, a benchmark Vietnamese Machine Reading Comprehension (MRC) dataset with both answerable and unanswerable questions, inspired by SQuAD 2.0.

The model is trained using the Hugging Face run_qa.py pipeline with a fixed hyperparameter configuration to enable fair comparison with other multilingual QA models (e.g. mBERT).

---

📌 Task Description

• Task: Extractive Question Answering (Machine Reading Comprehension)

• Language: Vietnamese

• Input:

  • A context paragraph (Vietnamese text)
  • A question related to the paragraph

• Output:

  • A text span extracted from the context or
  • An empty answer if the question is unanswerable

This setting follows the SQuAD 2.0 paradigm, where models must jointly:

1. Predict answer spans
2. Detect unanswerable questions

---

📚 Dataset: UIT-ViQuAD 2.0

UIT-ViQuAD 2.0 is a Vietnamese QA benchmark released for the VLSP 2021 MRC shared task, addressing a key limitation of earlier Vietnamese datasets by explicitly including unanswerable questions.

Dataset statistics

• Domains: Wikipedia-style articles (history, geography, culture, science, etc.)
• Annotation: Human-annotated answer spans + unanswerable labels

---

🧾 Data Format

1️⃣ Original SQuAD-style format (used for evaluation)

{
  "data": [
    {
      "title": "...",
      "paragraphs": [
        {
          "context": "...",
          "qas": [
            {
              "id": "uit_000001",
              "question": "...",
              "answers": [
                {
                  "text": "...",
                  "answer_start": 123
                }
              ],
              "is_impossible": false
            }
          ]
        }
      ]
    }
  ]
}

This format is required by the official evaluate-v2.0.py script for computing Exact Match (EM) and F1.

---

2️⃣ Hugging Face QA format (used for training & inference)

To train with run_qa.py, the dataset is flattened and normalized into the Hugging Face QA JSON format:

{
  "id": "uit_000001",
  "title": "...",
  "context": "...",
  "question": "...",
  "answers": {
    "text": ["..."],
    "answer_start": [123]
  }
}

---

🔄 Data Preprocessing Pipeline

The dataset is preprocessed in two stages:

🔹 Stage 1: UIT-ViQuAD → SQuAD format

Purpose:

• Align dataset structure with Stanford SQuAD 2.0
• Enable compatibility with the official evaluation script

Steps:

• Group questions under paragraphs
• Preserve answers, is_impossible
• Ensure exact span offsets

---

🔹 Stage 2: SQuAD format → Hugging Face QA format

Purpose:

• Train and infer using run_qa.py

Steps:

• Flatten paragraphs → qas
• Normalize answer fields to {text, answer_start}
• Keep empty answers for unanswerable questions
• Validate span consistency

---

🧠 Model

• Base model: xlm-roberta-base
• Architecture: Transformer encoder (XLM-RoBERTa)
• Head: Span prediction (start_logits, end_logits)
• Tokenizer: XLM-RoBERTa fast tokenizer

---

⚙️ Training Configuration

All models in this project share the same hyperparameters to ensure fair comparison.

Model:              xlm-roberta-base
Max sequence length: 512
Document stride:     256
Train batch size:    16
Eval batch size:     8
Learning rate:       2e-5
Epochs:              3
Optimizer:           AdamW
FP16:                Enabled
Seed:                42
Version 2 QA:        Enabled (unanswerable questions)

Training is performed using Hugging Face Trainer via run_qa.py.

---

📊 Evaluation Results (Private Test Set)

Official evaluation is conducted using the SQuAD v2.0 evaluation script.

{
  "exact": 47.25,
  "f1": 56.84,
  "HasAns_exact": 45.42,
  "HasAns_f1": 59.12,
  "NoAns_exact": 51.52,
  "NoAns_f1": 51.52,
  "total": 3712
}

• Exact Match (EM): strict span-level correctness
• F1: token-level overlap
• Performance is competitive with baseline systems reported in VLSP 2021

---

🚀 Usage

Load the model

from transformers import AutoModelForQuestionAnswering, AutoTokenizer

model = AutoModelForQuestionAnswering.from_pretrained(
    "linhanhvlog123/xlmr-viquad2.0-qa"
)
tokenizer = AutoTokenizer.from_pretrained(
    "linhanhvlog123/xlmr-viquad2.0-qa"
)

Inference (example)

from transformers import pipeline

qa = pipeline("question-answering", model=model, tokenizer=tokenizer)

qa({
    "context": "...",
    "question": "..."
})

---

🏁 Notes

• This repository provides a baseline XLM-R model trained with fixed hyperparameters.

• Further improvements may be obtained via:

  • Hyperparameter tuning
  • Larger batch sizes
  • Longer training
  • Data augmentation

---

📜 Citation

If you use UIT-ViQuAD 2.0 or this model, please cite the original dataset paper from VLSP 2021.