HotPotQA_DEBERT

DeBERTa-v3-base for Open-Domain Multi-Hop Question Answering

A fine-tuned DeBERTa-v3-base model for extractive question answering on the HotpotQA dataset. The model is designed to work as the reader component of a Retrieval-Augmented Question Answering (QA) pipeline, where a dense retriever first retrieves relevant documents and the reader extracts the answer span from the retrieved context.

---

Model Overview

• Base Model: microsoft/deberta-v3-base
• Task: Extractive Question Answering
• Framework: Hugging Face Transformers
• Dataset: HotpotQA (Distractor Setting)
• Language: English

Unlike generative language models, this model predicts the exact answer span from a retrieved context instead of generating free-form text.

---

Pipeline Architecture

Question
      │
      ▼
Dense Retriever (BGE-base-en-v1.5 + FAISS)
      │
Top-10 Retrieved Documents
      │
      ▼
DeBERTa-v3-base Reader
      │
Predicted Answer Span
      │
      ▼
(Optional) LLM Explanation Layer

The retriever identifies relevant Wikipedia articles, while DeBERTa extracts the answer span from the retrieved passages.

---

Dataset

The model was fine-tuned on the HotpotQA dataset.

Data Preparation

For each question:

1. Retrieve the top-10 Wikipedia articles using a dense retriever.
2. Assemble the retrieved passages into a context.
3. If retrieval misses supporting documents, use the gold supporting passages from HotpotQA.
4. Remove:
   • Yes/No questions
   • Samples where the answer cannot be aligned within the context

Final Dataset Size

---

Retriever

The reader depends on an external dense retriever.

Embedding Model

• BAAI/bge-base-en-v1.5

Vector Database

• FAISS Flat Index

Corpus

• 507,494 Wikipedia articles

Embedding Dimension

• 768

Retrieval Performance

---

Training Configuration

---

Tokenization

The model uses the official DeBERTa tokenizer.

Configuration:

• truncation="only_second"
• padding="max_length"
• max_length=512
• stride=128
• return_overflowing_tokens=True
• return_offsets_mapping=True

Answer spans are aligned using character-level offsets.

Windows that do not contain the answer are assigned start/end position 0.

---

Evaluation

Performance on the HotpotQA validation set:

Comparison with other transformer readers:

---

Example Usage

from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="YOUR_USERNAME/deberta-hotpotqa",
    tokenizer="YOUR_USERNAME/deberta-hotpotqa"
)

result = qa_pipeline(
    question="Which magazine was started first, Arthur's Magazine or First for Women?",
    context=context
)

print(result["answer"])

---

Intended Use

This model is suitable for:

• Open-domain Question Answering
• Multi-hop Question Answering
• Educational Assistants
• Search Systems
• Retrieval-Augmented QA Pipelines
• Research in Extractive Question Answering

---

Limitations

This model:

• Requires an external retrieval system.
• Cannot retrieve documents by itself.
• Produces extractive answer spans only.
• Does not generate explanations.
• Performance depends heavily on retrieval quality.
• Was trained only on English Wikipedia.
• Does not support Yes/No questions since they were removed during preprocessing.

---

Future Improvements

Potential improvements include:

• Hybrid BM25 + Dense Retrieval
• Cross-Encoder Re-ranking
• Paragraph-level indexing
• Iterative Multi-hop Retrieval
• Confidence estimation
• Citation generation
• Integration with an instruction-tuned LLM for grounded answer explanations

---

---

License

This project is intended for research and educational purposes. Please ensure compliance with the licenses of the original HotpotQA dataset and the DeBERTa-v3-base model.