Extractive_MetaQA

UKP-SQuARE/Extractive_MetaQA

Extractive_MetaQA is a machine learning model from UKP-SQuARE. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.

datasets: - squad - newsqa - hotpotqa - biu-nlp/qamr - searchqa - naturalquestions - triviaqa - duorc language: - en metrics: - squad ---

Downloads · 30 days

2% of all-time downloads

All-time downloads

312

Public

Repo size

878 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin439 MB · 100%

At a glance

Library: transformers
Model type: bert
Access: Public
Created: Jan 25, 2023
Updated: Jan 25, 2023
SHA: 09fc6aef

Model card

From the Hugging Face model README

datasets:

squad
newsqa
hotpot_qa
biu-nlp/qamr
search_qa
natural_questions
trivia_qa
duorc language:
en metrics:
squad

Model Card for Model ID

Checkpoint of MetaQA trained only on extractive QA datasets from MetaQA: Combining Expert Agents for Multi-Skill Question Answering (https://arxiv.org/abs/2112.01922)

Evaluation Results

{
    "SQuAD": {
        "exact_match": 86.73139158576052,
        "f1": 92.65156746563402
    },
    "NewsQA": {
        "exact_match": 55.84045584045584,
        "f1": 71.73547617592037
    },
    "HotpotQA": {
        "exact_match": 64.8135593220339,
        "f1": 79.61023604916922
    },
    "SearchQA": {
        "exact_match": 75.04122497055359,
        "f1": 81.37280639135817
    },
    "NaturalQuestionsShort": {
        "exact_match": 69.50763477718915,
        "f1": 81.30374741690376
    },
    "TriviaQA-web": {
        "exact_match": 77.18396711202466,
        "f1": 81.52989853015538
    },
    "QAMR": {
        "exact_match": 72.07531203723292,
        "f1": 83.9068616637681
    },
    "DuoRC": {
        "exact_match": 39.35626573106552,
        "f1": 51.033295034422466
    }
}

Library: transformers
Type: bert
Created: Jan 25, 2023
Updated: Jan 25, 2023