mT5-base-GermanQuAD-e2e-qg

tilomichel/mT5-base-GermanQuAD-e2e-qg

mT5-base-GermanQuAD-e2e-qg is a machine learning model from tilomichel. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.

This model is a finetuned mT5-base model for the task of answer-agnostic (or end-to-end) question generation. The approach from Lopez et al. was used called All questions per line (AQPL). This means a paragraph is pro…

Downloads · 30 days

2% of all-time downloads

All-time downloads

858

Public

Parameters

582M

4.7 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin2.3 GB · 50%

At a glance

Library: transformers
License: mit
Model type: mt5
Access: Public
Created: Jul 2, 2022
Updated: Dec 31, 2023
SHA: 8ff8fbf0

Try a prompt

Datasets

deepset/germanquad

Model card

From the Hugging Face model README

mT5-base finetuned on the GermanQuAD dataset for answer-agnostic question generation

This model is a finetuned mT5-base model for the task of answer-agnostic (or end-to-end) question generation. The approach from Lopez et al. was used called All questions per line (AQPL). This means a paragraph is provided as input and multiple questions are generated from it. Other models already tested this approach with the T5 model for English and German.

For finetuning this model only used the GermanQuAD dataset from deepset was used. The dataset was modified and filtered with scripts that can be found in another repository.

Training, test and evaluation data

For training and test the original split from GermanQuAD was used. As evaluation dataset the German split of the XQuAD dataset was used.

Training hyperparameters

The training parameters are provided in JSON and can be used with a training script provided in a repository

{
    "model_name_or_path": "google/mt5-base",
    "output_dir": "mt5-base-germanquad-e2e-qg",
    "overwrite_output_dir": true,
    "cache_dir": "model-cache",
    "dataset_dir": "e2e-qg-germanquad",
    "preprocessing_num_workers": 20,
    "max_source_length": 1024,
    "max_target_length": 128,
    "val_max_target_length": 128,
    "pad_to_max_length": true,
    "seed": 42,
    "do_train": true,
    "gradient_accumulation_steps": 64,
    "per_device_train_batch_size": 1,
    "per_device_eval_batch_size": 1,
    "learning_rate": 1e-4,
    "num_train_epochs": 10,
    "evaluation_strategy": "epoch",
    "logging_strategy": "epoch",
    "save_strategy": "epoch",
    "save_total_limit": 3,
    "dataloader_num_workers": 8,
    "ddp_find_unused_parameters": false
}

Training results

The evaluation is reported on XQuAD. The implementations and configurations can be found in another repository.

Library: transformers
Type: mt5
License: mit
Languages: de
Created: Jul 2, 2022
Updated: Dec 31, 2023