Downloads · 30 days
228
1% of all-time downloads
mt5-small-sum-de-mit-v1
deutsche-telekom/mt5-small-sum-de-mit-v1
mt5-small-sum-de-mit-v1 is a summarization model from deutsche-telekom. Use it when you need a shorter version of a longer text. It is set up for transformers. The card lists the license as mit.
This is a German summarization model. It is based on the multilingual T5 model google/mt5-small. The special characteristic of this model is that, unlike many other models, it is licensed under a permissive open sourc…
All-time downloads
28.4K
Public
Parameters
300M
2.4 GB on disk
Likes
14
Public
Repo makeup
Click a slice to open those files.
.bin1.2 GB · 50%
At a glance
- Task
- Summarization
- Library
- transformers
- License
- mit
- Model type
- mt5
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- May 18, 2023
- SHA
- fbb1db13
Datasets
Model card
From the Hugging Face model README
mT5-small-sum-de-mit-v1
This is a German summarization model. It is based on the multilingual T5 model google/mt5-small. The special characteristic of this model is that, unlike many other models, it is licensed under a permissive open source license (MIT). Among other things, this license allows commercial use.
This model is provided by the One Conversation
team of Deutsche Telekom AG.
Training
The training was conducted with the following hyperparameters:
- base model: google/mt5-small
- source_prefix:
"summarize: " - batch size: 3 (6)
- max_source_length: 800
- max_target_length: 96
- warmup_ratio: 0.3
- number of train epochs: 10
- gradient accumulation steps: 2
- learning rate: 5e-5
Datasets and Preprocessing
The datasets were preprocessed as follows:
The summary was tokenized with the google/mt5-small tokenizer. Then only the records with no more than 94 summary tokens were selected.
This model is trained on the following dataset:
| Name | Language | Size | License |
|---|---|---|---|
| SwissText 2019 - Train | de | 84,564 | Concrete license is unclear. The data was published in the German Text Summarization Challenge. |
We have permission to use the Swisstext dataset and release the resulting summarization model under MIT license (see permission-declaration-swisstext.pdf).
Evaluation on MLSUM German Test Set (no beams)
| Model | rouge1 | rouge2 | rougeL | rougeLsum |
|---|---|---|---|---|
| deutsche-telekom/mt5-small-sum-de-mit-v1 (this) | 16.8023 | 3.5531 | 12.6884 | 14.7624 |
| ml6team/mt5-small-german-finetune-mlsum | 18.3607 | 5.3604 | 14.5456 | 16.1946 |
| deutsche-telekom/mt5-small-sum-de-en-01 | 21.7336 | 7.2614 | 17.1323 | 19.3977 |
License
Copyright (c) 2021 Philip May, Deutsche Telekom AG
Licensed under the MIT License (the "License"); you may not use this work except in compliance with the License. You may obtain a copy of the License by reviewing the file LICENSE in the repository.
- Task
- Summarization
- Library
- transformers
- Type
- mt5
- License
- mit
- Languages
- de
- Created
- Mar 2, 2022
- Updated
- May 18, 2023