Downloads · 30 days
0
mT5-small-VT-span-mlm_deplain-apa
vera-8/mT5-small-VT-span-mlm_deplain-apa
mT5-small-VT-span-mlm_deplain-apa is a text generation model from vera-8. Use it when you need the model to write or continue text. It is set up for transformers.
Finetuned mT5-Model for German sentence-level text-simplification.
Access
Public
Updated May 7, 2024
Repo size
62.3 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors20.5 MB · 86%
At a glance
- Task
- Text Generation
- Library
- transformers
- Access
- Public
- Created
- May 5, 2024
- Updated
- May 7, 2024
- SHA
- 49f33aaf
Base models
Datasets
Model card
From the Hugging Face model README
Model Card for mT5-small-VT-span-mlm_deplain-apa
Finetuned mT5-Model for German sentence-level text-simplification.
Model Details
Model Description
- Model type: Encoder-Decoder-Transformer
- Language(s) (NLP): German
- Finetuned from model: google/mT5-small
- Task: Text-Simplification
Training Details
Training Data
DEplain/DEplain-APA-sent
Stodden et al. (2023): arXiv:2305.18939
Training Procedure
Parameter-efficient Fine-Tuning with LoRA. \
Vocabulary adjusted through Vocabulary Transfer (Mosin et al. ()).
Training Hyperparameters
- Batch Size: 16
- Epochs: 1
- Learning Rate: 0.001
- Optimizer: Adafactor
LoRA Hyperparameters
- R: 32
- Alpha: 64
- Dropout: 0.1
- Target modules: all linear layers
- Task
- Text Generation
- Library
- transformers
- Languages
- de
- Created
- May 5, 2024
- Updated
- May 7, 2024