Downloads · 30 days
22
3% of all-time downloads
LuxGPT2-basedGER
laurabernardy/LuxGPT2-basedGER
LuxGPT2-basedGER is a text generation model from laurabernardy. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
GPT-2 model for Text Generation in luxembourgish language, trained on 711 MB of text data, consisting of RTL.lu news articles, comments, parlament speeches, the luxembourgish Wikipedia, Newscrawl, Webcrawl and subtitl…
All-time downloads
659
Public
Repo size
1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin510 MB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- gpt2
- Access
- Public
- Created
- Aug 22, 2022
- Updated
- Oct 15, 2022
- SHA
- 96c07940
Model card
From the Hugging Face model README
LuxGPT-2 based GER
GPT-2 model for Text Generation in luxembourgish language, trained on 711 MB of text data, consisting of RTL.lu news articles, comments, parlament speeches, the luxembourgish Wikipedia, Newscrawl, Webcrawl and subtitles. Created via transfer learning with an German base model, feature space mapping from LB on Base feature space and gradual layer freezing. The training took place on a 32 GB Nvidia Tesla V100
- with One Cycle policy for the learning rate
- with the help of fastai's LR finder
- for 53.4 hours
- for 20 epochs and 7 cycles
- using the fastai library
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("laurabernardy/LuxGPT2-basedGER")
model = AutoModelForCausalLM.from_pretrained("laurabernardy/LuxGPT2-basedGER")
Limitations and Biases
See the GPT2 model card for considerations on limitations and bias. See the GPT2 documentation for details on GPT2.
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- License
- mit
- Languages
- lb
- Created
- Aug 22, 2022
- Updated
- Oct 15, 2022