Downloads · 30 days
24
4% of all-time downloads
LuxGPT-basedEN
laurabernardy/LuxGPT-basedEN
LuxGPT-basedEN is a text generation model from laurabernardy. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
--- GPT-2 model for Text Generation in luxembourgish language, trained on 711 MB of text data, consisting of RTL.lu news articles, comments, parlament speeches, the luxembourgish Wikipedia, Newscrawl, Webcrawl and sub…
All-time downloads
664
Public
Repo size
1 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin510 MB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- gpt2
- Access
- Public
- Created
- Aug 22, 2022
- Updated
- Oct 15, 2022
- SHA
- ead6d504
Model card
From the Hugging Face model README
LuxGPT-2 based GER
GPT-2 model for Text Generation in luxembourgish language, trained on 711 MB of text data, consisting of RTL.lu news articles, comments, parlament speeches, the luxembourgish Wikipedia, Newscrawl, Webcrawl and subtitles. Created via transfer learning with an English base model, feature space mapping from LB on Base feature space and gradual layer freezing. The training took place on a 32 GB Nvidia Tesla V100
- with One Cycle policy for the learning rate
- with the help of fastai's LR finder
- for 49.2 hours
- for 18 epochs and 8 cycles
- using the fastai library
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("laurabernardy/LuxGPT2-basedEN")
model = AutoModelForCausalLM.from_pretrained("laurabernardy/LuxGPT2-basedEN")
Limitations and Biases
See the GPT2 model card for considerations on limitations and bias. See the GPT2 documentation for details on GPT2.
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- License
- mit
- Languages
- lb
- Created
- Aug 22, 2022
- Updated
- Oct 15, 2022