Downloads · 30 days
34
2% of all-time downloads
yongzx/gpt2-finetuned-oscar-fr-ori-tok
gpt2-finetuned-oscar-fr-ori-tok is a text generation model from yongzx. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Downloads · 30 days
34
2% of all-time downloads
All-time downloads
1.9K
Public
Repo size
78.6 GB
Likes
0
Public
Click a slice to open those files.
.bin510 MB · 99%
From the Hugging Face model README
We use GPT-2 tokenizer.
We finetuned the wte and wpe layers of GPT-2 (while freezing the parameters of all other layers) on OSCAR's unshuffled_original_fr French data subset. We used Huggingface's code for fine-tuning the causal language model GPT-2, but with the following parameters changed
- preprocessing_num_workers: 8
- per_device_train_batch_size: 2
- gradient_accumulation_steps: 4
- per_device_eval_batch_size: 2
- eval_accumulation_steps: 4
- eval_steps: 1000
- evaluation_strategy: "steps"
- max_eval_samples: 5000
Final checkpoint: checkpoint-76500