Downloads · 30 days
4
1% of all-time downloads
gpt2-cda-religion
ao9000/gpt2-cda-religion
gpt2-cda-religion is a text generation model from ao9000. Use it when you need the model to write or continue text. It is set up for transformers.
- Dataset: Wikipedia-10 - Target modules = ["query", "value"]
All-time downloads
455
Public
Parameters
124M
249 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors249 MB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- gpt2
- Access
- Public
- Created
- Jul 22, 2024
- Updated
- Jul 22, 2024
- SHA
- f4a27268
Try a prompt
Model card
From the Hugging Face model README
Lora finetuning on Wikipedia-10, applying counter factual data augmentation (CDA)
- Dataset: Wikipedia-10
- Target modules = ["query", "value"]
{
"epoch": 2.8509064991759097,
"total_flos": 3.390811584548659e+16,
"train_loss": 1.3963899144530296,
"train_runtime": 5715.6632,
"train_samples": 22449,
"train_samples_per_second": 11.197,
"train_steps_per_second": 0.35
}
Training script: https://github.com/ao9000/bias-bench/blob/main/experiments/run_clm.py
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- Created
- Jul 22, 2024
- Updated
- Jul 22, 2024