Downloads · 30 days
10
20% of all-time downloads
phi2-cda-gender
ao9000/phi2-cda-gender
phi2-cda-gender is a text generation model from ao9000. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
- Dataset: Wikipedia-10 - Target modules = ["qproj", "kproj", "vproj", "dense", "fc1", "fc2"]
All-time downloads
50
Public
Parameters
2.8B
5.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.6 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- phi
- Access
- Public
- Created
- Jul 22, 2024
- Updated
- Jul 22, 2024
- SHA
- 12264a29
Try a prompt
Model card
From the Hugging Face model README
Lora finetuning on Wikipedia-10, applying counter factual data augmentation (CDA)
- Dataset: Wikipedia-10
- Target modules = ["q_proj", "k_proj", "v_proj", "dense", "fc1", "fc2"]
{
"epoch": 0.060952845355012227,
"total_flos": 1.04520375926784e+18,
"train_loss": 0.5059917987585068,
"train_runtime": 72742.6851,
"train_samples": 1049992,
"train_samples_per_second": 0.88,
"train_steps_per_second": 0.027
}
Training script: https://github.com/ao9000/bias-bench/blob/main/experiments/run_clm.py
- Task
- Text Generation
- Library
- transformers
- Type
- phi
- License
- mit
- Created
- Jul 22, 2024
- Updated
- Jul 22, 2024