Downloads · 30 days
20
3% of all-time downloads
roberta-base-webgpt-rm
theblackcat102/roberta-base-webgpt-rm
roberta-base-webgpt-rm is a text classification model from theblackcat102. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.
Reward model finetuned from existing pretrain model.
All-time downloads
579
Public
Parameters
125M
997 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin499 MB · 50%
Parameter types
How the weights are stored.
F32125M · 100%
Try a prompt
Datasets
Model card
From the Hugging Face model README
Reward Model pretrained on openai/webgpt_comparison
Reward model finetuned from existing pretrain model.
Things that aligned with the orignal papers
-
Overfits easily using rank loss
-
Small learning rate
Different from the papers
-
Small model performs bad due to lack of world knowledge, since the validation accuracy doesn't even reach 60%. OpenAI RM had 6B parameters.
-
Train using a 80-20 train-validation split on torch AMP settings
Other models I had tried
-
bloomz-560m : embedding size doesn't worth the training, since this dataset only contain english prompt
-
gpt2-large : not stable
-
gpt2-base : not stable
Performance on validation split
| model | val acc | val loss (rank loss) |
|---|---|---|
| roberta-base | 56.21 | 0.71 |
| roberta-large | 57.89 | 0.67 |
| electra-base | 57.02 | 0.70 |
| electra-large | 58.75 | 0.69 |
Tensorboard logs are located under runs/
Note:
- You will have to reweight this model output such that the mean rewards equals to 0
- Task
- Text Classification
- Library
- transformers
- Type
- roberta
- License
- apache-2.0
- Languages
- en
- Created
- Dec 25, 2022
- Updated
- Jul 23, 2023