llama-3.1-8b-oracle-rm-hh-rlhf-harmlessness

theSOL1/llama-3.1-8b-oracle-rm-hh-rlhf-harmlessness

Reward model trained for the harmlessness preference on HH-RLHF harmlessness; original HH train only.

Model

• Type: oracle
• Base checkpoint: meta-llama/Llama-3.1-8B
• Label type: hard
• Architecture: LlamaForSequenceClassification
• Tokenizer: PreTrainedTokenizerFast
• Transformers compatibility: transformers>=4,<5

Evaluation

Validation metrics are computed on the temporary eval split created by the training sweep.

For proxy models, the validation labels are produced by the corresponding oracle reward model.

Loading

from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("theSOL1/llama-3.1-8b-oracle-rm-hh-rlhf-harmlessness")
model = AutoModelForSequenceClassification.from_pretrained("theSOL1/llama-3.1-8b-oracle-rm-hh-rlhf-harmlessness", num_labels=1)

This repository intentionally contains only model, tokenizer, and configuration files. Optimizer states, scheduler states, checkpoints, and trainer state files are not uploaded.