g10yfg8d-rlhf-checkpoint-pythia-1b-irl

g10yfg8d-rlhf-checkpoint-pythia-1b-irl

This is the final RLHF model trained with irl reward model.

Model Information

• Base Model: EleutherAI/pythia-1b
• Reward Type: irl
• Dataset: allenai/real-toxicity-prompts
• Final Toxicity Score: 27.3724

IRL Configuration

• Likelihood Type: bradley_terry
• Normalization Strategy: none
• IRL Artifact: matthieubou-imperial-college-london/bayes_irl_vi/posterior_bradley_terry_05megofd:v0
• Use Raw Score: True

Usage

This model can be loaded using the HuggingFace Transformers library:

from transformers import AutoModelForCausalLM
from trl import AutoModelForCausalLMWithValueHead

# Load the model
model = AutoModelForCausalLMWithValueHead.from_pretrained("MattBou00/g10yfg8d-rlhf-checkpoint-pythia-1b-irl")

Training Configuration

The training configuration is saved in training_config.yaml.

---

language: en tags:

• rlhf
• final-model
• irl
• pythia-1b library_name: transformers pipeline_tag: text-generation

---