HH-BTRewardModel-roberta

HH-BTRewardModel-roberta

Bradley-Terry reward model trained on Anthropic/hh-rlhf dataset, using log-sigmoid loss:

$$ \mathcal{L} = -\log \sigma(r_{\text{chosen}} - r_{\text{rejected}}) = \log(1 + \exp(-(r_{\text{chosen}} - r_{\text{rejected}}))) $$

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("JERRYPAN617/HH-BTRewardModel-roberta")
model = AutoModelForSequenceClassification.from_pretrained("JERRYPAN617/HH-BTRewardModel-roberta")

text = "Your text here"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
    reward = model(**inputs).logits.item()