Downloads · 30 days
14
4% of all-time downloads
danielfein/meta-llama_Llama-3.1-8B_bt_reward_template_1205
meta-llama_Llama-3.1-8B_bt_reward_template_1205 is a text classification model from danielfein. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as llama3.1.
This model is a fine-tuned LlamaForSequenceClassification reward model based on meta-llama/Llama-3.1-8B. It was trained with TRL reward modeling and should be used to score stories, not to generate text.
Downloads · 30 days
14
4% of all-time downloads
All-time downloads
312
Public
Parameters
7.5B
150 GB on disk
Likes
0
Public
Click a slice to open those files.
.pt90.1 GB · 53%
From the Hugging Face model README
This model is a fine-tuned LlamaForSequenceClassification reward model based on meta-llama/Llama-3.1-8B. It was trained with TRL reward modeling and should be used to score stories, not to generate text.
This is a reward model, not a text-generation model. Load it with AutoModelForSequenceClassification and score the story directly as raw text. Do not apply a chat template or wrap the story in a prompt.
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
model_id = "danielfein/meta-llama_Llama-3.1-8B_bt_reward_template_1205"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
if tokenizer.pad_token is None:
tokenizer.pad_token = tokenizer.eos_token
if model.config.pad_token_id is None:
model.config.pad_token_id = tokenizer.pad_token_id
def reward(story: str) -> float:
inputs = tokenizer(
story.strip(),
return_tensors="pt",
truncation=True,
max_length=4096,
).to(model.device)
with torch.inference_mode():
return model(**inputs).logits.squeeze(-1).float().item()
chosen_score = reward(chosen_story)
rejected_score = reward(rejected_story)
print(chosen_score > rejected_score)
Trained with TRL reward modeling.