Downloads · 30 days
6
11% of all-time downloads
abhishekai/slm-500m-legal-rm
slm-500m-legal-rm is a text classification model from abhishekai. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.
A 517.8M-parameter scalar reward model for grounded legal/financial Q&A, trained on AI-generated preferences (RLAIF). This is the reward signal used to train slm-500m-legal-ppo.
Downloads · 30 days
6
11% of all-time downloads
All-time downloads
54
Public
Parameters
518M
2.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors2.1 GB · 100%
From the Hugging Face model README
A 517.8M-parameter scalar reward model for grounded legal/financial Q&A, trained on AI-generated preferences (RLAIF). This is the reward signal used to train slm-500m-legal-ppo.
This is not a chat model. It emits a single scalar per sequence and cannot
generate text. Load it with AutoModelForSequenceClassification, not
AutoModelForCausalLM.
Given a rendered prompt concatenated with a candidate answer, it returns one number. Higher means "closer to what Gemini 2.5 Flash preferred." Only differences between scores are meaningful — the Bradley-Terry objective it was trained with fixes score gaps, not absolute magnitude, so the raw value is arbitrary and not comparable across reward models.
| pairwise accuracy on held-out preference pairs | 0.820 |
| architecture | SFT backbone + fresh scalar head (num_labels=1) |
| training objective | -log σ(r(prompt+chosen) − r(prompt+rejected)) |
| preference pairs | 994 triplets (894 train / 100 eval) |
Every generating checkpoint in this project was re-scored by a Claude Sonnet judge on
four axes out of 10 over a shared set of 300 held-out prompts. A reward model has no
cell in that table: it emits a scalar preference score rather than an answer, so there
is nothing for an answer-quality rubric to grade. Its pairwise accuracy above remains the
relevant measure. The full table is in MODEL_INDEX.md in the project repository.
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
name = "abhishekai/slm-500m-legal-rm"
tok = AutoTokenizer.from_pretrained(name)
rm = AutoModelForSequenceClassification.from_pretrained(name, num_labels=1).eval()
rm.config.pad_token_id = tok.convert_tokens_to_ids("<|pad|>")
def score(question, context, answer):
msgs = [
{"role": "system", "content": "You are a precise legal and financial assistant. Answer only from the provided context."},
{"role": "user", "content": f"Context: {context}\n\nQuestion: {question}"},
]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
enc = tok(prompt + answer.strip() + "<|eos|>", add_special_tokens=False,
return_tensors="pt", truncation=True, max_length=1024)
# input_ids/attention_mask only — the tokenizer also emits token_type_ids,
# which LlamaForSequenceClassification.forward() rejects.
with torch.no_grad():
return rm(input_ids=enc["input_ids"],
attention_mask=enc["attention_mask"]).logits.squeeze().item()
The prompt must be rendered exactly as above. The reward model was trained on this template; scoring a differently-formatted string puts it out of distribution and the number becomes meaningless without any error being raised.
994 preference triplets built on-policy from the frozen SFT model: