Downloads · 30 days
3
6% of all-time downloads
Qwen3-8B-ORM-MixedPreference
koala1230/Qwen3-8B-ORM-MixedPreference
Qwen3-8B-ORM-MixedPreference is a machine learning model from koala1230. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This repository contains an Outcome Reward Model (ORM) fine-tuned from Qwen/Qwen3-8B, trained on the pairwise preference dataset Skywork/Skywork-Reward-Preference-80K-v0.2 and nvidia/Helpsteer3.
All-time downloads
54
Public
Parameters
312K
15.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors15.1 GB · 100%
At a glance
- Model type
- qwen3
- Access
- Public
- Created
- Jan 7, 2026
- Updated
- Jan 13, 2026
- SHA
- c698127a
Base models
Model card
From the Hugging Face model README
This repository contains an Outcome Reward Model (ORM) fine-tuned from Qwen/Qwen3-8B, trained on the pairwise preference dataset Skywork/Skywork-Reward-Preference-80K-v0.2 and nvidia/Helpsteer3.
Key Details
Base Model: Qwen/Qwen3-8B
Training Dataset: Skywork/Skywork-Reward-Preference-80K-v0.2 and nvidia/Helpsteer3
Loss Function: Bradley-Terry loss
Training Configuration:
Epochs: 1
Optimizer: 2e-6 learning rate
Scheduler: Cosine decay with warmup
Batch Size: 32
Usage
You can load the model using the HuggingFace transformers library:
from transformers import AutoModelForSequenceClassification, AutoTokenizer
# load model and tokenizer
rm = AutoModelForSequenceClassification.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
num_labels=1,
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
Evaluation
Performance on RewardBench 2
| Model | Score | Factuality | Precise IF | Math | Safety | Focus | Ties |
|---|---|---|---|---|---|---|---|
| koala1230/Qwen3-8B-ORM-MixedPreference | 71.4 | 61.1 | 49.4 | 73.2 | 86.4 | 92.9 | 65.5 |
Performance on RM-Bench
| Model | Chat | Math | Code | Safety | Easy | Normal | Hard | Average |
|---|---|---|---|---|---|---|---|---|
| koala1230/Qwen3-8B-ORM-MixedPreference | 66.5 | 62.3 | 59.2 | 93.1 | 90.1 | 75.3 | 45.4 | 70.3 |
License
This model inherits the license of the base model (Qwen3-8B) unless otherwise specified. Please review the original license before use.
- Type
- qwen3
- Created
- Jan 7, 2026
- Updated
- Jan 13, 2026