Downloads · 30 days
15
2% of all-time downloads
Jennny/llama3_8b_honest_rm_full
llama3_8b_honest_rm_full is a text classification model from Jennny. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as cc-by-nc-4.0.
This model is a fine-tuned version of Jennny/llama38bsftultrafb and serves as the Honesty Process Reward Model (PRM) introduced in the paper Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable…
Downloads · 30 days
15
2% of all-time downloads
All-time downloads
924
Public
Parameters
7.5B
30 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors15 GB · 100%
From the Hugging Face model README
This model is a fine-tuned version of Jennny/llama3_8b_sft_ultrafb and serves as the Honesty Process Reward Model (PRM) introduced in the paper Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable Rewards.
It is part of the MAHALO (Multi-Action-Head ALignment with PRM-guided DecOding) framework, which standardizes PRM training across verifiable and non-verifiable settings for step-level supervision. This specific model provides reward signals for the honesty dimension in open-domain conversations.
This model uses the LlamaForSequenceClassification architecture to evaluate the honesty of steps in a dialogue. It was trained as part of a multi-objective alignment setup to allow for controllable inference and improved human value alignment.
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 0.3012 | 0.4632 | 50 | 0.2989 | 0.886 |
| 0.2851 | 0.9265 | 100 | 0.2657 | 0.908 |
If you find this work useful, please cite:
@article{shen2025simultaneous,
title={Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable Rewards},
author={Shen, Yiran and Xia, Yu and Chang, Jonathan and Ammanabrolu, Prithviraj},
journal={arXiv preprint arXiv:2510.01167},
year={2025},
url={https://arxiv.org/abs/2510.01167}
}