Downloads · 30 days
3
1% of all-time downloads
Llama-3.1-8B-SafeRLHF-NBPO-stage2
promotion/Llama-3.1-8B-SafeRLHF-NBPO-stage2
Llama-3.1-8B-SafeRLHF-NBPO-stage2 is a text generation model from promotion. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.1.
pi2 from a genuine second stage of Algorithm 1: the pool regenerated from pi1, the training matrix rejudged in full (44,000 cells), the dual re-solved and the policy refit for 300 updates. Accepted by the Algorithm 1…
All-time downloads
383
Public
Parameters
8B
32.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors32.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama3.1
- Model type
- llama
- Access
- Public
- Created
- Sep 7, 2026
- Updated
- Sep 7, 2026
- SHA
- 1e1ed3f0
Try a prompt
Base models
Model card
From the Hugging Face model README
NBPO on SafeRLHF after a second outer stage
pi_2 from a genuine second stage of Algorithm 1: the pool regenerated from pi_1, the training matrix rejudged in full (44,000 cells), the dual re-solved and the policy refit for 300 updates. Accepted by the Algorithm 1 gate (held-out surpluses 0.240 / 0.148) but it does not improve on pi_1: +0.003 on the gated surplus and -0.008 on the reported worst-objective win rate.
- Panel: SafeRLHF
- Objectives: helpfulness, harmlessness
- Backbone / reference policy:
meta-llama/Llama-3.1-8B-Instruct - Training budget: 300 optimiser updates, global batch 16
- Reported in: Nash Bargaining Preference Optimization (NBPO), Appendix: A Second Outer Stage
Evaluation protocol: independent objective-wise win rate against the common reference,
judged by Llama-3.3-70B-Instruct on prompt-disjoint held-out prompts, both presentation orders.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama3.1
- Created
- Sep 7, 2026
- Updated
- Sep 7, 2026