Downloads · 30 days
4
15% of all-time downloads
Sudhanshu1985/gemma-2-2b-legal-rlaif
gemma-2-2b-legal-rlaif is a machine learning model from Sudhanshu1985. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as gemma.
RLAIF alignment of Sudhanshu1985/gemma-2-2b-legal-sft. A QLoRA reward model (Bradley-Terry, pref-acc 0.80 on gpt-4.1-mini preference pairs) scores on-policy samples; a new QLoRA policy is trained with RLOO policy grad…
Downloads · 30 days
4
15% of all-time downloads
All-time downloads
27
Public
Repo size
117 MB
Likes
0
Public
Click a slice to open those files.
.safetensors83.1 MB · 71%
From the Hugging Face model README
RLAIF alignment of Sudhanshu1985/gemma-2-2b-legal-sft. A QLoRA reward model (Bradley-Terry, pref-acc 0.80 on gpt-4.1-mini preference pairs) scores on-policy samples; a new QLoRA policy is trained with RLOO policy gradient + KL to the merged SFT. Mean reward 3.20 -> 3.36. RL-based counterpart to Sudhanshu1985/gemma-2-2b-legal-dpo. Load on the merged-SFT base (or google/gemma-2-2b + SFT adapter merged) with peft.