Downloads · 30 days
0
HHRLHF_Llama-3.2-1B_aligned_with_MARS_deberta_RM
payelb/HHRLHF_Llama-3.2-1B_aligned_with_MARS_deberta_RM
HHRLHF_Llama-3.2-1B_aligned_with_MARS_deberta_RM is a machine learning model from payelb. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Base model: meta-llama/Llama-3.2-1B-Instruct
Access
Public
Updated Apr 12, 2026
Repo size
62.3 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors45.1 MB · 72%
At a glance
- Access
- Public
- Created
- Apr 12, 2026
- Updated
- Apr 12, 2026
- SHA
- 176c1b6a
Model card
From the Hugging Face model README
payelb/HHRLHF_Llama-3.2-1B_aligned_with_MARS_deberta_RM
Base model: meta-llama/Llama-3.2-1B-Instruct
Alignment dataset: Anthropic/hh-rlhf
Reward model: payelb/HHRLHF_reward-model-deberta-v3-base_1k_fixed_MARS
Method: PPO alignment with LoRA adapters.
Notes:
- Reward normalization and clipping enabled
- KL control enabled
- pad_token_id/eos_token_id explicitly set
- DeBERTa RM loaded on a single device (no device_map='auto')
- Created
- Apr 12, 2026
- Updated
- Apr 12, 2026