llm-course-hw2-reward-model

🏆 Model Card for llm-course-hw2-reward-model

This model is a fine-tuned reward model based on HuggingFaceTB/SmolLM-135M-Instruct, trained on the HumanLLMs/Human-Like-DPO-Dataset dataset.
It has been trained using TRL to evaluate and rank responses based on human preferences, playing a crucial role in RLHF (Reinforcement Learning from Human Feedback) for models like SmolLM-135M-PPO.

---

📝 Overview

• Base Model: SmolLM-135M-Instruct
• Fine-Tuned Dataset: HumanLLMs/Human-Like-DPO-Dataset
• Objective: Learn to assign higher scores to more engaging, structured, and emotional responses.
• Use Case: Used in PPO-based RLHF training to reinforce human-like response quality.

Training Method

• The model was fine-tuned using Direct Preference Comparisons:
  • Each sample contains a chosen response (preferred) and a rejected response.
  • The model learns to assign higher rewards to the chosen response and lower rewards to the rejected one.
• This reward function was used in PPO fine-tuning to optimize response generation.

---