Qwen2-7B-hh-rlhf-harmless-base-sft

Model Card for Qwen2-7B Supervised Fine-tuned on HH-RLHF-Harmless-base Dataset

This model is a supervised fine-tuned version of Qwen2-7B, optimized using the HH-RLHF-Harmless-base dataset on completions only.

Model Details

Model Description

• Model type: Transformer-based language model
• Language(s) (NLP): English
• License: apache-2.0
• Finetuned from model: Qwen2-7B

Model Sources

• Repository: https://huggingface.co/Qwen/Qwen2-7B

Training Procedure

Training Data

The model was fine-tuned on the TLDR dataset, which consists of long texts paired with their summarized versions.

• Repository: https://huggingface.co/datasets/trl-internal-testing/tldr-preference-sft-trl-style

Training Hyperparameters

• Optimizer: Adafactor
• Learning Rate: 5e-6 (selected from grid search [5e-4, 1e-4, 5e-5, 1e-5, 5e-6, 1e-6])
• Batch Size: 64 (16 per_device_train_batch_size * 4 devices)
• Epochs: 1 (total 3 epochs)
• Padding: Right-side padding

Training Infrastructure

• Hardware: 4 * NVIDIA H100 GPU

Evaluation Results

This checkpoint represents the model with the lowest evaluation loss during the grid search process.

• Best Evaluation Loss: 1.795 (reference only)
• Achieved at: Epoch 1, Learning Rate 5e-6