llama-3.2-1b-riskkd-tokenwt-epoch1

Llama-3.2-1B — riskKD (token-weighted Ra-DPO), epoch 1

1B student from meta-llama/Llama-3.2-1B, SFT-initialized on HuggingFaceH4/deita-10k-v0-sft, then trained with the riskKD objective (TVKD + risk-aware DPO with kl_inv per-token weighting, alpha=1.0) against an 8B DPO-trained teacher as frozen reference, on pvdhihihi/ultra-feedback.

Hyperparameters: beta=0.1, lr=5e-7 constant, max_length=512, max_prompt_length=256, radpo_weight=1.0, radpo_alpha=0.5, radpo_confidence_level=0.99, radpo_token_weight_mode=kl_inv, radpo_token_weight_alpha=1.0. This is the epoch-1 checkpoint (step 902) of a 3-epoch run; epoch 1 was the best on the lm-eval suite (avg 43.02% over MMLU/TruthfulQA-MC2/Winogrande/HellaSwag/GSM8K/ARC-C).