community-loss-mask-1.5b

community-loss-mask-1.5b

Recipe: recipes/community/what-trl-does-with-the-loss-mask · Collection: Course and community runs

What TRL does with the export's loss_mask: nothing, unless you apply it. Two SFT arms on the same file. as_exported supervises every token (46,480 of 46,480); mask_honored supervises 5,168 of them, the assistant turns only.

Result, from the recipe README

The pre-registered primary did not move: neither arm leaked non-assistant text. What moved is reply length, in the direction the mask predicts. results.json is the full comparison.

Arms in this repo

The root holds the arm the recipe README's headline number reports. Every other arm is a subfolder named after it. checkpoints/ never ships.

Load

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "while-ai/community-loss-mask-1.5b")  # the headline arm
model = PeftModel.from_pretrained(base, "while-ai/community-loss-mask-1.5b", subfolder="as_exported")  # another arm

Reproduce

git clone https://github.com/whilehq/whileai-sdk && cd whileai-sdk/recipes/community/what-trl-does-with-the-loss-mask
python run.py && modal run train_modal.py

The recipe README pins the seed, the library versions and the GPU, and its Checks table says what the eval verified. Read the Learned section before quoting a number from this card.