pythia-410m-helpful-dpo

Pythia-410m DPO finetuned using original DPO code with the helpful subset of Anthropic-hh-rlhf dataset for 1 epoch.

Checkpoints are also uploaded.

Fully reproducible finetuning code is available on GitHub

wandb log

See Pythia-410m for model details (paper).

See further details of these models in the paper Attributing Mode Collapse in the Fine-Tuning of Large Language Models.

You can cite these models if they are helpful as follows:

<pre> @inproceedings{o2024attributing, title={Attributing Mode Collapse in the Fine-Tuning of Large Language Models}, author={O’Mahony, Laura and Grinsztajn, Leo and Schoelkopf, Hailey and Biderman, Stella}, booktitle={ICLR 2024, Mathematical and Empirical Understanding of Foundation Models (ME-FoMo) workshop}, year={2024} } </pre>

hf (pretrained=lomahony/pythia-410m-helpful-dpo), gen_kwargs: (None), limit: None, num_fewshot: 0, batch_size: 16

hf (pretrained=lomahony/pythia-410m-helpful-dpo), gen_kwargs: (None), limit: None, num_fewshot: 5, batch_size: 16