Mistral-7B-LoRA-RAHF-DUAL

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

The model is obtained through alignment training of the mistralai/Mistral-7B-Instruct-v0.2 model using the alignment algorithm mentioned in "Aligning Large Language Models with Human Preferences through Representation Engineering", with UltraFeedback dataset.

You can obtain the training code for RAHF at this link.

A small detail worth noting is that we superpose the representations extracted onto Mistral7B.

• Developed by: Wenhao Liu and Xiaohua Wang
• Funded by [optional]: [More Information Needed]
• Shared by [optional]: [More Information Needed]
• Model type: LoraModel
• Language(s) (NLP): [More Information Needed]
• License: apache-2.0
• Finetuned from model [optional]: mistralai/Mistral-7B-Instruct-v0.2

Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

BibTeX:

@article{liu2023aligning,
  title={Aligning large language models with human preferences through representation engineering},
  author={Liu, Wenhao and Wang, Xiaohua and Wu, Muling and Li, Tianlong and Lv, Changze and Ling, Zixuan and Zhu, Jianhao and Zhang, Cenyuan and Zheng, Xiaoqing and Huang, Xuanjing},
  journal={arXiv preprint arXiv:2312.15997},
  year={2023}
}