llama3_synthquestions_dpo_100k

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

This is the model from the paper From Real to Synthetic: Synthesizing Millions of Diversified and Complicated User Instructions with Attributed Grounding.

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

• Model type: Chat Model
• Language(s) (NLP): English
• License: CC-BY-4.0
• Finetuned from model: IgnoraZ/llama3_synthquestions_1m
• Finetuned with data: Preference dataset from IgnoraZ/SynthQuestions

For more details like hyper-parameters, please refer to our paper.

Model Sources

<!-- Provide the basic links for the model. -->

• Repository: https://github.com/Ignoramus0817/SynthQuestions
• Paper: https://www.arxiv.org/abs/2506.03968

How to Get Started with the Model

This is a model in HF format, which can be deployed with common inference frameworks like Transformers, vLLM, SGLang and so on.

We finetuned it with custom chat template instead of the default one from LLaMA. Please make sure to use the chat template in the tokenizer_config.json when inferring.

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Alignment Benchmarks

Citation

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

@misc{zhu2025realsyntheticsynthesizingmillions,
      title={From Real to Synthetic: Synthesizing Millions of Diversified and Complicated User Instructions with Attributed Grounding}, 
      author={Chiwei Zhu and Benfeng Xu and Xiaorui Wang and Zhendong Mao},
      year={2025},
      eprint={2506.03968},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2506.03968}, 
}

Model Card Contact

Please contact [email protected].