raat_ir0.7_d5k_0.5mix1.0

raat-ir0.7-d5k-0.5mix1.0

This model has been fine-tuned using the Divide-Then-Align (DTA) method, as proposed in the paper Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAG. The training was conducted on RAAT, a RAFT-based model built upon Llama2-7B-base.

Training Configuration

• training datasize: 5k
• idk_ratio: 0.7
• coe_cls: 0.5
• coe_sft: 1.0

Evaluation

For your reference, we include the evaluation results of this checkpoint on the same test set as in the original paper. Minor differences in metrics were observed.

<table> <thead> <tr> <th rowspan="2">Model</th> <th colspan="1">OQ</th> <th colspan="3">AQ</th> <th colspan="2">RH</th> <th colspan="3">AbQ</th> </tr> <tr> <th>Acc</th> <th>Rec</th> <th>Prec</th> <th>F1</th> <th>DR</th> <th>CUR</th> <th>ARec</th> <th>APrec</th> <th>AF1</th> </tr> </thead> <tbody> <tr> <td>raat-ir0.7-d5k-0.5mix1.0</td> <td>63.5</td><td>70.1</td><td>60.4</td><td>64.9</td><td>75.8</td><td>60.8</td><td>50.7</td><td>73.4</td><td>60.0</td> </tr> </tbody> </table>

Citation

@misc{sun2025dividethenalignhonestalignmentbased,
      title={Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAG}, 
      author={Xin Sun and Jianan Xie and Zhongqi Chen and Qiang Liu and Shu Wu and Yuehe Chen and Bowen Song and Weiqiang Wang and Zilei Wang and Liang Wang},
      year={2025},
      eprint={2505.20871},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.20871}, 
}