MiniCPM3-RAG-LoRA

MiniCPM3-RAG-LoRA

MiniCPM3-RAG-LoRA 由面壁智能、东北大学信息检索小组（NEUIR）和清华大学自然语言处理实验室（THUNLP）和共同开发，是一个专门面向检索增强生成（RAG）场景的生成模型。它在 MiniCPM3 的基础上，采用低秩适应（LoRA）技术，通过直接偏好优化（DPO）方法进行微调，仅基于两万余条开放域问答和逻辑推理任务的开源数据，在通用评测数据集上实现了模型性能平均提升约 13%。

欢迎关注 MiniCPM3 与 RAG 套件系列：

• 基座模型：MiniCPM3
• 检索模型：MiniCPM-Embedding
• 重排模型：MiniCPM-Reranker
• 面向 RAG 场景的 LoRA 插件：MiniCPM3-RAG-LoRA

MiniCPM3-RAG-LoRA developed by ModelBest Inc., NEUIR and THUNLP, is a generative model specifically designed for Retrieval-Augmented Generation (RAG) scenarios. Based on MiniCPM3, the model is fine-tuned using the Low-Rank Adaptation (LoRA) technique through Direct Preference Optimization (DPO). The fine-tuning process is based on over 20,000 open-source data points from open-domain question answering and logical reasoning tasks, leading to an average performance improvement of approximately 13% on general evaluation datasets.

We also invite you to explore MiniCPM3 and the RAG toolkit series:

• Foundation Model: MiniCPM3
• Retrieval (Embedding) Model: MiniCPM-Embedding
• Re-ranking Model: MiniCPM-Reranker
• LoRA Plugin for RAG scenarios: MiniCPM3-RAG-LoRA

模型信息 Model Information

• 模型大小：4B
• 最大输入token数：32768
• Model Size: 4B
• Max Input Tokens: 32768

模型使用 Usage

输入格式 Input Format

MiniCPM3-RAG-LoRA 模型遵循格式如下：

MiniCPM3-RAG-LoRA supports instructions in the following format:

Passages = "In the novel 'The Silent Watcher,' the lead character is named Alex Carter. Alex is a private detective who uncovers a series of mysterious events in a small town.\nSet in a quiet town, 'The Silent Watcher' follows Alex Carter, a former police officer turned private investigator, as he unravels the town's dark secrets.\n'The Silent Watcher' revolves around Alex Carter's journey as he confronts his past while solving complex cases in his hometown.",

Instruction = "Q: What is the name of the lead character in the novel 'The Silent Watcher'?\nA:" 

Input = 'Background:\n'+ Passages + '\n\n' + Instruction

环境要求 Requirements

transformers>=4.36.0

示例脚本 Demo

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
torch.manual_seed(0)

path = 'openbmb/MiniCPM3-RAG-LoRA'
tokenizer = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(path, torch_dtype=torch.bfloat16, device_map='cuda', trust_remote_code=True)

passages_list = ["In the novel 'The Silent Watcher,' the lead character is named Alex Carter. Alex is a private detective who uncovers a series of mysterious events in a small town.",
"Set in a quiet town, 'The Silent Watcher' follows Alex Carter, a former police officer turned private investigator, as he unravels the town's dark secrets.",
"'The Silent Watcher' revolves around Alex Carter's journey as he confronts his past while solving complex cases in his hometown."]
instruction = "Q: What is the name of the lead character in the novel 'The Silent Watcher'?\nA:"

passages = '\n'.join(passages_list)
input_text = 'Background:\n' + passages + '\n\n' + instruction

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": input_text},
]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)

outputs = model.chat(tokenizer, prompt, temperature=0.8, top_p=0.8)
print(outputs[0])  # The lead character in the novel 'The Silent Watcher' is named Alex Carter.

实验结果 Evaluation Results

经过针对RAG场景的LoRA训练后，MiniCPM3-RAG-LoRA在开放域问答（NQ、TQA、MARCO）、多跳问答（HotpotQA）、对话（WoW）、事实核查（FEVER）和信息填充（T-REx）等多项任务上的性能表现，超越Llama3-8B和Baichuan2-13B等业内优秀模型。

After being fine-tuned with LoRA for RAG scenarios, MiniCPM3-RAG-LoRA outperforms leading industry models like Llama3-8B and Baichuan2-13B across various tasks, including open-domain question answering (NQ, TQA, MARCO), multi-hop question answering (HotpotQA), dialogue (WoW), fact checking (FEVER), and information filling (T-REx).

许可证 License

• 本仓库中代码依照 Apache-2.0 协议开源。
• MiniCPM3-RAG-LoRA 模型权重的使用则需要遵循 MiniCPM 模型协议。
• MiniCPM3-RAG-LoRA 模型权重对学术研究完全开放。如需将模型用于商业用途，请填写此问卷。

• The code in this repo is released under the Apache-2.0 License.
• The usage of MiniCPM3-RAG-LoRA model weights must strictly follow MiniCPM Model License.md.
• The models and weights of MiniCPM3-RAG-LoRA are completely free for academic research. After filling out a "questionnaire" for registration, MiniCPM3-RAG-LoRA weights are also available for free commercial use.