drc-rag-llm-v1-tech-report

drc-8/drc-rag-llm-v1-tech-report

drc-rag-llm-v1-tech-report is a machine learning model from drc-8. Use it for the machine learning task on the model card, and read the license before you ship it in a product.

Downloads · 30 days

Access

Public

Updated Feb 11, 2025

Repo size

812 KB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.pdf812 KB · 99%

At a glance

Access: Public
Created: Feb 11, 2025
Updated: Feb 11, 2025
SHA: 75d09a9e

Model card

From the Hugging Face model README

DRC RAG LLM v1

📄 Technical_Report

DRC-RAG-LLM is a Retrieval-Augmented Generation (RAG) scenario–optimized LLM based on Qwen1.5. We developed two RAG scenarios tasks—Retrieve Chunk Citation and Chinese Textual Table Understanding, then fine-tuned DRC-RAG-LLM on these tasks while preserving most of the general language capabilities of the base model. In addition, DRC-RAG-LLM demonstrates strong performance compared to ChatGPT and GPT-4o on two RAG scenarios tasks.

Retrieve Chunk Citation

Model	Precision	Recall	F1
Fine-tuned
DRC-RAG-LLM-7B	73.61	90.24	81.08
DRC-RAG-LLM-14B	79.55	91.71	85.20
Non-Fine-tuned
Breeze-7B-Instruct-v1_0	28.53	14.47	19.2
Llama3-TAIDE-LX-8B-Chat	39.74	32.03	30.39
Llama-3-Taiwan-8B-Instruct	29.89	10.08	16.08
Qwen1.5-7B-Chat	22.12	7.48	11.18
Qwen1.5-14B-Chat	36.82	15.45	21.76
OpenAI Models
ChatGPT	60.00	52.68	56.1
GPT-4o	62.02	66.33	64.1

Chinese Textual Table Understanding

Model	Table QA	Table summarization
Fine-tuned
DRC-RAG-LLM-7B	66.6	56.6
DRC-RAG-LLM-14B	75.6	61.0
Non-Fine-tuned
Breeze-7B-Instruct-v1_0	59.4	55.6
Llama3-TAIDE-LX-8B-Chat	55.6	48.2
Llama-3-Taiwan-8B-Instruct	63.8	38.6
Qwen1.5-7B-Chat	54.2	41.4
Qwen1.5-14B-Chat	62.4	55.4
OpenAI Models
ChatGPT	70.0	48.2
GPT-4o	82.6	85.9

Created: Feb 11, 2025
Updated: Feb 11, 2025