Downloads · 30 days
0
Qwen1.5-7B-Chat-750Mb-lora
REILX/Qwen1.5-7B-Chat-750Mb-lora
Qwen1.5-7B-Chat-750Mb-lora is a machine learning model from REILX. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
使用以下8个数据集 对Qwen1.5-7B-Chat进行微调并测试，结果显示，微调后的模型在CEVAL和MMLU的评分上均有所提升。
Access
Public
Updated Oct 24, 2024
Repo size
40 MB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors40 MB · 92%
At a glance
- License
- other
- Access
- Public
- Created
- May 9, 2024
- Updated
- Oct 24, 2024
- SHA
- d7a39f70
Model card
From the Hugging Face model README
数据集
使用以下8个数据集
对Qwen1.5-7B-Chat进行微调并测试，结果显示，微调后的模型在CEVAL和MMLU的评分上均有所提升。
基础模型：
训练工具
https://github.com/hiyouga/LLaMA-Factory
测评方式：
使用opencompass(https://github.com/open-compass/OpenCompass/ )， 测试工具基于CEval和MMLU对微调之后的模型和原始模型进行测试。</br> 测试模型分别为：
- Qwen1.5-7B-Chat
- Qwen1.5-7B-Chat-750Mb-lora,使用8DataSets数据集对Qwen1.5-7B-Chat模型进行sft方式lora微调
8DataSets数据集：
大约750Mb的微调数据集
- https://huggingface.co/datasets/REILX/extracted_tagengo_gpt4
- https://huggingface.co/datasets/TigerResearch/sft_zh
- https://huggingface.co/datasets/silk-road/alpaca-data-gpt4-chinese
- https://huggingface.co/datasets/LooksJuicy/ruozhiba
- https://huggingface.co/datasets/microsoft/orca-math-word-problems-200k
- https://huggingface.co/datasets/alexl83/AlpacaDataCleaned
- https://huggingface.co/datasets/Sao10K/Claude-3-Opus-Instruct-5K
结果
| 模型名称 | CEVAL | MMLU |
|---|---|---|
| Qwen1.5-7B-Chat | 68.61 | 61.56 |
| Qwen1.5-7B-Chat-750Mb-lora | 71.36 | 61.78 |
License
This project utilizes certain datasets and checkpoints that are subject to their respective original licenses. Users must comply with all terms and conditions of these original licenses. The content of this project itself is licensed under the Apache license 2.0.
- License
- other
- Languages
- zh, en
- Created
- May 9, 2024
- Updated
- Oct 24, 2024