Downloads · 30 days
9
5% of all-time downloads
rhbbs/My-upload-test
My-upload-test is a text generation model from rhbbs. Use it when you need the model to write or continue text. It is set up for transformers.
由于目前的LLama2-chat模型很难约束其以中文进行问题回复,因此该模型旨在提供一个能以中文进行问答的LLama2-chat 13B 模型供大家研究使用。
Downloads · 30 days
9
5% of all-time downloads
All-time downloads
194
Public
Repo size
16.5 GB
Likes
0
Public
Click a slice to open those files.
.json36.4 KB · 72%
From the Hugging Face model README
由于目前的LLama2-chat模型很难约束其以中文进行问题回复,因此该模型旨在提供一个能以中文进行问答的LLama2-chat 13B 模型供大家研究使用。
该模型使用LLama2-chat 13B 作为基底模型,使用带embedding 和 LM head 的Lora训练方式训练。模型已完成参数合并,可直接使用。也可以手动将sft_lora_model同Llama2-chat 13B 进行合并。
训练数据使用BELLE项目中采样的50万SFT数据进行SFT训练。
Since the LLama2-chat model struggles to confine its responses to Chinese language when prompted with Chinese questions, the primary objective of this model is to provide a LLama2-chat 13B model that can engage in question and answer interactions in Chinese.
The model utilizes LLama2-chat 13B as its base model and is trained using the Lora training approach with the embedding and LM head. The model has undergone the Lora param merge and is now ready for direct use. It is also possible to manually merge the ./sft_lora_model with the Llama2-chat 13B model to obtain the combined model.
The training data is sampled from BELLE project, which consists of 500,000 SFT samples.
一些训练上的细节:
Some details in training:
该模型依然采用stanford alpaca 模版。因此在测试时且别忘记添加开场白。开场白如下:
"Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n\n${Your Content}\n\n### Response:\n\n"
对于带上文的对话,开场白如下:
"Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n\nHuman:${Previous Human Content}\nAssistant:${Previous Assistance Content}\nHuman:${Your Question}\n\n### Response:\n\n"
This model still using the Stanford Alpaca template. Therefore, don't forget to add prologue template. The prologue template is:
"Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n\n${Your Content}\n\n### Response:\n\n"
For dialogue with context, the prelogue template is:
"Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n\nHuman:${Previous Human Content}\nAssistant:${Previous Machine Content}\nHuman:${Your Question}\n\n### Response:\n\n"
本仓库的模型依照 Apache-2.0 协议开源,模型的权重的使用则需要遵循LLama2MODEL LICENCE。
This repository's models are open-sourced under the Apache-2.0 license, and their weight usage must adhere to LLama2 MODEL LICENCE license.
将会在近期逐步放出
I will release the following models: