vicuna-chinese-replication-beta

This is under a special license, please see the LICENSE file for details. LLaMA is licensed under the LLaMA license, Copyright (c) Meta Platforms, Inc. All Rights Reserved.

-- license: other

Vicuna Chinese Replication Beta

WARNING This is a highly experimental beta model for research only.

With the success of Vicuna which achieves impressive quality with mere 70K finetune data, we would like to do some preliminary experiment to see if how much a similar Chinese dataset can boost the performance of language model.

Data

We use the unfiltered ShareGPT as the English corpus and use two difference ways to translate into Chinese

• Simply use machine translation. The translated data are extremly noisy and contain lots of mistakes, so we only retrain a heavy filtered subset.
• Inspired by baize, we feed the first question to ChatGPT and ask it to translate into Chinese and continue self-chatting. The result is no where near as thoughtful as the real ones, we include them nevertheless.

We also include a small fraction (30k) of CoT data from FLAN and Chinese school math, resulting a total of 150k training data.

Model

We use Chinese-LLaMA-13B as the base model. It is continue-trained from LLaMA on Chinese corpus with no instruction finetune.

We follow exactly the same settings as Vicuna for finetune.

Result

Comparing with Alpaca-like model, the Vicuna replication tends to generate longer and more detailed answers. However it also comes with more severe hallucinations and being US-centric. Please check the examples below.

We try to use ChatGPT to score the answers, however we found ChatGPT has a strong tendency to give high scores to more detailed answer even it contains mistake, making the score unreliable.

Code Examaple

import torch
from transformers import (
    AutoTokenizer,
    AutoModelForCausalLM,
    StoppingCriteriaList,
    StoppingCriteria,
)

class StoppingCriteriaSub(StoppingCriteria):
    '''Checks if the last n tokens in the input_ids list match the stops list.'''
    def __init__(self, stops = []):
        super().__init__()
        self.stops = stops

    def __call__(self, input_ids, scores):
        id_list = input_ids[0].tolist()
        return id_list[-len(self.stops):] == self.stops

def generate_llama(text, max_new_tokens=256):
    '''Generate result using llama model'''
    context = template.format(text)
    input_ids = llama_tokenizer(context, return_tensors="pt").input_ids.to(llama_model.device)
    output_ids = llama_model.generate(input_ids, do_sample=True, top_p=0.8, stopping_criteria=stopping_criteria,
                                      max_new_tokens=max_new_tokens)
    decode_string = llama_tokenizer.batch_decode(output_ids, skip_special_tokens=True)[0]
    return decode_string.replace(context, "").replace("###", "").strip("\n")

if __name__ == "__main__":
    llama_model = AutoModelForCausalLM.from_pretrained("vicuna-chinese-replication").to("cuda:0")
    llama_tokenizer = AutoTokenizer.from_pretrained("vicuna-chinese-replication")
    stopping_criteria = StoppingCriteriaList([StoppingCriteriaSub(stops=[2277, 29937])]) # stop at ###

    # the template is based on Vicuna template question and ChatGPT's answer to it. It probably can be better tuned.
    template = ("###Human: 你好\n###Assistant: 你好！有什么我可以为您做的吗？\n"
                "###Human: 可再生能源和非可再生能源之间的关键区别是什么？\n"
                "###Assistant: 可再生能源是指来自自然资源的能源，这些资源可以持续地再生和利用，如太阳能、风能、水能等。"
                "这些资源不会因为使用而耗尽，不会对环境造成污染和影响。\n\n非可再生能源是指不能再生和持续利用的能源，"
                "如石油、天然气、煤炭等化石燃料。这些资源被消耗后就不能再生，同时使用它们会对环境造成严重污染和气候变化等问题\n\n"
                "###Human: {}\n###Assistant:")
    print(generate_llama("我能用lightning数据线给安卓手机充电吗？"))