SeaLLMs_-_SeaLLM-7B-v1-4bits

Quantization made by Richard Erkhov.

Github

Discord

Request more models

SeaLLM-7B-v1 - bnb 4bits

• Model creator: https://huggingface.co/SeaLLMs/
• Original model: https://huggingface.co/SeaLLMs/SeaLLM-7B-v1/

Original model description:

license: other license_name: seallms license_link: https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b/blob/main/LICENSE extra_gated_prompt: >- You agree to not use the models for any harmful, inappropriate, unethical or illegal purpose or intention. You agree to perform your own red teaming and provide related safety and security measures before deployment for any product relevant to our models and demos, and you must abide by and comply with local governance and regulations. In no event shall the models' authors be held liable for any claim, damages, or other liability arising from the use of the released weights, codes, or demos. The models and demos may be subject to export controls or restrictions in the United States or other countries or regions. You shall comply with applicable laws and regulations in your use of the demos. extra_gated_fields: Company: text Country: text language:

• en
• vi
• id
• ms
• th
• km
• lo
• my
• tl
• zh

---

<p align="center"> <img src="seal_logo.png" width="200" /> </p>

SeaLLMs - Large Language Models for Southeast Asia

<strong style="color: red">UPDATE: <a href="https://huggingface.co/SeaLLMs/SeaLLM-7B-v2.5">SeaLLM-7B-v2.5</a> is released with state-of-the-art performance in world knowledge and reasoning.</strong>

<strong style="color: red">IMPORTANT: <a href="https://huggingface.co/SeaLLMs/SeaLLM-7B-v2">SeaLLM-7B-v2</a> is released with state-of-the-art performance, and this Llama-based SeaLLM-7B-v1 is DEPRECATED. Please use SeaLLM-7B-v2 instead.</strong>

<p align="center"> <a href="https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b" target="_blank" rel="noopener"> 🤗 Tech Memo</a> &nbsp;&nbsp; <a href="https://huggingface.co/spaces/SeaLLMs/SeaLLM-Chat-13b" target="_blank" rel="noopener"> 🤗 DEMO</a> &nbsp;&nbsp; <a href="https://github.com/DAMO-NLP-SG/SeaLLMs" target="_blank" rel="noopener">Github</a> &nbsp;&nbsp; <a href="https://arxiv.org/pdf/2312.00738.pdf" target="_blank" rel="noopener">Technical Report</a> </p>

SeaLLM-7B-v1

This a 1st version of 7B SeaLLMs. It Vietnamese 🇻🇳, Indonesian 🇮🇩, Thai 🇹🇭, Malay 🇲🇾, Khmer 🇰🇭, Lao 🇱🇦, Tagalog 🇵🇭 and Burmese 🇲🇲. It have much lower capability than <a href="https://huggingface.co/SeaLLMs/SeaLLM-7B-v2" target="_blank" rel="noopener">SeaLLM-7B-v2</a>, so please use the SeaLLM-7B-v2 instead.

Visit our <a href="https://arxiv.org/pdf/2312.00738.pdf" target="_blank" rel="noopener">Technical Report</a> and <a href="https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b" target="_blank" rel="noopener"> 🤗 Tech Memo</a> for more details.

<blockquote style="color:red"> <p><strong style="color: red">Terms of Use and License</strong>: By using our released weights, codes, and demos, you agree to and comply with the terms and conditions specified in our <a href="https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b/edit/main/LICENSE" target="_blank" rel="noopener">SeaLLMs Terms Of Use</a>. </blockquote>

Disclaimer: We must note that even though the weights, codes, and demos are released in an open manner, similar to other pre-trained language models, and despite our best efforts in red teaming and safety fine-tuning and enforcement, our models come with potential risks, including but not limited to inaccurate, misleading or potentially harmful generation. Developers and stakeholders should perform their own red teaming and provide related security measures before deployment, and they must abide by and comply with local governance and regulations. In no event shall the authors be held liable for any claim, damages, or other liability arising from the use of the released weights, codes, or demos.

The logo was generated by DALL-E 3.

How to Run:

SeaLLM models work the same way as Llama-2, so the Llama-2 generation codebase should be sufficient to run. However, as this is a chat model, you should wrap the prompt/instruction using the following format function.

You should also turn off add_special_tokens with tokenizer.add_special_tokens = False.

BOS_TOKEN = '<s>'
EOS_TOKEN = '</s>'

B_INST, E_INST = "[INST]", "[/INST]"
B_SYS, E_SYS = "<<SYS>>\n", "\n<</SYS>>\n\n"

SYSTEM_PROMPT = """You are a multilingual, helpful, respectful and honest assistant. \
Please always answer as helpfully as possible, while being safe. Your \
answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure \
that your responses are socially unbiased and positive in nature.

If a question does not make any sense, or is not factually coherent, explain why instead of answering something not \
correct. If you don't know the answer to a question, please don't share false information.

As a multilingual assistant, you must respond and follow instructions in the native language of the user by default, unless told otherwise. \
Your response should adapt to the norms and customs of the respective language and culture.
"""


def chat_multiturn_seq_format(
    message: str,
    history: List[Tuple[str, str]] = None, 
):
    """
    ```
        <bos>[INST] B_SYS SytemPrompt E_SYS Prompt [/INST] Answer <eos>
        <bos>[INST] Prompt [/INST] Answer <eos>
        <bos>[INST] Prompt [/INST]
    ```
    As the format auto-add <bos>, please turn off add_special_tokens with `tokenizer.add_special_tokens = False`
    Inputs:
      message: the current prompt
      history: list of list indicating previous conversation. [[message1, response1], [message2, response2]]
    Outputs:
      full_prompt: the prompt that should go into the chat model

    e.g:
      full_prompt = chat_multiturn_seq_format("Hello world")
      output = model.generate(tokenizer.encode(full_prompt, add_special_tokens=False), ...)
    """
    text = ''
    for i, (prompt, res) in enumerate(history):
        if i == 0:
            text += f"{bos_token}{B_INST} {B_SYS} {sys_prompt} {E_SYS} {prompt} {E_INST}"
        else:
            text += f"{bos_token}{B_INST} {prompt}{end_instr}"
        if res is not None:
            text += f" {res} {eos_token} "
    if len(history) == 0 or text.strip() == '':
        text = f"{bos_token}{B_INST} {B_SYS} {sys_prompt} {E_SYS} {message} {E_INST}"
    else:
        text += f"{bos_token}{B_INST} {message} {E_INST}"
    return text

Citation

If you find our project useful, we hope you would kindly star our repo and cite our work as follows: Corresponding Author: [email protected]

@article{damonlpsg2023seallm,
  author = {Xuan-Phi Nguyen*, Wenxuan Zhang*, Xin Li*, Mahani Aljunied*,
            Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang,
            Chaoqun Liu, Hang Zhang, Lidong Bing},
  title = {SeaLLMs - Large Language Models for Southeast Asia},
  year = 2023,
  Eprint = {arXiv:2312.00738},
}