ChatBELLE-int4

Model Card for ChatBELLE-int4

Welcome

4-bit quantized model using llama.cpp. If you find this model helpful, please like this model and star us on https://github.com/LianjiaTech/BELLE !

Model description

ChatBELLE-int4 is based on 7B model and quantized to 4-bit.

The code of Chinese data generation and other detailed information can be found in our Github project repository: https://github.com/LianjiaTech/BELLE.

Download

Should you accept our license and acknowledged the limitations, download the model by clicking Download.

Model Usage

You can use this model with ChatBELLE, a minimal, cross-platform LLM chat app powered by BELLE using quantized on-device offline models and Flutter UI, running on macOS (done), Windows, Android, iOS(see Known Issues) and more.

macOS

• Download chatbelle.dmg from Releases page, double click to open it, then drag Chat Belle.dmg into Applications folder.
• Open the Chat Belle app in Applications folder by right click then Ctrl-click Open, then click Open.
• The app will prompt the intended model file path and fail to load the model. Close the app.
• Download quantized model belle-model.bin from this repo.
• Move and rename the model to the path prompted by the app. Defaults to ~/Library/Containers/com.barius.chatbelle/Data/belle-model.bin .
• Reopen the app again (double clicking is now OK).

Windows

• Stay tuned

Android

• Stay tuned

iOS

• Stay tuned

Limitations

There still exists a few issues in the model trained on current base model and data:

1. The model might generate factual errors when asked to follow instructions related to facts.

2. Occasionally generates harmful responses since the model still struggles to identify potential harmful instructions.

3. Needs improvements on reasoning and coding.

Since the model still has its limitations, we require developers only use the open-sourced code, data, model and any other artifacts generated via this project for research purposes. Commercial use and other potential harmful use cases are not allowed.

Citation

Please cite us when using our code, data or model.

@misc{BELLE,
  author = {Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Baochang Ma, Xiangang Li},
  title = {BELLE: Be Everyone's Large Language model Engine},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/LianjiaTech/BELLE}},
}