Downloads · 30 days
0
baichuan-13b-chat-fastllm
omer-w/baichuan-13b-chat-fastllm
baichuan-13b-chat-fastllm is a machine learning model from omer-w. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as bsd.
Converted INT8/INT4 files for fastllm with baichuan-13b-chat
Access
Public
Updated Sep 12, 2023
Repo size
—
Likes
2
Public
Repo makeup
Click a slice to open those files.
Other1.5 KB · 55%
At a glance
- License
- bsd
- Access
- Public
- Created
- Jul 28, 2023
- Updated
- Sep 12, 2023
- SHA
- 16158bb2
Model card
From the Hugging Face model README
Converted INT8/INT4 files for fastllm with baichuan-13b-chat
Directly download from Baidu Netdisk:
Link：https://pan.baidu.com/s/1zADu6rd749zkkNAfl-aqtg Code：jqkl
Updated time: 2023/07/27
baichuan-13b-chat-int4.flm:
+-----------------------------------------------------------------------------+
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| N/A 53C P0 56W / 250W | 7083MiB / 23040MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
baichuan-13b-chat-int8.flm:
+-----------------------------------------------------------------------------+
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| N/A 51C P0 162W / 250W | 13151MiB / 23040MiB | 95% Default |
+-------------------------------+----------------------+----------------------+
from fastllm_pytools import llm
model = llm.model("baichuan-13b-chat-int4.flm")
for response in model.stream_response("介绍一下南京"):
print(response, flush = True, end = "")
- License
- bsd
- Created
- Jul 28, 2023
- Updated
- Sep 12, 2023