Downloads · 30 days
0
baichuan2-13b-chat-fastllm
omer-w/baichuan2-13b-chat-fastllm
baichuan2-13b-chat-fastllm is a machine learning model from omer-w. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as bsd.
Converted INT8/INT4 files for fastllm with baichuan2-13b-chat
Access
Public
Updated Sep 12, 2023
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
Other1.5 KB · 61%
At a glance
- License
- bsd
- Access
- Public
- Created
- Sep 12, 2023
- Updated
- Sep 12, 2023
- SHA
- aed79143
Model card
From the Hugging Face model README
Converted INT8/INT4 files for fastllm with baichuan2-13b-chat
Directly download from Baidu Netdisk:
Link：https://pan.baidu.com/s/1Xsiif_1VzDyWFei1u5oJcA Code：wxbo
Updated time: 2023/09/11
baichuan2-13b-chat-int8.flm:
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:05:00.0 Off | N/A |
| 31% 36C P8 28W / 250W | 15420MiB / 22528MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
from fastllm_pytools import llm
model = llm.model("baichuan2-13b-chat-int8.flm")
for response in model.stream_response("介绍一下南京"):
print(response, flush = True, end = "")
Note: please use the lastest version of FastLLM (no later than 2023/09/11 main branch)
- License
- bsd
- Created
- Sep 12, 2023
- Updated
- Sep 12, 2023