Downloads · 30 days
0
chatglm3-6b-32k-int4
npc0/chatglm3-6b-32k-int4
chatglm3-6b-32k-int4 is a machine learning model from npc0. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
介绍 (Introduction) ChatGLM3-6B-32k 是 ChatGLM 系列最新一代的开源模型，THUDM/chatglm3-6b
Access
Public
Updated Dec 24, 2023
Repo size
7.4 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin7.4 GB · 100%
At a glance
- Access
- Public
- Created
- Nov 23, 2023
- Updated
- Dec 24, 2023
- SHA
- 79251347
Model card
From the Hugging Face model README
ChatGLM3-6B-32k-int4
介绍 (Introduction) ChatGLM3-6B-32k 是 ChatGLM 系列最新一代的开源模型，THUDM/chatglm3-6b
用 ChatGLM.CPP 基於 GGML quantize 生成 Q4_0、Q4_1 權重 weights 儲存於此倉庫。
Performance
| Model | GGML quantize method | HDD size |
|---|---|---|
| chatglm3-32k-ggml-q4_0.bin | q4_0 | 3.51 GB |
| chatglm3-32k-ggml-q4_1.bin | q4_1 | 3.9 GB |
Getting Started
- Install dependency
pip install chatglm-cpp transformers
- Download weight
wget https://huggingface.co/npc0/chatglm3-6b-32k-int4/resolve/main/chatglm3-32k-ggml-q4_0.bin
- Code
import chatglm_cpp
pipeline = chatglm_cpp.Pipeline("./chatglm3-32k-ggml-q4_0.bin")
pipeline.chat([chatglm_cpp.ChatMessage(role="user", content="你好")])
# Output: ChatMessage(role="assistant", content="你好👋！我是人工智能助手 ChatGLM-6B，很高兴见到你，欢迎问我任何问题。", tool_calls=[])
- Languages
- zh, en
- Created
- Nov 23, 2023
- Updated
- Dec 24, 2023