Downloads · 30 days
18
5% of all-time downloads
internlm-chat-7b-4bit-gptq-safetensor
cczhong/internlm-chat-7b-4bit-gptq-safetensor
internlm-chat-7b-4bit-gptq-safetensor is a feature extraction model from cczhong. Use it when you need embeddings to search or compare text. It is set up for transformers.
please use cczhong/internlm-chat-7b-4bit-gptq before I figure out why it did not work need install "pip install git+https://github.com/cczhong11/AutoGPTQ" before https://github.com/PanQiWei/AutoGPTQ/pull/189 got merged
All-time downloads
371
Public
Repo size
8.2 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.safetensors8.2 GB · 100%
At a glance
- Task
- Feature Extraction
- Library
- transformers
- Model type
- internlm
- Access
- Public
- Created
- Jul 7, 2023
- Updated
- Jul 7, 2023
- SHA
- 1a8db19e
Model card
From the Hugging Face model README
please use cczhong/internlm-chat-7b-4bit-gptq before I figure out why it did not work
how to use
need install "pip install git+https://github.com/cczhong11/AutoGPTQ" before https://github.com/PanQiWei/AutoGPTQ/pull/189 got merged
from transformers import AutoTokenizer, AutoModelForCausalLM
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
quantized_model_dir = "cczhong/internlm-chat-7b-4bit-gptq-safetensor"
tokenizer = AutoTokenizer.from_pretrained(quantized_model_dir, trust_remote_code=True)
model = AutoGPTQForCausalLM.from_quantized(quantized_model_dir, device="cuda:0",trust_remote_code=True)
response, history = model.chat(tokenizer, "你好", history=[])
- Task
- Feature Extraction
- Library
- transformers
- Type
- internlm
- Created
- Jul 7, 2023
- Updated
- Jul 7, 2023