Downloads · 30 days
137
10% of all-time downloads
Yi-1.5-9B-Chat-AWQ
modelscope/Yi-1.5-9B-Chat-AWQ
Yi-1.5-9B-Chat-AWQ is a text generation model from modelscope. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
language: - en - zh pipelinetag: text-generation tags: - Awq - int4 - yi1.5-9B-Chat - pytorch license: apache-2.0 licensename: apache-2.0 licenselink: LICENSE ---
All-time downloads
1.4K
Public
Parameters
8.8B
5.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.4 GB · 100%
Parameter types
How the weights are stored.
I328.3B · 94%
Try a prompt
Model card
From the Hugging Face model README
language:
- en
- zh pipeline_tag: text-generation tags:
- Awq
- int4
- yi1.5-9B-Chat
- pytorch license: apache-2.0 license_name: apache-2.0 license_link: LICENSE
About Quantization
我们使用modelscope swift仓库进行AWQ 4bit量化. 量化文档可以查看这里. 量化命令如下:
We use the modelscope swift repository to perform AWQ 4bit quantization. Quantization documentation can be found here. The quantization command is as follows:
# Experimental Environment: A100
swift export \
--quant_bits 4 \
--model_type yi-1_5-9b-chat \
--quant_method awq \
--quant_n_samples 64 \
--dataset alpaca-zh alpaca-en sharegpt-gpt4-mini \
--quant_seqlen 4096
Inference:
CUDA_VISIBLE_DEVICES=0 swift infer --model_type yi-1_5-9b-chat-awq-int4
SFT:
CUDA_VISIBLE_DEVICES=0 swift sft --model_type yi-1_5-9b-chat-awq-int4 --dataset leetcode-python-en
Original Model:
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Languages
- en, zh
- Created
- May 13, 2024
- Updated
- May 13, 2024