Qwen3-1.7B-AWQ

flin775/Qwen3-1.7B-AWQ

Qwen3-1.7B-AWQ is a machine learning model from flin775. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.

This is the AWQ(4bit) quantization model of Qwen3-1.7B, which is created by AutoAwq w/ the following config, featuring very low GPU memory usage, high throughput, and extremely fast response speed

Downloads · 30 days

0% of all-time downloads

All-time downloads

4.1K

Public

Parameters

1.7B

2.9 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors1.5 GB · 99%

Parameter types

How the weights are stored.

I321.4B · 82%

Base models

Qwen/Qwen3-1.7B

Model card

From the Hugging Face model README

This is the AWQ(4bit) quantization model of Qwen3-1.7B, which is created by AutoAwq w/ the following config, featuring very low GPU memory usage, high throughput, and extremely fast response speed

{
  "zero_point": True,
  "q_group_size": 16,
  "w_bit": 4,
  "version": "GEMM"
}

P.S. g_group_size is set as 16 for high accuracy.

LLM serving

This model does not work with vllm, as the minimum g_group_size required by vllm is 32. please check the model 'Qwen3-1.7B-AWQ-Group32' for VLLM.
It is recommended to serve the model with lmdeploy, works very very well.

It is recommended to install lmdeploy from git instead of the pip official repo, as the version of 0.8.0 in pip official repo has problem work with the latest transformers lib.
```
pip install git+https://github.com/InternLM/lmdeploy
```

Type: qwen3
License: apache-2.0
Created: Jun 11, 2025
Updated: Jun 14, 2025