Downloads · 30 days
9
11% of all-time downloads
namgyu-youn/Qwen3-8B-W8A16-INT
Qwen3-8B-W8A16-INT is a text generation model from namgyu-youn. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
- Developed by: namgyu-youn - License: apache-2.0 - Quantized from Model: Qwen/Qwen3-8B - Quantization Method: W8A16-INT
Downloads · 30 days
9
11% of all-time downloads
All-time downloads
82
Public
Repo size
9.5 GB
Likes
0
Public
Click a slice to open those files.
.bin9.5 GB · 100%
From the Hugging Face model README
# Perplexity (ppl) command
lm_eval --model hf --model_args pretrained=Qwen/Qwen3-8B --tasks mmlu --device cuda:0 --batch_size 8 --limit 100
# Perplexity (ppl) command
lm_eval --model hf --model_args pretrained=namgyu-youn/Qwen3-8B-W8A16-INT --tasks mmlu --device cuda:0 --batch_size 8 --limit 100
| Benchmark | ||
|---|---|---|
| Qwen/Qwen3-8B | namgyu-youn/Qwen3-8B-W8A16-INT | |
| mmlu | - | - |
vllm bench throughput --model Qwen/Qwen3-8B --input-len 256 --output-len 256 --num-prompts 100
vllm bench throughput --model namgyu-youn/Qwen3-8B-W8A16-INT --input-len 256 --output-len 256 --num-prompts 100
| Benchmark | ||
|---|---|---|
| Qwen/Qwen3-8B | namgyu-youn/Qwen3-8B-W8A16-INT | |
| Throughput (tok/s) | - | - |
vllm bench latency --model Qwen/Qwen3-8B --input-len 256 --output-len 256 --batch-size 1
vllm bench latency --model namgyu-youn/Qwen3-8B-W8A16-INT --input-len 256 --output-len 256 --batch-size 1
| Benchmark | ||
|---|---|---|
| Qwen/Qwen3-8B | namgyu-youn/Qwen3-8B-W8A16-INT | |
| Latency (ms) | - | - |