Downloads · 30 days
23
2% of all-time downloads
Qwen3-8B-GGUF-Q2KS-AS-AutoRound
Intel/Qwen3-8B-GGUF-Q2KS-AS-AutoRound
Qwen3-8B-GGUF-Q2KS-AS-AutoRound is a machine learning model from Intel. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This example model demonstrates how to use AutoRound’s compatibility to automatically generate a mixed-bit quantization recipe.
All-time downloads
1.1K
Public
Repo size
3.7 GB
Likes
4
Public
Repo makeup
Click a slice to open those files.
.gguf3.7 GB · 100%
At a glance
- Access
- Public
- Created
- Oct 20, 2025
- Updated
- Oct 20, 2025
- SHA
- eb3e977c
Base models
Model card
From the Hugging Face model README
Model Details
This example model demonstrates how to use AutoRound’s compatibility to automatically generate a mixed-bit quantization recipe.
For more details, please refer to user guide
Generate the model
auto-round > 0.8.0
from auto_round import AutoRound, AutoScheme
model_name = "Qwen/Qwen3-8B"
avg_bits = 3.0
scheme = AutoScheme(avg_bits=avg_bits, options=("GGUF:Q2_K_S", "GGUF:Q4_K_S"), ignore_scale_zp_bits=True)
layer_config = {"lm_head": "GGUF:Q6_K"}
ar = AutoRound(model=model_name, scheme=scheme, layer_config=layer_config, iters=0)
ar.quantize_and_save()
- Created
- Oct 20, 2025
- Updated
- Oct 20, 2025