Qwen3.5-9B-MixedInt4-AutoRound

Qwen3.5-9B Mixed-INT4 AutoRound

This repository contains a Mixed-INT4 quantized version of Qwen/Qwen3.5-9B, produced using Intel AutoRound.

The model weights were quantized to reduce memory requirements while preserving the original Qwen3.5 architecture, tokenizer, configuration, and multimodal capabilities.

Model Information

• Base model: Qwen/Qwen3.5-9B
• Model type: Multimodal causal language model with vision encoder
• Quantization: Mixed INT4
• Quantization framework: Intel AutoRound
• Weight format: Safetensors
• Native context length: 262,144 tokens
• License: Apache 2.0

Important Notice

This repository is an unofficial community quantization of Qwen3.5-9B.

The model architecture and original model behavior are provided by the Qwen team. Quantization may cause small differences in output quality, numerical precision, generation consistency, and benchmark performance compared with the original model.

No independent benchmark results are currently provided for this quantized version.

evaluation

Evaluation was performed using AutoRound’s evaluation CLI, powered by LM Evaluation Harness.

Serving with vLLM

If the installed vLLM version supports this model architecture and AutoRound quantization format, the model can be served using:

vllm serve YOUR_USERNAME/Qwen3.5-9B-MixedInt4-AutoRound \
  --trust-remote-code

Support for newly released model architectures and quantization formats may require a recent development build of vLLM.

Quantization Details

• Method: Mixed-INT4 AutoRound quantization
• Source weights: Qwen/Qwen3.5-9B
• Fine-tuning: None
• Architecture modifications: None intended

Exact quantization settings, calibration dataset, group size, and AutoRound version should be documented here when available.

Limitations

This model inherits the limitations of the original Qwen3.5-9B model.

Additional limitations may result from quantization:

• Reduced numerical precision
• Small changes in generated responses
• Possible degradation on sensitive reasoning or vision-language tasks
• Runtime compatibility differences between inference frameworks
• Potential differences in long-context behavior

Users should evaluate the model on their own workloads before production use.

Original Model

For complete information about the architecture, supported languages, context length, multimodal usage, benchmarks, intended uses, and limitations, refer to the original model card:

• Qwen/Qwen3.5-9B

License

The original Qwen3.5-9B model is distributed under the Apache License 2.0.

This quantized repository follows the license and usage requirements of the original model. Users are responsible for reviewing and complying with the original license terms.

Credits

• Original model: Qwen
• Quantization framework: Intel AutoRound