Qwen3-4B-Instruct-2507-GGUF

Qwen3-4B-Instruct-2507-GGUF

Qwen3-4B-Instruct-2507 is a 4-billion-parameter causal language model designed for advanced instruction following, logical reasoning, text comprehension, mathematics, science, coding, and tool usage, with significant improvements in these general capabilities and substantial gains in long-tail knowledge coverage across multiple languages. It operates in a non-thinking mode without generating explicit reasoning step tags, features 36 layers, 32 query and 8 key-value attention heads using GQA, and supports an extremely long native context length of 262,144 tokens.

The model is pretrained and post-trained for enhanced performance and alignment with user preferences, excelling in subjective and open-ended tasks with more helpful and higher-quality responses. It can be deployed efficiently via popular toolkits such as Hugging Face Transformers, SGLang, and vLLM, and supports extensive long-context understanding suitable for complex tasks. Code examples, benchmark evaluations, deployment instructions, and agentic tool-calling capabilities with Qwen-Agent are fully documented in its official repository on Hugging Face.

Model Files

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):