Qwen3-1.7B-Chat-GGUF

Qwen3-1.7B-Chat GGUF

This repository contains GGUF quantizations of Qwen3-1.7B Chat for efficient inference with llama.cpp, Ollama, LM Studio, and other GGUF-compatible applications.

The original model is a Direct Preference Optimized (DPO) version of Qwen3-1.7B-UltraChat-SFT, trained using UltraFeedback, Intel Orca, and Capybara preference datasets to improve conversational quality, helpfulness and alignment.

For complete information about the model, including training methodology, benchmarks, limitations, and usage with Transformers and Unsloth, please refer to the original model card.<br> View the GitHub repository for the complete post-training pipeline.

Available Quantizations

• BF16
• Q8_0
• Q6_K
• Q5_K_M
• Q4_K_M
• Q3_K_L

Ollama

This model is also available on Ollama:

ollama run ayushshah/qwen3-1.7b-chat:q4_k_m

Model page: https://ollama.com/ayushshah/qwen3-1.7b-chat

Use with llama.cpp

CLI:

llama-cli --hf-repo ayushshah/Qwen3-1.7B-Chat-GGUF --hf-file qwen3-1.7b-chat-q8_0.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo ayushshah/Qwen3-1.7B-Chat-GGUF --hf-file qwen3-1.7b-chat-q8_0.gguf -c 32768

For more information about llama.cpp, visit this link. Replace q4_k_m / q8_0 with any other quantization available in this repository.