Qwen3-1.7B-Depth-Aggressive-Q4_K_M-GGUF

Qwen3-1.7B Depth-Aggressive — Q4_K_M GGUF

A structured depth-pruned and recovery-trained variant of Qwen/Qwen3-1.7B, converted to GGUF and quantized as Q4_K_M for llama.cpp-compatible runtimes.

Model changes

• Transformer layers: 28 → 24
• Removed source layers: 9, 14, 19, 7
• Parameters: approximately 1.721B → 1.519B
• Parameters retained: 88.30%
• Recovery: 100 chat-recovery optimizer steps
• Recovery objective: assistant-only cross-entropy plus teacher distillation
• GGUF quantization: Q4_K_M

File

llama.cpp

llama-cli -m Qwen3-1.7B-Depth-Aggressive-Q4_K_M.gguf -cnv -ngl 99

Notes

The pruning experiment showed the aggressive depth-pruned 1.7B checkpoint at 24 layers and about 1.519B parameters before GGUF quantization. This repository contains the quantized GGUF build, not the full-precision Transformers weights.

License

The base model is distributed under the Apache 2.0 license. Review the original Qwen model repository for its complete terms and documentation.