Downloads · 30 days
544
17% of all-time downloads
nachikethreddyy/qwen3.5-8b-distilled-GGUF
qwen3.5-8b-distilled-GGUF is a text generation model from nachikethreddyy. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Fine-tuned Qwen3.5-8B for software engineering & coding tasks. GGUF-optimized version for local inference.
Downloads · 30 days
544
17% of all-time downloads
All-time downloads
3.2K
Public
Repo size
121 GB
Likes
0
Public
Click a slice to open those files.
.gguf29.8 GB · 100%
From the Hugging Face model README
Fine-tuned Qwen3.5-8B for software engineering & coding tasks. GGUF-optimized version for local inference.
| Variant | Size | Format | Best For |
|---|---|---|---|
| Full Precision (BF16) | 16.39 GB | Safetensors | Maximum quality, research |
| Q8 Quantized | 8.8 GB | Safetensors | Balanced speed/quality |
| GGUF F16 | 15.3 GB | GGUF | Ollama, llama.cpp, LM Studio |
ollama run nachikethreddyy/qwen3.5-8b-distilled-GGUF:F16
# Install
brew install llama.cpp
# Run
llama-cli -hf nachikethreddyy/qwen3.5-8b-distilled-GGUF:F16
nachikethreddyy/qwen3.5-8b-distilled-GGUFfrom transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"nachikethreddyy/qwen3.5-8b-distilled-GGUF",
device_map="auto"
)
Apache 2.0 (inherited from Qwen/Qwen3-8B)
For MLX/Apple Silicon: See qwen3.5-8b-distilled-MLX