Downloads · 30 days
26
11% of all-time downloads
Qwen3.6-35B-A3B-MTP-GGUF
saidonnet/Qwen3.6-35B-A3B-MTP-GGUF
Qwen3.6-35B-A3B-MTP-GGUF is a machine learning model from saidonnet. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Quantized GGUF versions of Qwen3.6-35B-A3B-MTP (Apache MoE) for efficient local inference.
All-time downloads
244
Public
Repo size
40.6 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf40.6 GB · 100%
At a glance
- Access
- Public
- Created
- Jul 25, 2026
- Updated
- Jul 25, 2026
- SHA
- f73de0e2
Model card
From the Hugging Face model README
Qwen3.6-35B-A3B-MTP-GGUF (Quantized)
Overview
Quantized GGUF versions of Qwen3.6-35B-A3B-MTP (Apache MoE) for efficient local inference.
Source
- Original: ggml-org/Qwen3.6-35B-A3B-MTP-GGUF
- Base Model: Qwen3.6-35B-A3B (Apache MoE, 35B parameters)
- BF16 Source: 67GB downloaded from ggml-org (official conversion, 1200+ downloads)
Available Quantizations
| File | Size | Quantization | Quality |
|---|---|---|---|
| Qwen3.6-35B-A3B-MTP-Q4_0.gguf | 19GB | Q4_0 | Good |
| Qwen3.6-35B-A3B-MTP-Q4_K_S.gguf | 19GB | Q4_K_S | Better |
Quantization Details
- Date: 2026-07-25
- Tool: llama.cpp (commit b1-fb92d8f)
- VM: Lightning.ai (16 cores, 60GB RAM)
- Commands:
Usage
llama-server
llama-cli
Features
- MTP Support: MoE Targeted Prediction for faster inference
- A3B Architecture: Apache 3-Branch MoE design
- 35B Parameters: Full model capacity in quantized form
- GGUF Format: Compatible with llama.cpp, llama-server, llama-cli
License
Same as original: Apache 2.0
Credits
- Original model: Qwen team
- GGUF conversion: ggml-org
- Quantization: saidonnet (2026-07-25)
- Created
- Jul 25, 2026
- Updated
- Jul 25, 2026