Downloads · 30 days
0
supakorn-emch/typhoon2.5-qwen3-4b-GGUF
typhoon2.5-qwen3-4b-GGUF is a text generation model from supakorn-emch. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
GGUF conversion of typhoon-ai/typhoon2.5-qwen3-4b for use with llama.cpp, Ollama, and other GGUF-compatible inference engines.
Downloads · 30 days
0
Access
Public
Updated Jun 30, 2026
Repo size
—
Likes
0
Public
Click a slice to open those files.
.md2.4 KB · 61%
From the Hugging Face model README
GGUF conversion of typhoon-ai/typhoon2.5-qwen3-4b for use with llama.cpp, Ollama, and other GGUF-compatible inference engines.
| Original Model | typhoon-ai/typhoon2.5-qwen3-4b |
| Architecture | Qwen3 (decoder-only) |
| Parameters | 4B |
| Context Length | 256K tokens |
| License | Apache 2.0 |
| Tokenizer | Qwen-style (<|im_start|>, <|im_end|>, <tool_call>, <tool_response>) |
| Quant | Size | BPW | Notes |
|---|---|---|---|
| Q2_K | ~1.6 GB | 2.96 | Smallest, lower quality |
| Q3_K_S | ~1.8 GB | 3.41 | |
| Q3_K_M | ~1.9 GB | 3.74 | |
| Q3_K_L | ~2.1 GB | 4.03 | |
| Q4_0 | ~2.2 GB | 4.34 | Legacy |
| Q4_1 | ~2.4 GB | 4.78 | Legacy |
| Q4_K_S | ~2.2 GB | 4.37 | |
| Q4_K_M | ~2.3 GB | 4.58 | ⭐ Recommended (balanced) |
| Q5_0 | ~2.6 GB | 5.21 | Legacy |
| Q5_1 | ~2.8 GB | 5.65 | Legacy |
| Q5_K_S | ~2.6 GB | 5.21 | |
| Q5_K_M | ~2.7 GB | 5.33 | |
| Q6_K | ~3.1 GB | 6.14 | Near-perfect quality |
| Q8_0 | ~4.0 GB | 8.50 | Near-FP16 quality |
| IQ3_S | ~1.8 GB | 3.44 | |
| IQ3_M | ~1.8 GB | 3.66 | |
| IQ4_NL | ~2.2 GB | 4.50 | Non-linear |
| IQ4_XS | ~1.7 GB | 4.25 | Non-linear |
./llama-cli -m typhoon2.5-qwen3-4b-Q4_K_M.gguf -p "สวัสดีครับ" -n 256
# Create Modelfile:
FROM ./typhoon2.5-qwen3-4b-Q4_K_M.gguf
# Import:
ollama create typhoon2.5-4b:q4_k_m -f Modelfile
ollama run typhoon2.5-4b:q4_k_m
convert_hf_to_gguf.py from llama.cppllama-quantize