Downloads · 30 days
71
5% of all-time downloads
becnic/Qwen3-4B-Thinking-2507-Heretic-GGUF
Qwen3-4B-Thinking-2507-Heretic-GGUF is a text generation model from becnic. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
Using <a href="https://github.com/ggerganov/llama.cpp/"llama.cpp</a release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b7120"b7120</a for quantization.
Downloads · 30 days
71
5% of all-time downloads
All-time downloads
1.5K
Public
Repo size
4.3 GB
Likes
0
Public
Click a slice to open those files.
.gguf4.3 GB · 100%
From the Hugging Face model README
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b7120">b7120</a> for quantization.
Original model: https://huggingface.co/becnic/Qwen3-4B-Thinking-2507-Heretic
Run them in LM Studio
Run them directly with llama.cpp, or any other llama.cpp based project
| Filename | Quant type | File Size | Split | Description |
|---|---|---|---|---|
| Qwen3-4B-Thinking-2507-Q8_0.gguf | Q8_0 | 4.28GB | false | Extremely high quality |
First, make sure you have hugginface-cli installed:
pip install -U "huggingface_hub[cli]"
Then, you can target the specific file you want:
huggingface-cli download becnic/Qwen3-4B-Thinking-2507-Heretic-GGUF --include "Qwen3-4B-Thinking-2507-Q8_0.gguf" --local-dir ./
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
huggingface-cli download becnic/Qwen3-4B-Thinking-2507-Heretic-GGUF --include "Qwen3-4B-Thinking-2507-Q8_0.gguf/*" --local-dir ./
</details>
| Parameter | Value |
|---|---|
| direction_index | 19.42 |
| attn.o_proj.max_weight | 1.23 |
| attn.o_proj.max_weight_position | 22.34 |
| attn.o_proj.min_weight | 0.69 |
| attn.o_proj.min_weight_distance | 10.42 |
| mlp.down_proj.max_weight | 1.12 |
| mlp.down_proj.max_weight_position | 29.64 |
| mlp.down_proj.min_weight | 1.08 |
| mlp.down_proj.min_weight_distance | 20.24 |
| Metric | This model | Original model (Qwen/Qwen3-4B-Thinking-2507) |
|---|---|---|
| KL divergence | 0.06 | 0 (by definition) |
| Refusals | 6/100 | 96/100 |
Qwen3-4B-Thinking-2507 has the following features:
NOTE: This model supports only thinking mode. Meanwhile, specifying enable_thinking=True is no longer required.
Additionally, to enforce model thinking, the default chat template automatically includes <think>. Therefore, it is normal for the model's output to contain only </think> without an explicit opening <think> tag.
For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our blog, GitHub, and Documentation.
Supported languages: English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.