Downloads · 30 days
27
10% of all-time downloads
Qwen3-4B-Instruct-2507-heretic-GGUF-BF16
Auguments/Qwen3-4B-Instruct-2507-heretic-GGUF-BF16
Qwen3-4B-Instruct-2507-heretic-GGUF-BF16 is a machine learning model from Auguments. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
Am I blind, or are there no BF16 GGUFs of this? I don't trust quants if I don't know if they were made from BF16 or FP16.
All-time downloads
278
Public
Repo size
10.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf10.7 GB · 100%
At a glance
- Library
- gguf
- License
- apache-2.0
- Access
- Public
- Created
- Aug 31, 2026
- Updated
- Aug 31, 2026
- SHA
- 3f812d5b
Base models
Model card
From the Hugging Face model README
heretic-org/Qwen3-4B-Instruct-2507-heretic
Am I blind, or are there no BF16 GGUFs of this? I don't trust quants if I don't know if they were made from BF16 or FP16.
Highest quality conversion - BF16.
In this repo:
- Base model in BF16, and a custom Q4_K_M converted from BF16
Q4_K_M_custom retains more weights in Q8 or Q6_K, specifically those where Qwen3 is most prone to quality loss.
llama.cpp quant command:
./llama-quantize --tensor-type "blk.\d+.attn_v.weight=q8_0" --tensor-type "blk.\d+.attn_k.weight=q6_k" --tensor-type "token_embd.weight=q8_0" qwen3-4b.bf16.gguf qwen3-4b.q4_k_m_custom.gguf Q4_K_M
- Library
- gguf
- License
- apache-2.0
- Created
- Aug 31, 2026
- Updated
- Aug 31, 2026