Downloads · 30 days
104
13% of all-time downloads
ewchampion/Ternary-Bonsai-4B-TQ2_0-GGUF
Ternary-Bonsai-4B-TQ2_0-GGUF is a machine learning model from ewchampion. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Requantized from prism-ml/Ternary-Bonsai-4B-gguf's F16 baseline to GGML's native TQ20 ternary type (~2.06 bits/weight), using stock llama.cpp (tag b9498) — not PrismML's custom fork/packing.
Downloads · 30 days
104
13% of all-time downloads
All-time downloads
779
Public
Repo size
1.3 GB
Likes
0
Public
Click a slice to open those files.
.gguf1.3 GB · 100%
From the Hugging Face model README
Requantized from prism-ml/Ternary-Bonsai-4B-gguf's F16 baseline to GGML's native TQ2_0 ternary type (~2.06 bits/weight), using stock llama.cpp (tag b9498) — not PrismML's custom fork/packing.
Sanity-checked via llama-server's OpenAI-compatible endpoint on CPU (Metal has no TQ1_0/TQ2_0 matmul kernels at this llama.cpp version — this model must run with -ngl 0 / CPU-only until upstream adds Metal support):
<tool_call> format): FAIL, reproducible across 3 trials at temp=0. The model emits the correct tool-call JSON payload but consistently drops the opening <tool_call> tag (only emits the closing </tool_call>), breaking llama.cpp's native tool-call parser (finish_reason: stop instead of tool_calls). Do not enable native/strict tool-call parsing for this specific quantized model without a more lenient parser or further investigation.License: Apache 2.0 (carried over from base model).