Downloads · 30 days
16
18% of all-time downloads
FardoX/sasori-qwen3-14b-tq2p-q4k6
sasori-qwen3-14b-tq2p-q4k6 is a text generation model from FardoX. Use it when you need the model to write or continue text. The card lists the license as cc-by-4.0.
Ternary quantization of Qwen3-14B using Sasori TQ2P (dual trit-plane) with 6-bit compressed scales (q4k6), error feedback (EF/GPTQ), and no rotation (rotA dropped).
Downloads · 30 days
16
18% of all-time downloads
All-time downloads
89
Public
Repo size
10.8 GB
Likes
0
Public
Click a slice to open those files.
.gguf10.8 GB · 100%
From the Hugging Face model README
Ternary quantization of Qwen3-14B using Sasori TQ2P (dual trit-plane) with 6-bit compressed scales (q4k6), error feedback (EF/GPTQ), and no rotation (rotA dropped).
A large-scale campaign (genscale/k2ef, 9 commits, 6 model families × K2/K3, GSM8K + MATH-500, McNemar paired exact test, Jul 2026) proved that rotA (QuaRot-class Hadamard rotation) never helps: it is null on MATH-500 (McNemar ns, 3/3 seeds, p≥0.49) and task/family-dependent on GSM8K (worsens Llama ~6pp). The gain previously credited to the ship-point was entirely from EF (error feedback). Dropping rotA saves compute with no quality loss.
| Property | Value |
|---|---|
| Base model | Qwen3-14B |
| Format | TQ2P (dual trit-plane, K=2) |
| Scale storage | q4k6 (6-bit sub-scale + fp16 super-scale per 256) |
| Bits per weight (linears) | 4.625 |
| EF coverage | 280/280 linear layers (100%) |
| Embeddings | F16 (not quantized) |
| GGUF size | ~11 GB |
| GGML type | id 59 (TQ2P_Q4K6) |
| Calibration | 512 C4 sequences, single seed |
-ngl 99 GPU)| Task | N | Seeds | Strict/Exact | Flexible/Verify |
|---|---|---|---|---|
| GSM8K 5-shot | 200 | 1 | 0.93 | 0.94 |
| MATH-500 4-shot | 200 | 3 | 0.500 ± 0.013 | 0.615 ± 0.000 (math_verify) |
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp && git checkout 6f3a9f3
# Apply the Sasori patch (see https://github.com/Adriel007/sasori)
git apply sasori/llama-cpp/tq-llamacpp-b9509.patch
cmake -DGGML_NATIVE=OFF -DGGML_AVX2=ON -DGGML_CUDA=ON -B build
cmake --build build -j$(nproc) --target llama-server llama-cli
# CPU only
llama-cli -m qwen3-14b-tq2p-q4k6.gguf -p "Hello" -n 50
# GPU offload (much faster)
llama-server -m qwen3-14b-tq2p-q4k6.gguf -c 16384 -ngl 99 --port 8080
Sasori: ternary quantization toolkit for CPU-first deployment. FardoX/sasori (github.com/Adriel007/sasori).
Full results and methodology: see the Sasori PhD repository.