Downloads · 30 days
41
1% of all-time downloads
maczzinatui/Qwen3.6-27B-MTP-TQ3_4S-GGUF
Qwen3.6-27B-MTP-TQ3_4S-GGUF is a machine learning model from maczzinatui. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
TQ34S quant of Qwen/Qwen3.6-27B (Apache 2.0), produced via turbo-tan's llama.cpp-tq3 CUDA fork. Benchmarked on a Blackwell RTX 5060 Ti (16 GB). Mean AEON score: 0.525 (120 scored / 150 cases, 2048 budget).
Downloads · 30 days
41
1% of all-time downloads
All-time downloads
5.7K
Public
Repo size
14.2 GB
Likes
0
Public
Click a slice to open those files.
.gguf14.2 GB · 100%
From the Hugging Face model README
TQ3_4S quant of Qwen/Qwen3.6-27B (Apache 2.0), produced via turbo-tan's llama.cpp-tq3 CUDA fork. Benchmarked on a Blackwell RTX 5060 Ti (16 GB). Mean AEON score: 0.525 (120 scored / 150 cases, 2048 budget).
| File | Size | Quant | BPW |
|---|---|---|---|
Qwen3.6-27B-MTP-TQ3_4S.gguf | 14.2 GB | TQ3_4S (turbo-tan) | ~3.8 bpw |
TQ3_4S (TurboQuant 3-bit 4-state) is a custom weight format unique to turbo-tan/llama.cpp-tq3. Stock llama.cpp and nixpkgs llama-cpp will exit with unknown quantization at load time. Use the llama-server/llama-cli from the tq3 fork.
These numbers are a light baseline, not a thorough TQ3 evaluation. The mesh's bench framework is built for production agent workload regression-detection on the local stack, not for the kind of multi-axis sweep that upstream quant maintainers typically publish. Specifically:
What this IS good for: a quick signal that the quant (a) loads, (b) runs at sane throughput, (c) produces coherent output across math, instruction, reasoning, and coding. What this is NOT good for: claiming "this is the best quant of this model," reproducing academic benchmark results, or substituting for upstream's validation work.
For a rigorous view, see Qwen/Qwen3.6-27B (parent model) and turbo-tan/llama.cpp-tq3 (quantizer). Raw bench reports are attached as BENCH-*.md files in this repo.
Benchmarked on Blackwell RTX 5060 Ti 16 GB, full suite.
| Category | Mean | N |
|---|---|---|
| Overall | 0.525 | 120 |
| coding | 0.933 | 30 |
| math | 0.467 | 30 |
| reasoning | 0.367 | 30 |
| instruction | 0.333 | 30 |
| Difficulty | Mean | N |
|---|---|---|
| easy | 1.000 | 8 |
| medium | 0.833 | 12 |
| hard | 0.700 | 20 |
| expert | 0.500 | 32 |
| frontier | 0.312 | 48 |
Profile: Strong on coding (0.933), weak on instruction-following (0.333) and reasoning (0.367). Typical Qwen3.6-TQ3 profile — excels at structured code tasks, struggles on instruction-heavy and frontier-difficulty cases.
The companion ROCmFP4_FAST quant for AMD RDNA4 scores 0.558 on the same suite. Cross-format parity is within ~3pp on most categories; the AMD arm benefits from a higher coding score but the overall profile is similar.
# Build turbo-tan's tq3 fork
git clone https://github.com/turbo-tan/llama.cpp-tq3
cd llama.cpp-tq3
mkdir build && cd build
cmake .. -DGGML_CUDA=ON -DCUDA_DOCKER_ARCH=sm_120
make -j$(nproc)
# Serve with production-equivalent flags
llama-server \
-m Qwen3.6-27B-MTP-TQ3_4S.gguf \
--host 0.0.0.0 --port 8081 \
-ngl 99 -c 131072 -t 12 \
-ctk q4_0 -ctv q4_0 \
-np 1 --batch-size 512 --ubatch-size 128 \
--jinja --metrics -rea off
# Requires the tq3 fork and the BF16 source GGUF
llama-quantize --allow-requantize Qwen3.6-27B-BF16.gguf \
Qwen3.6-27B-MTP-TQ3_4S.gguf TQ3_4S
| File | Description |
|---|---|
Qwen3.6-27B-MTP-TQ3_4S.gguf | The quantized model (LFS-tracked) |
README.md | This model card |
BENCH-aeon-full-suite.md | AEON bench results (150 cases, 120 scored) |