Downloads · 30 days
154
34% of all-time downloads
badtheorylabs/Tinfield-1-Q8_0-GGUF
Tinfield-1-Q8_0-GGUF is a machine learning model from badtheorylabs. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
Tinfield 1 at 8 bits in GGUF, 188 GB across six split parts. This is the near-lossless reference build and the file to quantize from if you want to make your own.
Downloads · 30 days
154
34% of all-time downloads
All-time downloads
456
Public
Repo size
188 GB
Likes
4
Public
Click a slice to open those files.
.gguf188 GB · 100%
From the Hugging Face model README
Tinfield 1 at 8 bits in GGUF, 188 GB across six split parts. This is the near-lossless reference build and the file to quantize from if you want to make your own.
Every quality figure published for the smaller builds is measured as KL divergence against this file.
Quantizing starts from a GGUF, and converting the BF16 weights
yourself means downloading 354 GB and running a multi-hour conversion that produces exactly this file. Start
here instead and go straight to llama-quantize.
If you just want to run the model, take Compact at 72 GB or Mini at 61 GB. Both fit a 64 GB machine. This build is 188 GB and is not meant for everyday serving.
One trap will cost you a whole attempt. llama.cpp's K-quants use 256-value super-blocks, and a tensor whose row width is not a multiple of 256 silently falls back to something much larger. This model has three row widths:
| Tensor | Row width | Use |
|---|---|---|
ffn_gate_exps, ffn_up_exps | 2560 | any K-quant |
ffn_down_exps | 640 | a 32-value block type: iq4_nl, q4_0, q5_1, q8_0 |
per_layer_token_embd (n-gram table) | 160 | same, 32-value blocks only |
token_embd, output | 248320 | any K-quant |
The n-gram table is 51B parameters on its own, so leaving it to fall back costs about 25 GB of file size.
A 4-bit build following the pattern Unsloth and NVIDIA both use on this architecture, quantize the routed experts and leave the rest alone:
llama-quantize --allow-requantize --imatrix imatrix.gguf --keep-split \
--tensor-type "ffn_(gate|up)_exps=q4_K" \
--tensor-type "ffn_down_exps=q5_1" \
--tensor-type "per_layer_token_embd=iq4_nl" \
tinfield-1-Q8_0-00001-of-00006.gguf out.gguf Q8_0 32
The final argument is Q8_0, not Q4_K_M. Using Q8_0 as the base type means anything you do not name
explicitly stays at 8 bits, which is what keeps attention, the routers and the embeddings intact.
Run with --dry-run first and read the per-tensor plan. That is how you catch a fallback before spending
the hours.
You do not need to build an importance matrix, which is the step that needs the model in VRAM. Unsloth
publishes one for the base at unsloth/Qwen3.8-Flash-Next-GGUF. Our fine-tune never touched the experts or
the n-gram table, so those tensors are bit-identical to the base and its statistics apply directly.
llama-server -m tinfield-1-Q8_0-00001-of-00006.gguf --jinja \
--temp 1.0 --top-p 0.95 --top-k 20 \
-fa on -ctk q8_0 -ctv q4_0 -ngl 999 -c 131072
llama.cpp must be at commit 972d2313 or later; older builds do not know this architecture and fail to
load rather than warning. --jinja is required or the chat template is ignored and tool calling breaks.
| Build | Size | By |
|---|---|---|
| Tinfield 1 Q4_K_XL | 111 GB, 5.03 bpw | needmorevramog |
Not built or measured by us. Quantizing your own is covered in QUANTIZING.md.
Qwen Community License 1.0, following the base model.