bitnet-b1.58-2B-4T-tq2_0-gguf

BitNet b1.58 2B4T — TQ2_0 GGUF (mainline llama.cpp)

TQ2_0 conversion of Microsoft's BitNet b1.58 2B4T (MIT), converted on mainline llama.cpp with synapticode-ai/llama.cpp (two small converter patches; conversion walkthrough in docs/tq2_0-anatomy.md).

bf16-equivalent quality at 2.06 bits per weight.

File

Conversion is deterministic: a fresh bf16 → TQ2_0 conversion on the pinned tree reproduces this file byte-for-byte.

Quality (perplexity, measured)

llama-perplexity, WikiText-2 raw test set (corpus sha256 bbf94c53a05abe9e…), CPU (-ngl 0), substrate = synapticode-ai/llama.cpp v0.1.0 (table) — re-measured on v0.1.0-relu2, see the dated note. Same corpus, commands, and binary for both columns.

Dated note, 2026-09-10. The PPL gate binary used for the figures in this table predated the 23 Aug 2026 relu2 fix: it ran the 2B4T FFN as SiLU (build-info 010014c29). Re-measured on tag v0.1.0-relu2 (build-info b936d9918), same corpus (WikiText-2 test, sha256 bbf94c53…) and contexts: TQ2_0 c512 14.4815 ± 0.110, c2048 12.5566 ± 0.092; bf16 through the same engine c512 14.5066 ± 0.110, c2048 12.5856 ± 0.092. The conversion delta is parity-class — within the per-leg interval at both contexts, sign not interpreted. Server rows, grounding results and comparator rows stand; the absolute 2B4T PPL rows in the table below are retired.

2B4T is QAT-trained natively ternary; TQ2_0 is its native alphabet. Absolute values reflect an instruction-tuned model scored on raw text and are comparable only within this methodology.

Throughput (measured, hardware disclosed)

Mac mini M4 Pro, CPU-only (-ngl 0; TQ2_0 has no Metal path): 237–279 tok/s prompt eval, 89–112 tok/s generation.

Run

./build/bin/llama-cli -m bitnet-2b4t-tq2_0.gguf -ngl 0 -p "hello"

Source model © Microsoft, MIT. Conversion: Synapticode · [email protected]