Zamba2-7B-Instruct-v2-1BP

Zamba2-7B-Instruct-v2 — 1BP format

Zyphra/Zamba2-7B-Instruct-v2 (Apache-2.0) converted to 1BP — the single-file model format used by 1bit.systems's inference engine.

1BP packs everything a loader needs into one memory-mappable file: a 256-byte header with model config, a variable-length tensor index, and Q4NX-tiled (32x256, 4-bit groups of 32 with bf16 scale/zero-point) weight data.

Architecture: Mamba2-hybrid state-space (attention interleaved every 6th layer)

Converted from zamba2-7b-instruct-v2-q8_0.gguf (Q8_0/BF16 source, not a 4-bit GGUF, to avoid compounding quantization error through a second 4-bit pass). Structurally verified against the source GGUF (exact tensor/element count match) and numerically verified (dequantized values within expected 4-bit quantization tolerance of the reference).

Usage

git clone https://github.com/bong-water-water-bong/1bit-systems
cd 1bit-systems
cmake -B build/engine/npu -S engine/npu -DCMAKE_BUILD_TYPE=Release   # ONEBP_SUPPORT is on by default
cmake --build build/engine/npu --target npu_engine_universal -j8
./build/engine/npu/npu_engine_universal /path/to/Zamba2-7B-Instruct-v2.1bp --model-tag zamba2-7b-instruct-v2

Attribution

Base model: Zyphra/Zamba2-7B-Instruct-v2, Apache-2.0. This repo contains a re-quantized (Q4NX, 4-bit) derivative in a different on-disk format; the original license terms apply to the underlying weights.