zamba2-7b-instruct-hxq

Zamba2-7B-Instruct-HXQ

2.0x smaller from BF16. 81-layer hybrid Mamba2+Transformer. Largest HXQ hybrid model.

Zamba2-7B-Instruct compressed from 14.7 GB (BF16) to 7.5 GB. 213 linear layers compressed, 573 exact tensors preserved. No calibration data. Just pip install and from_pretrained().

Install and Run

pip install "helix-substrate[hf]"

import helix_substrate  # registers the HXQ quantizer with HuggingFace
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("EchoLabs33/zamba2-7b-instruct-hxq")
tokenizer = AutoTokenizer.from_pretrained("EchoLabs33/zamba2-7b-instruct-hxq")

inputs = tokenizer("Explain the theory of relativity in simple terms:", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

That's it. import helix_substrate registers the quantizer. from_pretrained() handles the rest automatically.

Benchmark

Good to Know

• GPU recommended — 7.5 GB requires 10+ GB VRAM. Use device_map="auto" for multi-GPU.
• Fine-tunable via LoRA — compressed weights remain frozen, but LoRA adapters attach to each HelixLinear layer via HelixLinearSTE. See helix-substrate for training infrastructure.
• Requires helix-substrate — the quantizer is not built into transformers. You need pip install "helix-substrate[hf]".
• Requires transformers >= 4.45 — for Zamba2 architecture support.
• mamba-ssm recommended — without it, falls back to a slower sequential code path.
• PPL pending — requires cloud GPU eval (model doesn't fit on 4 GB T2000).

What is HelixCode?

HelixCode is a universal weight compression codec based on vector quantization:

• Each weight matrix is replaced by a 256-entry codebook (float32) + uint8 index matrix + optional sidecar corrections for outlier values
• The compressed form is the executable — HelixLinear performs codebook[indices] @ x directly, no decompression step
• Works on any nn.Linear regardless of architecture (Transformer, Mamba, MLP, CNN)
• No calibration data required — unlike GPTQ/AWQ, codebooks are fit from the weights alone

How It Works

1. import helix_substrate registers the hxq quantizer with HuggingFace
2. from_pretrained() reads quantization_config.quant_method = "hxq" from config.json
3. The quantizer replaces 213 nn.Linear modules with HelixLinear shells before weight loading
4. Safetensors populates the codebook, indices, and sidecar buffers directly
5. The model runs in compressed form — no decompression needed

Architecture Details

Zamba2-7B-Instruct is a hybrid architecture with:

• 81 total layers (Mamba2 + shared Transformer hybrid)
• hidden_size=3584, attention_hidden_size=7168, 32 attention heads
• mamba_d_state=64, mamba_d_conv=4
• vocab_size=32000

213 linear layers compressed (162 Mamba projections, 38 attention/MLP, 26 LoRA adapters). Normalization layers, embeddings, conv1d, and Mamba-specific parameters (A_log, D, dt_bias) are stored at full precision.

Compression Receipt

Compressed modules:  213
Exact tensors:       573  (norms, embeddings, conv1d, A_log, D, dt_bias, LoRA)
Skip tensors:        243  (from original model)
Total keys:          1425
Dense size:          14.7 GB (BF16)
Compressed size:     7.5 GB
Compression ratio:   2.0x
PPL delta:           pending (cloud GPU eval)
Gate 1:              PASS (structural validation + SHA256)

Companion Models

Same codec, same pip install, multiple architectures:

Citation

@software{helix_substrate_2026,
  title={Helix Substrate: Universal Weight Compression via HelixCode},
  author={EchoLabs},
  year={2026},
  url={https://github.com/echo313unfolding/helix-substrate}
}

License

Apache 2.0 (inherited from Zyphra/Zamba2-7B-instruct).