MiniCPM5-2B-Hadamard-GSQ

MiniCPM5-2B FQuant adaptive research release

FQuant major update — 2026-09-12. This model is backed by the current FQuant calibration-aware pipeline, including deterministic Hadamard/Rademacher transforms, activation-weighted scales, weighted randomized SVD, sensitivity- guided mixed precision, architecture-aware loading, and numerical runtime checks. The release contains 3 weight shards, 561 indexed tensors, and reproducibility metadata. FQuant core validation passes 12 tests and the MiniCPM numerical/integration suite passes 6 tests.

This is a substantial engineering update to the FQuant model path rather than a metadata-only repack. It is still an experimental quality-oriented release: the current artifact is approximately 4.02 GiB, broad benchmark accuracy, long-context recall, and optimized llama.cpp throughput remain under evaluation.

This is an experimental calibration-aware release of openbmb/MiniCPM5-2B. It is intended for PyTorch and Transformers evaluation with the custom files in this directory. The artifact was built with deterministic Rademacher signs, block Walsh-Hadamard rotations, activation-weighted scale selection, weighted randomized SVD, robust L0-L3 layer diagnostics, and a mixed-precision policy.

The current quality probe stores 234 high-sensitivity projections in BF16 and 60 remaining MLP gate/up projections in groupwise INT8. The serialized model is approximately 4.02 GiB versus 4.69 GiB for the BF16 source, a 1.16x size ratio. This is a quality-oriented intermediate point; it deliberately trades some compression for lower logit drift. The release report records the exact layer rank map and calibration moments.

Update — 2026-09-12

This local release was refreshed after a numerical review of the loader and KV-BSS path. The update validates GQA head divisibility and attention-mask shapes, keeps fully masked attention rows finite, initializes scratch models deterministically, and adds cache-versus-uncached logit parity checks. The current repository test suite passes 7 tests. KV-BSS now computes attention scores in FP32 and contains non-finite query/key/value inputs at the attention boundary, so a malformed score cannot crash generation. The model artifact itself is unchanged by these runtime checks; its calibration report remains the source of truth for the measured quality probe above.

Load

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "./MiniCPM5-2B-Hadamard-GSQ-adaptive",
    dtype=torch.bfloat16,
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    "./MiniCPM5-2B-Hadamard-GSQ-adaptive",
    trust_remote_code=True,
)

The custom loader is required because the weights use FQuant tensor names (weight_bf16, qweight_int8, scales_int8, qweight_packed, scales, and low-rank factors) rather than a standard Transformers linear weight.

Validation snapshot

On three fixed English, Chinese, and Russian prompts stored in adaptive_bifurcation_report.json, averaged by valid token position and compared with the raw BF16 model and the previous GSQ release:

This is a short three-prompt probe, not a benchmark or a claim of general quality improvement. Run a held-out task evaluation before treating the artifact as a default release.

llama.cpp status

Stock llama.cpp cannot load this artifact directly yet. A compatible path will need a converter and a small backend extension for the paired input rotation, mixed BF16/INT8/INT4 tensors, and low-rank residual factors. Folding the whole operation into ordinary GGUF weights would remove the compression advantage, so the llama.cpp adapter is intentionally deferred until the PyTorch quality tests are complete.